Brent Warner 0:00
We’re seeing more and more AI bots popping into our online meetings. What do we need to know? This is the higher ed tech podcast season 6, Episode 13. You

Tim Van Norman 0:23
Welcome to today’s higher ed tech Podcast. I’m Tim VanNorman, the Instructional Technologist at Irvine Valley College and Adjunct Professor of Business at Cypress College.

Brent Warner 0:33
And I’m Brent Warner, Professor of ESL here at IVC. We both enjoy integrating technology into the classroom, which is what this show is all about.

Tim Van Norman 0:41
Welcome. We’re glad you’re here with us.

Brent Warner 0:44
So Tim, there’s one thing that’s not going on in this meeting between you and me right now, which is that there is no bot together. It’s just you and me. Amazing (laughter)

Tim Van Norman 0:55
Well, actually, we’re recording it, but yes, yeah, – We can talk about that. We can talk about that later. But yes, we, neither of us, have AI bots automatically joining meetings, which I am very thankful for.

Brent Warner 1:11
Alright, so so we know that the bots thing, so more and more of us are seeing these things pop up, right? It’s like, oh, hey, there’s Tim Van Norman, and there’s also Tim Van Norman, or sometimes in a little less tricky ones, like it’ll say something like, otter is in your meeting, or whatever else it is. But this is kind of what we’re talking about, is this idea of now, and we’ll get it. We’ll get into it. But more and more people are sending bots, AI bots, into their meetings for all sorts of purposes, many good and potentially useful and many possibly problematic, right? But we’ll kind of work through the good, the bad and the ugly of it, and I think there’s a lot to get into. Tim, so are you ready? Absolutely

Tim Van Norman 1:57
All right, so in a whole lot of meetings lately, as you said, bots are showing up all over the place, and there’s some really cool things. By the way, I’ve been one of those people who in all of my meetings, I do have a bot when I set up the meeting that sends me a message at the end with a summary of the meeting. Okay? And if, like, I’m in a meeting with you and we’re talking about it, then I would maybe send you the meeting, because we both were in it. Okay, okay, okay, I love it for that type of thing. Hey, it’s a quick summary of everything that went on, but we’re both. We both were there, so we both know what is right and what is wrong.

Brent Warner 2:43
Yes, as long as we check it quickly,

Tim Van Norman 2:45
Yes, as long as we remember exactly. And so a lot of what we’re going to talk about are the fears that people have. And by the way, I think many of them are very justified, very and and what some of the things that we’ve actually seen in our meetings when people have used these bots. So I love the fact that we take something like an otter, okay, and yes, we’re going to use some brands. We don’t, we’re we don’t use these brands, necessarily. Some of them we do. I personally love otter. I think otter is a great tool, but we’re going to give some specific brands and talk about them in here. But there are lots. There are. This is, you know, a couple of the many. So one of the things that it does is, during the meeting, it’ll pop up, and somebody can actually change the notes and massage the information during the meeting, which is great. We use it, for instance, at Irvine Valley, we use it for our for note taking. Students who have the note taking accommodation, they don’t have to wait for a student’s information to come to them before they get the notes. They literally can have it live in the class. Wonderful tool. We’ve been using it for several years. Really makes a huge difference for suit certain students. It’s great that way.

Brent Warner 4:12
Yeah. And can I add something to this? One’s live, yeah. So there’s something a friend of mine recently, so I was doing a presentation, and he’s like, Hey, I’m going to record this with AI to can you do me a favor, and when you’re presenting, can you first lay out your agenda, and then, and then afterwards, as you’re going through each piece, clearly, speak out the matching language to the agenda pieces, right? Because he wanted me to say, first I’m going to talk about a, then I’m going to talk about B, finally, I’m going to talk about C. All right, first up a, let me, let me talk about a, right? And so he’s like, he’s like, you might not normally talk like that, but when you say it, and the AI processes it, then it’s going to capture that information. Or know what is? Supposed to be looking for when it’s going through everything. And so it’s really interesting to me. I was like, Okay, I don’t really mind doing it here. And of course, you know, I mean, that’s useful and better for that particular output. But it’s also interesting to say, like, Okay, now we’re trying to rearrange how we might actually communicate just in a regular meeting, so that the bot can pick up and and present the information most accurately later on. I thought it was really interesting to think about

Tim Van Norman 5:27
Well, and some of it’s useful anyway, right? But that’s not the way. It depends on how the meeting is structured. So do you have an agenda? If you’ve got an agenda, you’re kind of doing that, although you don’t go ahead of time necessarily, and read the agenda aloud and then go back through the agenda, but it is kind of structured in that way. But a lot of meetings that you and I have, for instance, they’ll hit 20 topics, and we only knew three of them at the beginning of the meeting, so it’s a little harder to do that, and that’s actually one of the concerns that I often have, is when I see these, if somebody is sending the bot and they’re not showing up as well, they don’t really know if the items that are to do were accomplished or not, right? So I’ve seen a number of cases where, oh, Tim is going to present this, Tim is going to talk about this. Tim is going to and if the bot had paid any attention, it would realize that I, that’s what I did, right?

Brent Warner 6:27
So I so you say, Hey, I’m going to send an email about this, and you do it right now, right? And then, so, in your meaning, right? It’s happening in the moment, but in the bots interpretation of it, it’s, that’s an action item for you to do later, right? Exactly. Yeah. Okay. So, again, so, but these could be misunderstanding. I guess this goes into the hallucination thing with the bots, right? Or the or the, you know, the the wrong interpretation of the information. So again, you want to scan through everything at the end and make sure that what it’s saying is actually what you’re what you were communicating in the meeting. But also, I should point out, Tim, we’ve done this once or twice, and it’s actually really nice to do, because then you really refresh yourself. I mean, it’s a very good study guide. It’s like, it’s like, checking your notes right after the meeting. We tell our students to do this all the time. Like, you take notes, leave the classroom, sit down on a bench, review for five minutes. You’ll be way better off than if you just, like, let it sit there for a day or two and maybe come back to parts of it,

Tim Van Norman 7:24
Absolutely. And and the reality is that it most of the time. What it does take good notes on. It takes really good notes on. And so it’s really easy to trust it when you don’t know for sure. And that’s, that’s why, you know, some of us have a little bit of a fear of people sending the bots and just assuming that it must be accurate.

Brent Warner 7:51
Well, those bots do talk with a lot of that – I mean, from day one – that authority, the authoritative voice, where they where they talk like that’s been an issue all the way through. So, yeah, okay, good.

Tim Van Norman 8:02
Absolutely. So there’s a lot of different reasons for using these AI summaries for people who need to take minutes. It’s been a godsend. They don’t have to write down every single you know, what every single person said and all of that stuff that said. The flip side, if they’re taking minutes, do they really say Oh? Tim Van Norman said this. Brent Warner said this, you know, and name each person and exactly what they said about every topic, probably not to that degree, especially if it’s an ideation time rather than something that, oh, Tim presented this, yeah, yeah. And often the bot doesn’t tell the difference. And so it might be something that you’re you’re throwing around ideas, or maybe even you disagree with it, and you’re saying, Well, some people say such and such, and all of a sudden comes back and says, Tim Van Norman said this. Well, wait, that’s not what was meant by the context. So just like we say, with all things, AI, you’re responsible for the output.

Brent Warner 9:09
Yeah, and actually, I want to follow up on that just a little bit too, because actually, I don’t know this, and this is just a question, but on the back end of some of these, Tim is, is it possible? So if you’re if your responsibility, let’s say you’re an administrative assistant. Administrative Assistant, and your job is just to be taking the notes. Is, does it have an option where it’s producing the notes live and you can edit them live as they’re going or does it all just kind of come through at the very end, after some processing time?

Tim Van Norman 9:39
So there’s a couple of different ways it happens. Otter, for instance, has it where you can actually edit it, live, okay, put breaks in and stuff like that, and so that, which is one of the things I really do like, is that you don’t have to go back through it later and correct that.

Brent Warner 9:55
That’s cool. Yeah. So if you had to, I also point out, if you had with your students, you. I gotta know, a lot of people are not going to want to do this, but it’s just maybe a fun little idea, which is, what if your students got access to the notes live and they were annotating the notes and changing them as they came through to show their own understanding? That would be kind of a cool way to say, like, hey, let’s do a, you know, verification system with you guys in this

Tim Van Norman 10:20
Absolutely and, and the beautiful part is, you know, and we’ve got faculty who I believe are actually doing exactly that in Otter, because it’ll send that out to the class, and then the class takes the notes, and then at the end, they can then download it, and they have the notes that everybody has worked on throughout the class available to them. So I think it’s a great tool that way. And one of the reasons I really do like that type of thing,

Brent Warner 10:46
You can even just make like, a quick five minute activity at the end of class or at the end of a lecture and say, hey, it’s everybody’s job to go clean this up before you start. You know, using it as your actual notes.

Tim Van Norman 10:57
Absolutely, absolutely.

Brent Warner 10:58
Okay. Cool.

Tim Van Norman 11:01
So that said also, it’s one thing about a class, but not necessarily the whole class should be recorded. Yes, okay, you may have times when you’re you stop the recording. Everybody else leaves, and now you’re having a one on one meeting with a student, and the bot hasn’t left.

Brent Warner 11:24
This is problematic for several reasons too.

Tim Van Norman 11:25
Oh, lots and lots of reasons. Okay, but also, most of the bots actually do an audio recording, yes, on top of the the transcript so that, so that somebody can go in and and actually hear it and make sure it’s accurate, but… (laughter)

Brent Warner 11:45
Start imagining what these problems are. So one of the ones that I come up against quite often, Tim is like, hey, we finished the meeting. We want to recap, or maybe respond to, some of the challenges that came up in the meeting. And so if that recording, even if the meeting is kind of over and everybody’s out, it’s still possible that someone could go click and watch the rest of the live updates that, you know, whatever the bot is transcribing. So they’re they seem like they’re not in the meeting, but they could be watching that transcription. And so, you know, I hope that most of the time we’re collegial, but like, sometimes I’ve been in things and I’m like, Oh my gosh, can you believe so and so said such and such, right? And, you know, I mean, you could get yourself into some hot water with something like that, right? And so I think there’s, there’s quite a few little things that we might not think about on first step and go, like, Wait a second. If someone’s savvy, or they know what they’re doing, there could really be some issues going on,

Tim Van Norman 12:41
Right. There’s also certain meetings that cannot be recorded for all kinds of reasons.

Brent Warner 12:48
Yeah, so what? What might some of those reasons be that people are may well always pay attention to take,

Tim Van Norman 12:54
for instance, the academic senate meetings, they’re not supposed to be recorded. Yeah, there’s lots going on there. Um, you know, there’s so there’s very specific meetings that, for various reasons, should never be recorded. And now, all of a sudden, a bot goes in and it’s recording it, right? And and by the way, the host might not have know anything about it. Yeah, it’s not necessarily that the host even has a clue that it’s going on. It’s not necessary. It’s not theirs, it’s somebody else’s. And yet somebody is in recording that could be potentially a problem, because you don’t know what’s going to happen with that recording.

Brent Warner 13:39
Yeah, so that’s one, another one that I’ve heard come up here, Tim, I think, I think you’ve heard this one too, is people just not going to meetings and sending the bot in, quote, unquote, in their place, right? So what could be some problems? I mean, I think, well, we’ll assume that some of them might be obvious, but like, you know, we might not think about it. It’s a it’s a newish thing, right? So what could be problems with this?

Tim Van Norman 14:07
Well, as we’ve talked about the accuracy, do they really know what actually happened, right? Are they listening to the complete recording to verify that that information is accurate? If they did, if they didn’t have time to show up for the meeting. It’s highly unlikely they’re going to have time to review that meeting audio to make sure that that the bot was accurate. There’s a reason that they did it. No, I’m in four meetings at the same time. I’m not going to I’m not going to do that, and it means that I think back at there was an old, old movie, real genius, where this kid shows up for class, and finally, by the end of the semester, it’s this kid and the the teacher, it. The professor is a, basically a cassette player. And all of the students are cassette players in the classroom. And this one kid, you know, and that’s kind of what it kind of feels like. It’s it can get to, yeah, I know that’s an extreme, but if you need somebody’s opinion, they they need to be there to give an opinion in the meeting.

Brent Warner 15:26
Yeah, it’s funny. It’s funny. You said that. I actually, I put that, that clip into a recent presentation, because that’s the kind of the joke where it’s like the it’s a little montage scene, right? Where the the main kid, the main character in the show in the movie, walks into class, all the students are there, the teachers, they’re teaching, right? And then slowly, like he keeps going in, and slowly, fewer and fewer students are there, and it’s just tape recorders going on and like, listening to it. And then the last scene is he walks in, and the teacher is also replaced now with a tape recorder playing, and all the students are listening to it, right? So we’re just kind of in this world where nobody’s actually interacting or doing anything with each other, but I like your point to it too, Tim, which is the without the interaction, then there’s no real productivity, right? I mean, it’s just like information give, information take, and that’s it. But the but without, like when we talk to each other, when we challenge ideas, when we’d ask questions, that’s when the real learning and the real productivity starts coming out. And that will get lost. The more and more people just say, well, I’ll just have, you know, some other thing, do it. And even though AI can theoretically respond to things like, at what point is it going to start just make I mean, pretty soon it’ll start just making up options, because we didn’t plan to, you know, set it up with information to respond to certain points that you know might be abstract or unusual.

Tim Van Norman 16:51
Well. And the reality is, if you’re not going to be in it, why can’t this just be an email,

Brent Warner 17:01
the lifelong question, right? Another meeting that should have been an email, right? Yes, yeah, yeah, for sure. Okay, so what are some possible solutions here? Tim, what are some things we can do?

Tim Van Norman 17:13
One is, when you are using one of those otter read AI, something like that, if you’ve ever created an account, you probably are using it, whether you realize it or not. So number one, figure out if you’re using it.

Brent Warner 17:28
How would, you know?

Tim Van Norman 17:31
Ask people when you’re joining a meeting? Hmm, did it? Did an AI? Join the meeting? You know, take a look at that type of thing. Join the meeting early and see if your name comes up as an AI. It’s amazing how many people have said, Oh, no, I didn’t do it and I’m like, it says Brent Warner, a read, AI, yeah, you know, or something like that. No, I haven’t ever seen it from your I’m using you as an example. But yes, exactly, and, and so other but that might join before you, and what happens is it puts a often, it’ll put a message in the chat. Anybody who joins after that doesn’t ever see that message,

Brent Warner 18:14
Yeah, oh yeah, because the the chat history is not kept until you join the meeting, right? Exactly.

Tim Van Norman 18:20
So. So if you join early, then you would see that in the chat, and that that’s kind of a nice thing to see. But also, if you’re using those, for instance, I use otter. I love otter. I’ve, on purpose several times gone back into the settings for otter. Every time they do an update that I realize that they’ve done an update, I go back into the settings, and I look to see to make sure it’s not connected to my zoom, to teams, to whatever the system is that I’m using. And do that in Otter, do that in read and stuff. And you you may find that, oh, hey, it actually is using it, yeah, and, and you can turn it off. There’s nothing wrong with those packages. It’s just I feel almost violated sometimes when, when the person’s not there, but, or I don’t know if the person’s there, yeah.

Brent Warner 19:15
I mean, it’s kind of that like that. You know, you’re not quite important enough for me to take my own time out of vibe that comes across, I don’t, I don’t know that that’s the intentional meaning, but that does come across sometimes when when these come into the meeting. So and then the other part Tim, I want to talk about too, is like, one of the questions that came up for us on the AI Task Force is, essentially, should these be allowed in our meetings, right? Or, like, is it okay for these things to come in? And I think that’s going to come down to, like, a discussion with your department or or with whoever you know you’re having that meeting with. Now, like you said, Hey, maybe there’s somewhere you absolutely should not be recording anything, and that’s already a rule, okay? Maybe that needs to be Claro. For example, at the at the beginning of an academic senate meeting, right? Like might say, do not record this or something like that, right? But just to clarify to the actual people who are in the room, that’s not stopping that bot necessarily from being there. But then you might also have, like meetings in other settings where you say, hey, we want all hands on deck. You need to be there, not a bot, right? Something like that, where you’re explicit about the expectations of the meeting and

Tim Van Norman 20:26
understand that most of the bots have an ability, if you’re watching it, if you’re watching for it, they have the ability to either kick them out of zoom, okay, or you can go in and it will say to have me stop, type in stop, or something into the chat, and it’s watching the chat, and then it will do that theoretically. So understand that there are things like that that are going on that you can do in, in zoom, for instance, their their automatic one, you can go in and you can say, okay, stop and delete all of the history. No, and so we’ll delete the history for that, or stop as of now, or whatever. So, so it’s, it’s got some of these tools built in that are, that are pretty nice, but you have to, as the host, you got to pay attention to whether you want this or not, and other people in the meeting may or may not be feel, feel as comfortable, and that’s why you know, as you talked about the AI Task Force, that’s why we don’t come down and say yes or no. Is because it’s, it really is a gray area.

Brent Warner 21:45
Yeah, big time. Okay, so we mentioned these, Tim otter read, a couple of these ones. I do want to briefly. I mean, we kind of talked about how these things can sneak in, but we should talk about read and that, I think there’s a lawsuit with it, right? Essentially, like read.ai, if you’re out there using it, just be aware of what’s going on with it. Is as soon as someone else starts accepting it into their meeting, then basically the way they set it up was, it’s kind of like this tendril, like, yeah, that’s a virus, basically, right? Because then it’s like, oh, you accept it, then we’re going to install it on your thing, and every time you join a meeting, then it’s going to be spread out to all the people who are in the meeting with you. And then it’s going to be automatically activated across all of these things, which is really pretty insidious, right? Like, that’s A, not A, not a setup that I’m comfortable with at all. And, like, it’s not the type of organization that I want to support, but, but, you know, they’re out there. People go, people go, no, well, it’s free. I’m just going to plug it in and get the access to this information. To this information. Well, you know that whole thing about the free, right? If it’s free, then you’re, you’re the, you’re the product, right? You’re paying

Tim Van Norman 22:50
for it one way or another, absolutely, all right, I do understand these are really useful tools, yeah. So one of the things that I’ve seen is somebody will record a meeting and then load it into otter after the fact, and have otter create a summary out of it, and which is a great way of using the tool, yeah, because now they get that summary that they were looking for. Now it doesn’t say as much as, oh, Tim said this, and Brent said that because it’s recorded after the fact, but it can do a really nice job of of summarizing, yeah, and pulling out key points and stuff like that. So it’s not, we’re not against the tool,

Brent Warner 23:34
yeah, yeah. Well, I use otter all the time. I actually use it for the opposite use, which is otter for full transcription. I never use the summary feature, but when we do these recordings, then I do always upload it into otter, and then I want that full transcript. So otter likes to push the the summary first, right? And I’m like, get that out of my face. And what I do is, I’m switching. I’m like, I want the transcript only. And then I go through and I edit the transcript, you know, make sure whatever, like, we always talk about these weird tools with weird spelling, so I try and try and update some of those things or make sure that the language makes sense. But So there, there are often times two choices, right, transcript, again, check for accuracy or summary, right and, and, and again, again. Check for accuracy.

Tim Van Norman 24:20
Always

Brent Warner 24:22
Cool. So I like that. There’s a lot of these uses for them, but, but I definitely see like, hey, there’s some people that hate them, some people that love these things, some people that feel invaded upon. You know, some people feel like it’s saving them tons of time. There’s a lot of different legitimate approaches to all of this, Tim, but I don’t think that this is disappearing anytime soon, right? So, so you have to build some policies or expectations for yourself or for your school or department to make sure that the way that you’re using it there falls in line with your, you know, with the needs and rules of where you work.

Tim Van Norman 25:02
Absolutely.

Thank you for listening today. For more information about this show, please visit our website, at the higher ed tech podcast.com

Brent Warner 25:14
As always, we do want your feedback, so please go to the higher ed tech podcast.com and let us know your thoughts

Tim Van Norman 25:20
for everyone at IVC that’s listening. If you need help with technology questions, please contact IVC technical support. You have questions about technology in your classroom, please stop by a 322, or contact me. Tim Van Norman AT T van norman@ivc.edu

Brent Warner 25:36
And if you want to reach out to me about the show, you can find me on LinkedIn at Brent G Warner,

Tim Van Norman 25:41
I’m Tim Van Norman,

Brent Warner 25:43
and I’m Brent Warner, and we hope this episode has helped you on the road from possibility to actuality. Take care, everybody.