Brent Warner 0:00 This is the HigherEdTech Podcast Season Five, Episode 14: Customizing AI bots in Discord with Professor Zhen Zhai. Tim Van Norman 0:20 Welcome to today's HigherEdTech Podcast. I'm Tim Van Norman, the Instructional Technologist at Irvine Valley College. Brent Warner 0:27 And I'm Brent Warner professor of ESL here at IVC. We both enjoy integrating technology into the classroom, which is what this show is all about. Tim Van Norman 0:35 Welcome. We're glad you're here with us. So Brent, you've missed it. You've been in Japan, and we've had rain. Yeah. Yeah. Brent Warner 0:44 I hope everyone's okay. Tim Van Norman 0:46 Yeah, hopefully, you know, it settles down at some point. But yes, it's been we've had lots of rain. Brent Warner 0:53 So a wet, wet step into the spring, right? Tim Van Norman 0:57 Yeah. So it's been it's been fun. And I'm hoping that it will calm down. But you know, that's what life is right now. Brent Warner 1:07 Yeah, A series of raining on the parade. So, Tim, last time we did, we had Ric on the show, and we were talking about integrating AI using chat GPT. And then you're like, we got to keep this going. We have a part two to this. So. So what are we doing today? Tim Van Norman 1:28 So today, we've got professor of C++ from Irvine Valley College, that's our IVC versus the other one was Imperial Valley College. Zhen is has created a chatbot for her students, based in discord. And so this allows students to get help when they need it on project assignments, and get the help behind that because of what she's put into this Chatbot. So Zhen, welcome to the show. Zhen Zhai 1:59 Hello, hello, everyone. Welcome. Thank you for having me. Tim Van Norman 2:04 Thanks for joining us. So you and I've been talking for a while on different things with regard to AI and stuff like that. So can you give us a little background? A little bit about yourself and why you've gotten into AI? And what what is it about AI that intrigues you? Zhen Zhai 2:24 Yeah, so I come from a computer science background I actually attended or family cottage. So coming back, it's been great. And I've been working full time in Irvine. So I'm local. So I know that I'm a college looking for a professor to teach C++. And I took all the classes here, when I was in IVC. They'd be like, Oh, it's fun to get back. So I got connected with the professor and become a thing that I do after my work once a week. So I've been doing that for a couple of years now. And last year was when AI started to get on the news and GBD started to become a thing. For me, I have been doing AI throughout my career for the past six years. So I was like, Okay, well, this is become more and more user friendly, and very accessible. And people are getting used to it. So that that include my students. So I basically will ask students like, Have you been using chat GPT? Are you aware of this? Last year, about the fall semester about September was when I do see a big push on people raise their hand when I asked like, Hey, have you used or heard of GPT and, you know, have used chat GPT. So at that point, it felt like, hey, like people know about the people who at least have expectation that they will be able to interact with it and use it in a reasonable way. That's when I started to have the idea of integrating, you know, what I do on a day to day job, but like, also help with the teaching. Tim Van Norman 4:13 Nice. And so you've developed a little system you call octopus, and it integrates with discord. And so how does this help your students? What do you what did you create to assist your students? Zhen Zhai 4:31 Yeah, so one of the things that I noticed was that because I work during the day, I am a part time professor so I don't have like dedicated office hour other after class I stay meaning that during the week, students will have questions for me that I may not be able to get back to them by the way, that include like, they will send me a message. Suddenly, you know, some of the code that they're debugging and maybe like, I work during the day, I will not be able to check my email when I get home after dinner, it's like seven or 8pm, I look at the email, and they've been stuck on it for a while. And all that they need to do is they just forgot a semicolon on one of the line, I'd be like, oh, like, I wish I could get back to you sooner. So you won't wait for a day to know that, oh, I'm just lying at the end, you forgot a semicolon first, first. And it's just, I couldn't get to people in time. That's one of the things that I always want to find a way to do. And that is why I developed this kind of a GPT based system. And the reason I use this court is that it does provide covertly for me to integrate box like this got the score bot can be easily just put it into the score channels. And I can control the bar to you know, respond to certain messages, I can give different permission to the bar. So the score as a platform is pretty friendly, when I want to introduce a bar to my students and give them the freedom to ask questions, either in a group setting or in a private message setting. So Brent Warner 6:10 I want to pause here for a second because I know there's people out there listening going wait, you said too many like discord bot program, you know, like all these things, which are great. But we just want to kind of clarify some of these terminologies here so that the people are following along with it. So we have talked about discord in the past on the show. And if you're not aware, that is a chat platform that students are very popular with students and with gamers and things like that. But basically, what you're saying is that inside of discord, you can create a mini program, right, this is the bot. And you you have connected that to chat GPT. So it's not exactly the same thing as working with Chad GPT. Directly. It's a kind of a customized, focused version of chat GPT that works through the chat, the chat bot system inside of discord, is that right? Zhen Zhai 7:04 That's correct, yes. Does core bot allow me to directly leverage a program that I write and to be able to leverage the open AI API instead of chat GPT, which is a product they developed on top of the API, but underlying Same, same assets, right, they're using ChatGPT is basically a user interface that allow people to interact with their language models. But they also give out just direct access to the language model, to people who feel comfortable writing a small program to control how exactly want my user, in this case, my students to be able to interact with GPT. So that is what I did. And yes, so 10 mentioned octopus, I named that bot octopus, because at that time, my Octopus teacher was very popular. I really love that show. I was like, I'm gonna have my Octopus pa off, right? So that's why I named my octopus that I made icon for it. So it's like students, so like they're interacting with friendly bot. So yeah, so that's all that I do. Very cool. Nice. Tim Van Norman 8:19 Nice. So thinking about your prompting the prompts and stuff like that. So you use the the API so that you can actually give the information. Every time a student asks a question, you send certain pieces of information to chat GPT or to open AI, basically. And then it replies based on that information you give correct. Zhen Zhai 8:47 That's right. So you, you can use chatGPT, right, it's the same thing. So instead of directly having students interact with chat, GPT raw, which is like what people can interact with when they go onto their website, if I put something in between. So I will then be able to control how I will like chat GPT to interact with my students, I'll be able to give specific information. So that church of data refer, like as reference for church authority to answer their question with, I can also stop that from sending information like a solution to my project assignment directly back to student because I don't want them to get the solution from TPT I want you to be able to like Teach them or walk them through and help them along the way. So I got to make those changes. And in the middle layer. Nice. Brent Warner 9:42 Follow up on that, because I think one of the things that people struggle with with GPT are these negative commands, right? Don't do this and don't do that. And it still does it right. So it's like, you know, this is this. So So what you're saying is when you're kind of hanging into it a little at this deeper level, you're able to really make filter There's that work, rather than kind of the weird language of English saying, you know, don't give away the answer. And they're like, Okay, we won't give away the answer. The choice was a or, like, I just said not to do that, right, and it keeps doing it. So. So there are ways around that if you get a little bit more sophisticated. Zhen Zhai 10:19 Yes, I actually found that, as the GPT model improves, is actually doing better at respecting my natural language instruction, like don't do the avoid doing that. But when I first started us, only GPD 3.5 was available. I was like, Okay, I'll use that. But yes, definitely suffer from the things that you're mentioning, where I say, don't do this, and they're like, I'm gonna do it anyway. So that is why I implement that, like script to filter that out when I see it coming back from church TV, for example, the status code back, and I specifically told to not to, but you know, so in my program, or in my this middle layer, that I will filter it out, I will detect, is there any code in your answer that you provide it? If there is, we do it again, until there's no code in the response? And then I expose that back to the student. Tim Van Norman 11:20 So that you're alluding to what I want. My next question to be in that is, you said that you're able to see what the responses are. So you get to, I'm assuming that you're not you physically, are not watching as the students typing in this, the response that comes back and all that, but you have a log or you have some way of knowing how it responded, and then you can I assume, do additional training to tweak it to be more accurate. Would that be the case? Zhen Zhai 11:56 Yes, some of that is true, right. So there are a couple ways that I can monitor how GPT responds to my students question why they see the score is though, because I encourage group chatting. So I set up a group for each project assignment. And for that group, the context that I provide to chatGPT is exactly that project assignment, and how I could help to them for that specific project assignment. So in that group, I definitely will be able to see the history because I'm also in the same group. The night what's nice about the group is students will also help each other and student will correct GPD if there's something that it means like, because students is they are so some students with or have already done the project assignment and know that No, that's wrong. But I have been getting into the history and check. And if there's something that's inaccurate, I can remove, delete the bot message, which is also one of the features that discord bot provides that I have full control and managing like, oh, that response from TV is not correct. I can just delete it from the chat history. But while students are directly messaging, the bar without in a group chat, because they do like the privacy and the freedom to you know, just directly chatting with the tight GPT. That is where I use my program to log their conversation history. So I can see how they interact with the bot. And if there's something that's not correct, I can ping the student, let them know, I'd be like, hey, like that is not what I meant. But what's interesting is for group chat, because there's so many people interacting with the same bar at the same time, the bot that I have hooked up does not have memory, meaning that if student asked one question that I will respond to that was no question. And then another student asked another question, the ball will only look at that question and respond. But in direct messages and private messages, but does have memory, I allowed about to know a history of what the two of you have been talking about, because that's important information that student may want to leverage. I do see students will come with like, how do I start for this project assignment? And UbD will give them like, Okay, you do this step one, step two, step three. And then after that you then we'll be like, tell me more about step three. So the part need to know Oh, what's that three? Oh, it was from my previous conversation. So I allow that in private messages. Probably because that I do allow history memory for direct messaging. I actually found that the answer provider in that has been a lot more accurate. So I haven't actually run into things where I look at it be like, No, that's not what I meant like, that hasn't happened as much for the private messages. But in the group message, I definitely do a more I will go both up when I see things that's correct, or delete when it's even. For both different channels, I got to monitor. Tim Van Norman 15:16 Nice. So you mentioned earlier that you had started with Chet GPT 3.5. So are you still on that? or what have you moved to? And and what do you think of the different levels that you've gone through? Zhen Zhai 15:33 So I started with 3.5. Because four was so expensive. I was like, Oh, I don't I don't want this to blow up. My budget was 3.5, I do not really know, or do not have a good expectation of like, how different or how, whether I'm paying for the gap from people back before. Is that worth it? I didn't know. So I started with 2.5. I definitely see problem all like first that I previous mentioned with friends question, but it doesn't really respect what I told her not to do. Right? That's definitely one big problem. And another thing is, it says student asks something that's outside the scope of my class, it will just start hallucinating. So if I have the information that I provide, like, for example, my project assignment is to not only ask question about those, it's pretty good, because I provided that information, and they look it up right there. And we can be able to provide very accurate answer. Anything, I've set up the things that I provide. It just instead of saying I don't know, and it's like start making things up. It's like, that's not good. So after seeing that, meaning that I have to go in and bleed as both things quite frequently, I'd be like, Okay, this is a lot of work. I don't like that. So I bumped into for you has gotten better. At you know, it has lower error rate. And also in my system, I did say that if you don't know if they you don't know. Right? So it's one of the things that do you actually respect what I tell you to do? It got better from 3.5 to four for sure. But you know, it's done sometimes do do things that I don't really wanted to do, like provide things that I'd be like, That's No, I do not have this project assignment you just completely made that. Definitely happened still before. And I went for turbo came out. And it's a lot cheaper, therefore, and they told me that performance would be better than four, why not? So I immediately switched to a four turbo that has made things a lot better. So I talked about I have a hard filter of like, Hey, if you respond me with code, you have to do it again, right. And I'm gonna send that back to student because I don't want a student to just get a copy and copy and paste into their project assignment. My hardware there have not been triggered as frequently. Meaning that it's been doing better i respecting what I told you to do and not to do. So I definitely see that they can prove my import terrible. Brent Warner 18:06 So that's interesting. So you can watch and see how well it's doing. Because so so you're saying, when that feedback comes back as code, it sends you a flag and says, hey, it's trying to do this, right? And then And then so you can keep an eye on how accurate or inaccurate the bot is actually being with its work, because it's because it's sending up the flags that are going through the filter? Zhen Zhai 18:32 That's correct. Yes, I log a lot of information on which part of system a triggered, including like whether a trigger in my heart. So they're saying that do it again, you can have code I told you. So you know. So yes, I lock those information. So I know how much those are being triggered. Very Tim Van Norman 18:48 cool. Nice. So that that's actually something that I hear a lot about is Wait, how does, how do I make sure the students aren't cheating, using something like this. And so you've got a filter set up. You've also alluded to cost. So if you don't mind, don't you don't have to get into specifics. But how expensive is this? And where do you find? Where would you think if I was going to do something like this? And maybe not want as many logs as you're wanting? What What would I be looking at cost wise compared to what you're? You're seeing? Zhen Zhai 19:26 So the GPT costs is actually not as much so I teach a class of about like 55 students. And last semester when I did this, like I said, at the beginning, only about 1/3 of them raise their hand be like yeah, I've used TPP I think ended up having maybe half of the class have experienced things like experience that GPP the way I decided on Discord, but throughout the entire semester I have five project assignments every single project is on the right before the due date. A my Costco spiked up. And when I say buy top, it will be like $7 a day. Meaning that when it's a normal day, it's like $1, maybe two bucks. Right? So you can definitely see like, oh, well, let's do they're asking three times more or even five times more question the day before it's due. Okay. But that's kind of the cost. It's not like throughout the semester, I'll be like, $35 total, what? Open, I build me on their account. So it's not that much. The one that is a little bit more costly is the part that I host my middle layer program, right? Because I was always the best, okay, I wanted to do this and not this year, all the filters, here are the flags, I need to lock off them and to check what my program doesn't work. So because I'm doing very sophisticated logging, that's more like expensive. I'm doing it on Google Cloud. So the expired they paid, they charge me per month, for the instance that I set up on Google Cloud, it's about it comes down to about like 70 bucks per month. So about 200 for the entire semester. Brent Warner 21:21 Okay, that's right. So so, so if someone's looking to do things like this, they're kind of saying, Okay, well, they might start with the simpler version, for example, they might just say, Hey, I'm going to do the thing that ends up costing $30 a semester for just just working with, with open API's platform, right, which is still cheaper than having your own chat, GPT account, right, your own personal $20 or $25 a month or whatever, whatever it is. So if you're just trying to build it into work with your students, and for that setting, it's still going to be cheaper than than your normal setup. And all the students don't have to necessarily have their own chat GPT accounts, which then starts really adding up to costs, right. But you also have your second layer, which is the hardware and getting server space, and all those things, which may be the average teacher, it's not not going to be necessary for them. And it's kind of we're I'm just trying to like look at layers of like, how do I step into this, if it's something that I want to do without without too much expense? So, so okay, I can see how that can balance out a little bit. Tim Van Norman 22:32 So as you look at the future, you've done this now for a semester and a few weeks. What, what do you see? isn't quite there yet. What do you see for that future? What options do you see that you're interested in pursuing? Given the opportunity? Zhen Zhai 22:57 Yeah, so currently, I'm only having GPT to understand a small portion of my entire class, which is just the project assignment, because that's where students have a lot of question trying to really understand them. Well. If you are talking about the teacher, I want to be happy to not know too much about the plus plus for when I interact with my students. So for example, C++ is such an advanced language. And it it's actually pretty good understanding and knowing what to do on the language, but my class is like very scoped limits, right? I only teach certain concepts of the plus part and not the other whole big ocean out there are C++ that you can teach one semester for an intro to programming, I only teach things that will be important for the class. So I will, I would like to try to see whether GPT can answer the question about my project assignment. With only the part of the C++ I wanted to leverage. That will probably require me to find a way to fit in either transcripts of me teaching or slide information. The thing about programming that's tricky is that I don't put a lot of words on my side, I put code and I explain it, I can walk through it. So giving the slide to GPD will not be as helpful than like actually having a transcript of me teaching. But I am hoping that GBD will be able to be very specific on you know, students asking questions on my class, and limited to my class only. So that it will be very precise. Another thing that students really want, but I'm not sure how to pursue that is that intro to programming is very important for them to learn debugging or under They found their code, why their code is not working? What's wrong? Which part I typed wrong? Like, for example, did I miss a semicolon? The arithmetic doesn't tell me that, you know, is that I hope there's a way that I can leverage GPT to read a portion of students reading code and tell it that, okay, this could have this part of the problem. However, I haven't figured out a way for GPT to do that. But without giving students a direct answer right away, because they just be like, well, there's a better way to do it. And here's the solution. easily do that. So that will be you know, what I can work on in the future. And maybe like, as GPT, iterate and improve, it may be able to, like we images, and look at the code in my screenshot, understand what I'm trying to teach. So we'll see things change every month now for AI. Brent Warner 25:56 I love that, because that's the same thing for language. It's like, I don't you don't want to give I mean, we're just talking about different languages, of course, but like, you know, in in the language class, it's the same thing is that we want to work on eliciting the right answer from the students not giving it to them so that they can just step right, you know, and not really learn from it. So. So it's, it's that trick of being able to understand how am I going to push that, to get them to think about it and spend the time working on that change, instead of just giving it to them for nothing? So yeah, I that's one thing that I'm really looking forward to with the AI as well as to be able to better set parameters and say, in this case, do this. And this in that case, don't you know, don't do that, right. Because it just feels right now like it's you're just kind of swimming in the open ocean. And anything can happen, right as compared to in the lane of a pool where you have a specific place you're trying to get to at the end, it's, it's, uh, it'll be great when that becomes more available for all of us. Yeah, for sure. Tim Van Norman 26:56 So, Zhen, thank you so much for joining us today. This has been great. I've loved hearing more about this. And you know, you've brought things that we hadn't even talked about before. So thank you. Thank you, for coming. Zhen Zhai 27:14 Thank you for having me. Tim Van Norman 27:20 Thank you for listening today. And this episode, we took a look at using AI in the class with Professor Zhen Zhai. For more information about the show, please visit our website at the higher ed tech podcast.com. There you'll find our podcasts and links to the information we've covered. Brent Warner 27:36 As always, we do want your feedback. So please go to the higher ed tech podcast.com. And let us know your thoughts. And if you have ideas for future shows, there's a link where people, any of you can give us your topic ideas. Tim Van Norman 27:48 For everyone at IVC. That's listening. If you need help with technology questions, please contact IVC technical support. If you have questions about technology in your classroom, please stop by the IVC Training Center in a 322 or contact me Tim Van Norman AT T van norman@ivc.edu. Brent Warner 28:05 And if you want to reach out to me about the show, you can find me on LinkedIn at Brent G Warner and Zhen, is there a way for people to contact you if they're if they're looking for you? Zhen Zhai 28:15 You can also find me on LinkedIn at Zhen Zhai. Tim Van Norman 28:18 So I'm Tim VanNorman Brent Warner 28:20 and I'm Brent Warner and we hope this episode has helped you on the road from possibility to actuality. Take care everybody