Abstract

How do you teach technical AI skills to people starting from zero, and what does it take to build a sequence of courses rather than one-off workshops? Mark Spindler, who trains teaching assistants (TAs) for the introductory programming courses in Gateway Computing at the Whiting School of Engineering, describes using large language models (LLMs) for role-play TA training and an AI-built web app for COPUS (Classroom Observation Protocol for Undergraduate STEM) observation. He argues for partnerships beyond the history department, because people with deep experience of true novices may be a better resource for humanities students starting from scratch than graduate students in computer science alone.

Chapters

Transcript

AI-generated transcript. It was produced from the recording by speech recognition and cleaned up with AI. It has not been edited by the speaker and may contain errors. The captions on YouTube are AI-generated in the same way. Each timestamp links to that moment in the video.

[0:04] Too many floors we have to deal with here. Is the audio ready for Zoom? Great. So yeah, I'll get started. Welcome everyone. I'm Mark Spindler. I work here in Johns Hopkins University in the Whiting School of Engineering. So let me tell you a bit more about what my role is here. So I primarily support the teaching assistants in what are called our gateway computing classes. So that's intro programming courses like Python and Java and a bit of MATLAB for relevant engineers. And so I'm not the teacher of the intro Python course, but I help coordinate and support and train all of the TAs for that course. I'm also part of a team across the STEM fields. So there's one of me for mathematics, for biology, for physics, for chemistry.

[1:05] And we all work together and with Hopkins Teaching and Learning Center to support the TAs, especially for these intro STEM courses that so many students across the university need to take. Not so often history majors, but we do have some from time to time. I also assist with a variety of other tasks. So one thing that's very relevant here is I also work with the TAs and the co -teacher for Louis Hyman's Introduction to Computational History course. He's giving a session about that tomorrow, actually. And I do things like facilitate some educational research, a little bit of everything, short version. everyone in the room can hear me all right great so here's a little roadmap for what i'm going to be talking about today uh the first thing is i'm going to talk about how i've used AI in the training of TAs and then

[2:09] i'm going to and uh before i move on i just want to clarify when i say AI i mean very specifically like modern large language models like ChatGPT or Claude And when I say TAs, if you're in a different context where maybe you're thinking, is it undergraduate TAs or graduate TAs or even new teachers like some postdocs or something like that, all of that can be included when I say TAs. The second portion, I'll talk about teaching AI use and skills that are related to that, two novices. And some of this might have a little bit of overlap with Louis's overview of that computational history course. and then I'll really spend time emphasizing and explaining the value that I see in external partnerships so if you're someone who's interested in working to build some similar course or a sequence of courses there's probably others at your university that you may not have considered or other benefits of collaboration that

[3:11] are worth thinking about and lastly I'll spend a little bit of time just putting things together and talking about things to keep in mind when you're building a course or a sequence of courses, if that is one of your goals or something you're interested in exploring. So first, let's get into training TAs. I have two examples of ways in which I've used AI and one that is practice for one-on-one help. So often you have the TAs either a help room. They're giving one-on-one help with a student. And for me, my context is usually those intro programming courses, but this can certainly be applied more broadly in other contexts. And the main reasons that we'd want to use AI in a situation like this are that these students or these new teachers, they need to develop good habits both generally and in the discipline

[4:16] in the topic that they're tutoring. And you don't want all of the learning to be done with the real students sort of like their guinea pigs. Ideally, they'd have some training beforehand and on the side. And there's only so much time that maybe the instructor for the course or other supporters of the TAs like myself can really provide. And so an AI is a good way to get some extra practice. and when I say extra practice I mean that the AI could sort of play the role as a student and comment on what the TA is doing well and what they might change and you might be concerned you know AIs are not perfect and so they might make some errors either in talking about the code as a student or giving feedback to the TA and in this kind of context those errors are actually broadly okay because the students themselves, they're not going to accurately say what their problem is or describe the situation all the time.

[5:19] And the TA will have to deal with that. And in the rare instance where maybe the AI is giving some advice, like you might want to phrase things this way, you know, the TA still has the autonomy to say, you know, I don't know that that's supportive enough, or I think that's too wordy, or I think that's too advanced an explanation And even if they take some advice, which maybe isn't optimal, they'll still get more advice from instructor and the AI later, and they'll get experience, and that will sort of wash out in the end. So it doesn't have to be perfect to be helpful. And just to give you a more concrete picture of what this might look like, here's a bit of a sample dialogue. Now, I don't expect us to read it. I just want to highlight sort of what this looks like. So the AI might first start with, here's the topics. What topic do you want to focus on? So that way they can prepare for something specific. These are topics from intro Python, but it could be about whatever

[6:22] course. And then it would play the role of a student. And the student would say, oh, I'm stuck on this thing. And here's this example. And I'm wondering this about this piece of code in the case of a Python course. And then it asks, you know, could you help me understand it or whatever the student's question is that it's a variety of things and it's always random, which means you get a variety of practice. And then you as the TA will provide your response. And then, you know, depending on exactly how it's set up, either it would continue the role play for a bit or it might give immediate feedback. And so here I asked a question to the student and the AI is saying, you know, what's good is that you're making them focus on a concrete case instead of these abstract concepts. But here's a way of phrasing that the AI likes better. And then I can decide for myself, do I really think that's a better way to respond to the student in this case? And I can also continue the conversation.

[7:24] Maybe the student has a follow-up question after my advice. and then I give some follow-up advice and then maybe after some more back and forth, the AI will give me more comments on how I did. This is just what it might look like. And there's no particular AI that is much better or worse for this kind of purpose. And to give you an idea of how did I set this up, this is not the focus of this talk, but just to give you the context, I'll go over what a prompt might look like. And so it might start with some background about the goal. So your goal is to help TAs be effective tutors for Intro Python. And in addition to the prompt, you might also attach, say, a list of topics for the course so it has that context when it's giving advice. And then you might tell it the workflow. So just explain, okay, you're going to ask about what topic to focus on, and then you're going to role play as the student,

[8:25] and then you're going to give feedback. maybe a little vague as to what the back and forth should be. You might want to be more specific, like after three interactions with the student, then give feedback or something like that. And lastly, it doesn't hurt to include some guidelines and guardrails and make it focus more on your goals. So maybe you want to make sure that the AI is not judgmental or dismissive to the TA, because in this context, the TA is kind of like a student themselves. And maybe you also want to emphasize, you know, stick, stay on topic and be concise. Because if, say, in your help, if you're in a help room situation, you don't have time to give, you know, long-winded explanations of every related concept. So this is just what something might look like. a slightly different example of using AI to train TAs or

[9:26] is to build tools and to give you some context for those who may not have seen something like this there are observational tools used to track very precisely what is going on in the classroom either it doesn't always have to be a TA it could be for a teacher's benefit or just to understand different teaching styles So one that I focus on particularly because I'm on a team of STEM educators, UC Boulder has one abbreviated COPUS. It's the Classroom Observation Protocol for Undergraduate STEM. and like I was saying, this gives you a data-informed approach to give feedback so it's not just, you know, you sit in the classroom and you just observe and write down things that are notable, but you can really track, okay, here's how much time was spent lecturing. Here's how much time students were waiting for their questions to be answered or whatever it is that you want to track and that can help understand exactly what the teacher's or TA's habits are, areas of improvement, and

[10:28] how to work towards goals. and to when I say building tools so UC Boulder's COPUS is kind of like you know it's a protocol and it's designed maybe there's a little bit of software or a spreadsheet program that they use and maybe they fill things in a spreadsheet box by box for every minute of everything that's going on but we found, we being my team of STEM TA supporters, we found that we wanted something that was a little easier to use in real time and so a colleague of mine, Liam Browne, used AI to develop a web app that made it a little easier to track things and then AI is very good at supporting coding so we were able to use AI to add features to it over time and adjust it to meet our needs. Just so this is not vague, here's a picture of a real COPUS output

[11:32] showing the sorts of things you might track. So when was lecturing going on? When was the teacher writing on the board? When were students asking questions, et cetera? Those are some examples of something you might track. And AI has been useful in developing the app that our team uses. Great. Next, I'd like to shift gears to talk about training novices in AI and related skills. So first, there are a number of different ways to use AI. And AI is really great. It can do all sorts of amazing things. but there's still a big difference between using it and using it effectively. And that involves understanding the breadth of what is possible with AI and combining it with other tools and also understanding when to be skeptical of it and how to test whether the answers it's giving you really are correct or meeting your

[12:35] needs. And so in some straightforward uses, maybe you have a corpus of text and you just ask the AI something along the lines of, find me an example where this sort of thing happened. And then it goes in and searches through and finds it. It's kind of like a better search. And you can easily just check, you know, is that snippet of text on the page? Excuse me. Does it say, is it actually an example of the thing I was looking for? You can just sort of read that yourself. But there are more in-depth uses that you can do. So maybe you want to import treasury data from the internet using an API that's a, usually don't spell it out, it's an application programming interface, I believe. Yes. And so that basically imports the data into some program that you're writing maybe in Python. And then maybe you want to do some statistics and make some nice plots to really understand what this data means.

[13:37] and if you want to do all of that, it's not ideal to just ask the AI and get a plot out and hope it's right, although with some of the more advanced AIs, it may often be right, but you often want to involve actual, say, code yourself, and you want to interrogate that code and maybe modify it. So this is a more complicated use that takes some training. And also, if you're doing something more complicated, how do you know that you're confident in the answers of this plot that came out at the end when you ran the Python code or when you got the result of the AI? That's something that definitely takes training. And so there are some skills for effective AI use that a course like computational history might do. And the most basic skill before any of that complicated use stuff is just comfort and trying prompts of the AI. A lot of novice students, they're coming to this

[14:38] and, you know, they're in that, they're used to knowing exactly what to do next or exactly what to try next. Cause you know, maybe especially in some of the more, some of the other computational courses they're used to, you know, the textbook says, first you do this step or first you enter this line into the program. But if you're just getting You really have to get comfortable just trying things. Ask the AI, you know, for the answer. And maybe it doesn't give it to you right away. Or ask the AI, how do I do this? So that's actually something that's one of the first skills that's important. Louis Hyman correctly identified that, you know, you really want to be playing with it to get a feel for it. and the next thing that you might want to focus on is teaching the uh how you would generate useful prompts and the variety of prompts and ideas that you might apply AI to because you know maybe they

[15:41] have a couple of ideas or they've seen a couple things but they might not realize oh i can ask AI that i can get something useful out of AI this way if i ask it for code instead of the answer. I can get more reliable results. All these sorts of things are things that have to be shown to novices. But once that's been done, then they can really run wild. And one thing I was alluding to is often you want the AI to generate some code in a programming language like Python, say. And so even if the students aren't necessarily going to be writing some code from scratch, the AIs are very good at writing basic code, they should be able to read it. An analogy that I like from Louis is to books, academic books, like A Short Course in Reading French. If you're reading some French papers on history or some other humanities subject, maybe you don't need to be able to speak French or write your own French essays,

[16:43] but you need to know something about how to read it. And we're in a similar situation now with AIs and code. So maybe the AI will generate some code, but you should know something about how to read it so you can say, does this part look like it's doing what I asked? Does this part look, do I want to modify this part with or without the help of AI? Reading code can really help make that use more effective. Another thing you might want to do is maybe the AI spits out a table of data or it spits out Python that spits out a table of data and then in your favorite spreadsheet program you want to manipulate it. You want to move things around. You want to calculate things and really get a feel for what that data means and what your questions about it might be. And even though an individual question you already have the AI might help you answer or write code to answer, I find that you explore data more thoroughly if you get your hands on it in a spreadsheet.

[17:43] And if you're dealing with a novice who hasn't done much computational anything with a computer, that's something you'd have to teach. Lastly, I just want to put a couple of plugs for some other sessions. There's teaching history with AI later today. And teaching computational history I mentioned is tomorrow about the course that I sort of have in mind. another thing to be aware of when you're teaching these skills to novices is that very often there's a lot of technical challenges so maybe you want to focus on the history you want to focus on calculating the statistics but they're going to run into technical challenges and you have to be prepared to minimize those and deal with those with a variety of techniques so one thing this an anecdote that I'd like to share is someone in a similar course, they wanted to

[18:43] work on something. And so the first task was, okay, install this on your computer. But they were doing some research with some sensitive data, and they could not install things on the computer that they preferred to use. That caused a hiccup in their ability to engage with the course. So it's a rare thing, but something to keep in mind. More commonly, if you're teaching programming, it's very common to use a complicated program that has a lot of sub windows or tabs inside of it to write code and to work with code and work with things that it outputs. And the thing about that is a novice is very easily going to get lost or confused between them. You know, they want the something and they don't see it because they're not on the right sub window or tab or they know they want to paste the code into the program but i pasted it into the wrong box because there's three different boxes and they weren't sure which one so you really want to try and use

[19:46] the simplest tools and i'll reiterate that in a bit another thing that comes up for all sorts of intro computational courses is file and folder management when you get to the more advanced tools. So if it's something you can run in a browser, then you often don't have to worry about this too much. But if you're using a more advanced tool that you have to install on your own machine, then you probably need to know where the files are and how they're organized in folders and make sure they're set up in the right way for the code that you're going to run. And especially with smartphones and better search on machines, a lot of people of all ages don't have this skill already, and that's something that they might have to build if they're going to use these more advanced tools. Another thing you might have to deal with if you're teaching more than one person is different setups. So some people are going to be on a Windows machine, and some

[20:48] people will be on a Mac, and some people will use Microsoft programs like Excel, and some people will prefer work slightly differently. And you'll have to, it's hard to account for every situation, but you'll want to worry about this variety of different situations that the students are coming to this work in. And I've touched upon this a bit already, but just to summarize some recommendations in light of all of these goals. So you really want to focus on the easiest and most accessible tool. So not the most common tools that programmers use, but something like Google Colab is straightforward for you just paste the code in and it basically just works for you. If you can, you want to test multiple environments. Maybe, you know, if you use a Mac, try to borrow a Windows machine just to make sure that you can get things working there. also whether or not you have TAs I really recommend doing rehearsals so both

[21:52] to show the TAs a variety of ways that the students might be approaching things and also to identify any issues or confusion that a new student or you know any other novice might have when they're looking at this assignment or this activity. Because so many things can go wrong, it's good to have a dry run and say, oh yeah, if the student does it this way, then you need to give them this advice. But if they do it this way, they might run into this issue. It's a really valuable thing. And related to that, for a class itself, I strongly recommend having live computational work time. If you just say, go home and use the AI and use your coding programs to get to the answer, then they won't have the real-time support and they won't have the benefits of watching a demo where they can ask a question in the middle. And AI can only help with this so much.

[22:57] So they can they ask AI, you know, I got this error, how do I fix this issue? But they can't exactly typically ask an AI, could you do a live demo of the like three different ways I might do this in Excel? And then so I can decide which works best for me and also what issues I might run into and ask a question in the middle. And like, that's not, that's not exactly something you can ask AI today, but it's something that you or a TA could do in the classroom. and as I said certainly for some errors because you're not going to be able to test every single computer every single program that students might be dealing with but AI is pretty good at giving some suggestions for troubleshooting so they're trying to run some code and got the error there's no plotting package in your Python installation well you can ask the AI how do I fix that if I have this program set up for Python and it will give you good advice to get around that.

[23:58] And if the students are shown that that's something they can do, they might feel more comfortable and save your time and the TA's time by solving some of the errors they run into themselves. Next, I want to move on to talking about external partnerships. And there's a lot of different things I have to say, but first I want to go over some benefits of non-traditional partnerships. So what I mean is not just with others in, say, your history department and not just with whoever the maybe go-to AI person in the computer science department is. So full disclosure, you know, I'm in the School of Engineering. I work traditionally with these intro programming courses, and I don't have

[24:59] a significant history background. So when I work with the TAs for Intro to Computational History, I might ask questions that someone who has done work in history would never ask because I don't have that experience. And the students, especially if this is an intro course, the students, the undergraduate students also won't have experience doing work in history. And so my questions can identify blind spots or things that could be clarified or connections that could be made more explicit in whatever the course material is. And I also have a lot of experience in these intro programming courses with the training of technical novices. so just like I'm giving you in this talk I've seen all sorts of issues with say folder management from a novice who's trying to

[25:59] learn how to code or you know other issues that people conceptual issues that people who are learning to say read code for the first time might run into and there are some computer science faculty who certainly who teach these courses and they might be a great resource for this kind of benefit. But not everyone in the CS department is necessarily going to have experience working on the basics and having the patience and instincts for what issues a novice might run into. But just to clarify, you know, I'm talking about teaching these AI and related skills to novices, perhaps in the context of history. But there is another session later on working with computer scientists for history research. So if you're interested in that, go to that session.

[27:01] and lastly because my role is this sort of external role in helping these TAs and I'm coming at this outside of the history department in the case of computational history I'm have a I'm uniquely suited to help the TAs for something like computational history because I'm not the instructor they report to. So there's no authority issue to be worried about there. And I'm a novice to history. You know, if they're embarrassed about not knowing technical things, I can be the one who doesn't know the history things. And so both of those aspects help them feel more comfortable in interacting with me and learning and prepping before, say, a lab session through things with me and identifying places where students might run into trouble.

[28:05] Another aspect I want to emphasize is that, you know, at many schools, there is a computer science department and they have their own intro computer science courses that teach students to write and read code. And even if the goals of a computational history course won't be exactly the same As I said, you know, they probably wouldn't be writing code from scratch. There's still a lot of overlap and you could benefit from that. So you know, a novice in either sort of course is still going to be grappling with what does this basic code in Python mean? How do I read it? And they're also going to be grappling with similar technical issues of, you know, maybe how do I install the program on my machine that I paste the code into? working right. And in terms of designing a programming course, designing a course where students are going to gain these skills, as I said, there's a lot of overlap and

[29:07] the people who teach intro programming, they've already seen a ton of issues and already solved a lot of problems that you could run into if you don't benefit from what they've already gone through. And so even just chatting about what should I worry about if I'm doing a lesson on reading this aspect or setting this up could be helpful. And perhaps more directly, maybe there's even assignments that could be exchanged and modified. History course has interesting applications of code. And even if in the history course, course, the AI might not have written the code, it still could be an interesting application for the computer science course, write code that answers this interesting history question. And vice versa, there's probably plenty of exercises and questions and activities about getting a better feel for what

[30:10] this code is doing that might be able to be ported over into a computational history type course. And lastly, I don't know what your positions are like or how high a view you might have of the curriculum at your universities, but there could potentially even be connections as some computational history course or sequence is built. Maybe a student who is perhaps, how do I say this? A student who maybe is struggling or drops an intro programming course, maybe they would benefit from starting off with a computational history course, getting a feel for reading the code before they really have to focus in on writing complicated code. Or maybe vice versa, someone who has just taken the intro computer science course might benefit from taking the first or second in a sequence of computational

[31:11] history courses where they can apply those skills with AI to answer some really interesting questions. And what is a prerequisite or a suggestion for what? That's something to be considered. Next, I just want to describe how these partnerships can really support my context of TAs in particular. And so I want to emphasize that even if, say, the teacher of the course is very experienced with teaching computational tools and very experienced with teaching history, they might not be as experienced with teaching their TAs, with teaching other educators. How you interact in a positive way, in a way that will support learning with your students. And that's where someone like me can come in and help support that for the TAs that I work with.

[32:14] And this is important for a course that has TAs because even if the instructor has some office hours, the TAs are more often the front line. Students are more often comfortable speaking to a TA compared to a professor. And the TAs need to be equipped to help those students effectively. And what I can do for TAs, especially those who are new to either teaching at all or to tutoring this particular topic, is to help them see through the eyes of a student. so you know if we're working through if we're doing a rehearsal of some computational lab activity i can point out you know this lab has introduced a lot of new vocabulary so you may want to slow down and make sure that they understand before you say something that uses five new terms at once or maybe this is a type of mistake that a student using Excel might make and I can also

[33:20] emphasize general pedagogical practices and for working with students things like rapport and encouragement and celebration are important and some of that is especially important in this kind of context where maybe a student is a history major history is their comfort zone and then they're going out of that to delve into these technological things these computational tools that they're not used to, and they really need all the support that the TA can give. And just to be clear, while it's great if there's someone from a teaching and learning center or someone like me to support the TAs, this and the best TAs is not necessarily a substitute for the instructor designing the course well and engaging thoughtfully with what the activities what the assessments are going to be. If that foundation is not there, there's only so much that the

[34:21] TA could do. So I alluded to a teaching and learning center. If you're at a university or similar that has something like that, I really encourage you to reach out and make connections if you're not already doing that. They probably have a lot of resources to help teaching go well. And for the things that you're very experienced with teaching and the programs that already exist, maybe you don't need any of that help. But when you're building a new course, it's not a bad idea to draw upon all the resources at your disposal. Here at Johns Hopkins University, our center is called CTEI, the Center for Teaching Excellence and Innovation. And they offer a lot of things. These are just some examples. You've never looked at a teaching and learning center or want to know what Johns Hopkins has specifically. So one thing it has is a TA orientation, which is

[35:22] general training for all the TAs across the university. Every TA is required to take this and they learn some general pedagogical principles and also things like laws like FERPA and other things they need to know. But it's not always going to cover all the specifics for the specific topic the TA is going to be working with. And so that's where department or division or course -specific trainings are still important, absolutely. And that's why I'm the teaching and training specialist for the Gateway Computing Program, because the intro-programming TAs need intro-programming-specific training. sort of general orientation. Another thing that CTEI offers is a workshop called Best Practices in University Teaching. This is primarily for university faculty and is the sort of

[36:24] thing that is good to take if you haven't developed a new course in a while or you just need a refresher. It has a lot of good information for people at all levels. Another offering that I've been involved with myself in helping to run is the Teaching Institute. So this is a different workshop intended primarily for grad students and postdocs who are starting to build their own courses and coordinate their own lectures and apply all sorts of design principles for these courses. and they get a broad sort of introduction to a lot of different pedagogical topics and they get to put them into practice and get feedback on that. And one thing I like about this is our teaching institute is not limited to just Johns Hopkins people. So we have people from other universities both here in Maryland but also coming from very far away sometimes to come to our teaching

[37:28] institute and get this training for someone who's maybe about to teach their first course. And our CTEI offers many other things. They have podcast discussion groups, discussions about educational research, a lot of other trainings, both for, say, grad students who want to get some better teaching experience and more training on their CV, all sorts of things. And another thing, aside from training specifically that are offered, is they also have a wealth of educational technologists. So, you know, in addition to people who are teaching an intro programming course and any other people in a useful role like

[38:30] myself, the Teaching and Learning Center here has educational technologists who are going to be familiar with AI. They're going to be familiar with installing things on machines. They're going to be familiar with the differences between Windows and Macs. And they might be able to be a good consultant if there's anyone like that at your university. Okay. The last section is just some summary and comments on building a course or building a sequence of courses in something like computational history and some things we should consider if you're embarking on that. So the first thing I want to emphasize, if there's one thing you take is that training novices is achievable. It's not this insurmountable goal. Even if, you know, they're coming from this

[39:32] history background, they haven't done anything fancy with AI at all yet, it's absolutely something you can do. And a main reason I know this is because they do the same thing in intro computer science courses. Some of the students in an intro computer science course have done programming tutorials and have some experience, But a lot of the students are complete novices, just like a majority of history students say. And if you're making a course, you want to think very carefully about what you're assessing. because the computational side of things, especially in light of modern AI, is going to make it slightly less obvious what you should be assessing and what you should be building students towards. So if they solve one question that you pose to them, answer this question based

[40:33] on census data and whatever, that's great. will they be able to answer the next question that's on something different that uses related skills with, say, reading Python? And if you assess the process and you assess that they're doing things in a way that you can expect to work the next time or they won't have to learn too much to solve the next problem, then you're really setting them up for success both later in the course, in the next course, and in any historical work they do later if they want to apply these skills. and I was just going on about the Teaching and Learning Center, we definitely want to apply best practices, especially when students are going out of their comfort zone and you're combining multiple topics together and making these connections. You want students to build a good foundation, especially both within a course and in a sequence of courses. I said it before, but I'll say again, a time for live work with help right there

[41:36] is critical for something like this. learn this by reading the textbook or go do it on your own and just ask the AI when you need help. That's not enough for this sort of course. As I alluded to, advanced applications are going to need more advanced tools. So if you just need to paste a bit of code and get something out of it, some web tool like Google Colab that I mentioned before is probably enough. more advanced computational work, you're likely going to need to install applications on the student's machine. I mean, the students will install it and when, and manipulate files and folders on that machine. And all of this increases the amount of things that can go wrong. And so you need to be prepared for that if you're going to get into that more advanced work.

[42:38] and lastly especially if you're thinking about building a sequence of courses then you're going to have to scaffold things because there might not be too many people in your history department that have a lot of experience with AI or reading Python code and then also the TAs if you have TAs they might only have limited experience and they won't have experience teaching students these concepts. So, you know, when a first course runs, the TA will gain some experience teaching students these basic computational concepts and AI usages. And then for a next course, you ideally would want a TA who has already taught or tutored something like this before. And similarly for instructors and co-instructors. So it's going to take time have a vision of, say, a sequence of courses that really gets into some advanced applications for historical research.

[43:39] And you're not going to get it in a day or in a semester or in a year. This can't be done quite that way. So with that, I'd like to open the floor for some discussion. And that can be questions for me. That random comments, if you want to share things that you're doing at your university, or if you just want to talk about, you know, my goal, I'd like to have an introduction to computational history course at my school, and I'm running into this issue, or I anticipate this issue, or this was or was not helpful, or anything like that. I open the floor. Yeah. Thank you. Brilliant. You said that you're pitching it kind of like the income comp side, but obviously they are trying to learn, say, history at the same time as the technical side.

[44:42] If you were measuring kind of like the learning of computational tools over the course of that combined force, how far sort of the fall of the standard income comp side of like learnings? Great question. And I'll repeat it for the people on Zoom. And so if I understand correctly, basically something like introduction to computational history is teaching both history and computational tools. And there's some overlap with, say, an intro programming course. And I believe the question is basically kind of how much overlap is there? How far into an intro programming course or something similar would they go in this course that tries to blend these two topics? and I mean the short answer is it really depends uh and as I said Louis is going to give a talk tomorrow on his particular real course that he offers but I would say that probably for computational history

[45:44] you typically want to combine more skills than the narrow focus of most intro programming courses. So an intro programming course, say in Python, is going to teach a lot about maybe here's how you write Python and here's how you plot graphs and you'll do a little bit of calculation. But for computational history, maybe you really need to run some statistics on some data. Maybe you really need to grab data from some government repository. and those skills might involve Python, but they won't necessarily be taught at all in the intro computer science course. But the flip side is, of course, they're not writing code from scratch. AIs are very good at writing code, especially for, you know, isolated one-off things like write me code that grabs the data and plots the graph. Great. It'll just do that for you and give you the code.

[46:50] So a lot of the code writing concepts wouldn't be taught in this sort of computational history course. But, you know, whether you say, okay, a third of intro CS is included or a quarter, you know, I can't measure it and I can't say what's ideal for your case. But it's a great question and good for everyone to think about. Yeah. Yeah. I really appreciate your talk. focusing a lot of your experience and your recommendations on teaching students. I was just wondering how you would change your recommendations if you're talking about teaching researchers or like PhD students or professors in how to use computational tools that are not hybrid. Sure. So the question is, you know, I was focusing on teaching students, but what about teaching teaching PhD students, teaching other professors and current researchers

[47:50] in history, some of these computational tools. And a lot of what I said, uh, can apply, but probably the context in which you're say teaching professors, these tools might not be one where you have a TA to support you or someone to support a TA to support that teaching. And so those parts of the talk wouldn't really apply. But I think that, you know, someone who hasn't used AI much, someone who's never read any programming code needs to build up skills in a very similar way, whether or not they're an undergraduate student, whether or not they're already doing some research in history. And as I said, there's another talk about working with computer scientists. So if you don't have time to go through a whole course or course sequence with the professors and PhD students, maybe you can collaborate more with computer science department and get help from them on answering some questions as well.

[48:52] Yes. I teach a logic introductory class with TAs. And one of the problems I confront with is the common ones, how can the TAs discern when papers are written by data? How can I help the TAs use AI to, again, expose this kind of cheating? Okay. So the question was, in a course that has a lot of TAs, how could, how might the TAs identify, say, papers that have been written with AI and how might we deal with that sort of cheating? I will say that my focus here has really been in teaching the use of tools. And so the one thing that I said that might be relevant is that in the type of course that I'm talking about,

[49:55] the use of AI is something that would itself be assessed. And so I'm not someone who could really speak to how can we detect if the essay was written by AI. There are a lot of methods that people have tried and a lot of issues with various methods. But if someone is going to use AI, you still want to know do students understand, you know, the code it's writing? Do students understand the comments it is making about some historical data or whatever it is? So the short answer is I might not be the best person to answer that question. And it is something that a lot of people are struggling with. I'm talking really about courses that sort of rethink things of, okay, what can students gain with AI rather than teaching students the separate skill of how does one write an essay without an AI? Something like that.

[51:03] Other comments or shares about what's going on in your world? Yeah. Is there more detailed scaffolds around those you've developed? Or is that just kind of like three or four thresholds that you've identified as useful methods for tools? OK. So the question is basically, these skills for effective AI use, is this is there a more fleshed out ladder of skills that has been developed or you know what more can i say about this does that capture the question um yeah so i would say that these are just maybe even less than a framework these are sort of observations i've made about

[52:03] different ways that people can use AI in general and specifically how they are using it in the introduction to computational history course. In terms of scaffolding, it would really, you know, there's a lot of other things that one can do with AI that I didn't touch upon at all. Like, you know, so I'm not thinking about how do you organize a large amount of data and filter it in ways that are useful to answer historical questions. That's another skill, maybe a skill that would happen in a later course even, to do that right and to do that in a way where people could build on it later on. Because you might be able to ask an AI, give me the names of the five letters that pertain to this topic, and maybe you can do that, but you can't have, you know, but if you want to categorize things

[53:06] and sort of archive things and build an archive, that's like, there's probably a whole world of advanced skills in that and how AI can help with that that aren't even in my head. Um, so the short answer is this is just very incomplete. Yes. Building up what you're saying, I wonder whether, I'm wondering whether there's been any move in your institutions are thinking about you know some of the fields that are being covered in intro to computational history might well be as well frame it as like introduction to computation in the humanities and teach essentially the pathologies of you know gateway course for this and then there's the second course which might be intro to computation or no longer intro moving onward with kind of thinking in history where you would deal with some of these questions about creating a pipe data pipeline or dealing with larger instruction larger sources of data so have there been um moves within your institution to think about creating a scaffolding towards which that might be

[54:06] drawing in a larger set of students for the introduction from across different disciplines but then becoming more high level or follow-up yeah that's that's a great question so let me uh reiterate it for the people on zoom basically so these skills them apply more broadly in the humanities. So is, say, Johns Hopkins University considering taking some of these skills and teaching them in a broader maybe introduction to computational humanities course that could then have follow-ups like advanced computational history that would then go into those more advanced skills as needed and maybe a different advanced course for another field. Um, I'd say it's very early days, certainly, uh, for our AI informed computational history program, but I, I liked the idea. And, uh, if we could, if, if there's interest and if we can do that, um, that, that would be great.

[55:11] Uh, So, yeah, I don't know. I don't have a great sense of who the audience in the room is, but does anyone have any related goals to this that they'd like to share? Like, at our university, we're also looking at computational humanities or whatever it is. I'm just curious. seeing nothing in the room i will attempt to check on Discord in case yeah no it's all very new okay yeah there is a question in Discord so let me see if i can uh reiterate and address that So the question is, do you think it's best to have a structured course

[56:13] sequence versus trying to fold some of these skills into extant sources? For context, I'm a medievalist. All historians comment on how much we have to teach in our surveys, but my field survey covers a full millennium. I'm very interested in trying to incorporate more and new skills into my courses, but I struggle with the practicalities. Any advice? and then there's a appreciation for mentioning centers for teaching and learning um so i would say going back to that kind of quote-unquote hierarchy of skills some some of the basic things and the basic play with it and answering some questions and summarizing things those sorts of skills and just introducing AI as a way to get a handle on those. And maybe also for other things like study skills. You know, if you have a millennium of material to cover, maybe the AI can help you

[57:13] create a study guide, create practice, create prompts to think about. So those sort of more basic AI skills, I think, could be introduced to a course like a survey on medieval history very reasonably. but I imagine that basically I would say that working manipulating things in spreadsheets beyond simply opening a table of data or reading Python code those really require the investment of a significant percentage of a course and that's not something you're going to easily be able to add on or even like add on piecemeal through a sequence of history courses I think it really deserves its own course, like maybe introduction to computational humanities, for example. Great question. Thank you. All right. Well, if you have any follow-ups for more discussion, you will

[58:14] just want to connect, you can absolutely email me. I'm Mark Spindler, and my email is mspindler at jhu.edu. Thank you all for coming, both here and online, and have a great rest of your time.