Abstract

Dean Chris Celenza describes testing a large language model (LLM) on a page of Beneventan script from a manuscript at Oxford, and concludes that expertise has shifted rather than ended. Louis Hyman then argues that historians' habit of source criticism and their ability to build inspectable data pipelines, including optical character recognition (OCR) pipelines for bureaucratic archives, make the discipline more relevant. Hyman closes by introducing the next speaker, Marnie Hughes-Warrington.

Chapters

Transcript

AI-generated transcript. It was produced from the recording by speech recognition and cleaned up with AI. It has not been edited by the speaker and may contain errors. The captions on YouTube are AI-generated in the same way. Each timestamp links to that moment in the video.

[0:06] Good morning. Welcome to Baltimore. Welcome to Johns Hopkins University. Our first introduction will be by Dean Chris Celenza, who is one of us and who is excited to introduce the conference. I'm very delighted that he offered to be here to welcome us to Johns Hopkins and to the AI and History Conference. Well, everybody, as Louis said, I'm very happy that you're here. And we're really delighted that you're in this relatively new building. This is a building that is known as the Agora Institute Building, although around here in the vernacular, we pronounce it Agora, which as a classicist does pain me to hear that. But that is the way it is. So we call it the Agora Building. It's a beautiful building. still in effect being worked on. So one of the things you'll notice is that there's an automatic system for shading and it is not operative today.

[1:09] So we're just going to have to push through it together as allies and friends, shoulder to shoulder. And I'm just really so happy about this conference. It's amazing to me how many people not only are here in Baltimore with us, I'm so glad that we at Johns Hopkins are hosting this, that we're doing it in collaboration with the American Historical Association, that my own School of Arts and Sciences, which I lead, is the key partner here. I think that's all wonderful. Welcome to all the guests online as well. I see that the numbers are, I think there were over 1,300. I saw this over 1,500 people are participating in some way or another today. So this is a strikingly interesting event at a strikingly interested moment. And I was honored to be able to introduce. And as I told Louis, I've got about three hours of remarks. I'm kidding. No, I'm kidding. I've had about 10 minutes of remarks. But I did want to sort of walk you through a sort of shock to the system that I felt that I can see as the Dean of the School of Arts and Sciences is happening

[2:11] throughout all the disciplines. And I'll just, I'll show you a little something here. I'm sorry if you can't see it because the light in the room. I'm hoping the guests online can see it. What you see here is a manuscript from Oxford University. By training, I'm a historian of the later Middle Ages and early modern period in Europe, and I'm a historian of the Latin language and how it's used. And so one of the things I have a teaching interest in is teaching a subject called Latin paleography, and that's a subject basically where you effectively learn how to date and decipher were old Latin texts from Roman antiquity all the way up through the era of printing. And it's a field that has been, I would say, very artisanal, very apprentice -oriented. You learn it by doing it, by staring at a page where you feel like you know you know the language, but you can't make out a single word, and then you learn to pick out a letter and an abbreviation, and you realize you can actually learn this. And so somewhat against my better judgment because of my administrative

[3:12] obligations, I taught a PhD seminar last semester in this field. And so in the spring semester of last year, and it was a great seminar. We had PhD students from history, classics, art history, literature, and so on. We had a great time in that seminar. We went through the sort of canonical ordering in a canonical way. And at one point during the semester, I decided to put one or two of these pages like this that you tend to study in a class like this through an LLM and just ask it what it made for it. This here, what you see here is an example of what's called Beneventan script. It is a Latin script that was perfected in the 11th century in the abbacy of Monte Cassino by an abbot named Desiderius. And it spread and it's part of a variety of scripts that evolved after the reforms in the 9th century of the Emperor Charlemagne and his cultural cohort. And so there's a bunch of different scripts that evolved and you can kind of trace where they are and how they look and so on. This one I've always found quite beautiful, but it's also sort of forbidding when you first see it, and it's

[4:14] different. What you see here, effectively, at the beginning here, it's a little bit cut off. This bird here, if you look on the side, there's a bird. This is the letter I. And you see up on the top there, you see the words. Not sure if I can get this pointer there. You see the words there. there's an n and then you see a p and r i linked together an n a c and an i a p i and o and you see this is in principio the next word is erat was the next word is verbum in the beginning was the word et verbum erat apud deum you see a standard abbreviation for god and so on et deus erat verbum and god was the word now these are things that the standard abbreviations things like that You just sort of learn in the artisanal way of doing this course. When you first look at that, it's very hard to make those letters out. And indeed, what winds up happening is you see certain letters, like this E here. It's hard to know that that's an E, the way it looks. If you look at this C here in the word principio, that sort of looks like an E.

[5:18] It's what we call the broken C of Beneventan script in the kind of microfield that it is. You see other characteristic abbreviations. For example, this P with a line in it is a per, P-E-R, et cetera, et cetera. So there's stuff like that. And so I thought if there's anything that an LLM won't be able to figure out, and this was about six months ago that I did this, it's something like this. It's weird. It's characteristic. It's sort of very sui generis, as it were. So I put it through an LLM again six months ago, and it said, well, this looks like Beneventan script. I said, oh, and it said it probably comes from Dalmatia, so like the Montenegro area. It was right. It had realized what it was, that this is this manuscript in Oxford. And then I asked it, would you decipher it? Would you actually decipher the text, transcribe it? And it did about six months ago and made a few mistakes.

[6:19] It sort of missed that broken C that I pointed out. And then I said, well, you did a great job, but I think you missed a couple of things. And the model said, oh, I'm sorry, I'm sorry, let me do it again. And it did it again, and it got it right. So I tried doing it again about a week ago on a different model, a different LLM. And it basically was perfect. And not only was it perfect, it transcribed the abbreviations, doing it in precisely the way that professional paleographers do, by putting a parenthesis around the extended word. And I realized during this whole period, I felt the ground shifting underneath my feet. And I'm a historian. I tend to use this as a tool. This isn't my primary area of specialization. I teach it as it needs to be taught. So it didn't feel like the world was collapsing, but it felt like, wow, this is really different. So it made me realize a couple of things. First, for this specific field of Latin paleography, it was so technical and so artisanal, at least I felt, almost as if the questions being asked were self-evident, as if the answers

[7:22] would arise just from doing the work itself. So it was almost like you didn't think you had to reflect that much on the questions. This new moment forces us to reflect on what questions are we really asking? It's not that human expertise has ended in this case. It's that the expertise has moved a little bit. It's shifted. You still have to know stuff, right? I mean, you have that beautiful letter I in the shape of the bird. You realize a book is more than just letters. It's more than just decoding physical artifact, right? There's artists involved. There's, you know, an equipe that's working on the text. It's not just one writer. It's a painter. It's, you know, there's inks. You can trace all these things. You still have to think about what are the human questions you, the historian, in this case, are trying to ask. And so I realized that, and that was, again, in my own field. This is how this happened. The one thing I'd say before I close is just that as I, you know, as the Dean of the School of Arts and Sciences, I have the great privilege of basically getting to know as well as I can, folks in all the humanities, all the social sciences, and all the natural and quantitative sciences. And kind of each place in that map, I've sensed similar senses of the ground

[8:26] shifting between people's feet. Some people jumped on the bus very early. Our biophysicists, for example, when they realized that years ago now with AlphaFold, you could do things in decoding the nature of proteins that would have taken years, if not decades, days before that you could do it in a matter of hours, they jumped right on it. Our physicists have felt, I remember talking to one who said like he felt like he had an existential crisis for a moment, then he got beyond it. And the last time I saw him, he was holding a laptop in his hand, walking with it because he had like three AI agents working on something and they were all working and he wanted to see what they were. So I think this is a moment where, you know, we will have to learn together. I do think as a historian of the book, this is a moment, I would argue, really, truly unlike any other in human history in the way we process information. I think it's a profoundly epistemologically unsettling moment because the nature of what you know, who knows it, where the knowledge resides, how it's produced, and so on is, I think, calling into question. And then finally, for most of us in this room, I suspect, our humanists, we care a lot about writing.

[9:29] This is a real moment for writers. And I've seen reactions all over the map. I've seen people, you know, like purity tests. No, you can't do this. this, you shouldn't do that. I just think it's too early. I think we have to give each other grace. I think we have to work together, learn together, experiment with it, and see where it goes. And this is why, again, just to close, I am just so thankful and delighted that you all are here. I'm so thankful to the folks online that you're here. I think it's going to be events like this that help us move forward as a community. And so with that, I'll thank you for your attention. I'll turn it back over to my good friend and colleague, Louis Hyman. Thank you, Louis. Everyone is, okay, good. Everyone's sitting down. I'm so glad. I'm old school, so I need a podium, as many of you do as well, I assume, in your teaching. My name is Louis Hyman. I am a historian here. I'm first in the organizer, and I'm so glad to see all these spaces here in front

[10:32] of me. Welcome again, and thank you, Dean Celenza, for your opening remarks. American Historical Association for being our partner in this. I want to do a couple thanks before we get started to the staff at JHU, SNF Agora, the Center for Economy and Society who made this possible, and Cytrynbaum, Young Song, who is an amazing event planner, Kelly Milo, Brontë Lake, John Handel, Simon Halliday, Heather Michaelson, her amazing AV team. Can we just thank them for putting this all together. This would not have been possible without their help. I'm looking at you. Thank you so much. This really is, the building is like democracy, a work in progress. So yeah, the curtains don't come down, but we get to sit down and have food. So there's that. I want to thank you to the more than 1,500 of you who are both in person and online from 37 plus countries who are trying to figure out, like

[11:33] everyone in this room, just like me, just like Dean Celenza, what all this AI stuff is all about. So when ChatGPT came out a few years ago, I was very dubious. It gave me a absolutely ridiculous listicle about the causes of World War I. And I thought, this is so dumb. What a gimmick. This is the next Bitcoin. And then I asked it to write me a script in Python. And I was hooked. Like you, in the last few years, my scholarship and teaching have been turned upside down by large language models, what we generally call AI. And mostly, to my continued surprise, I find it to be in a good way. In my teaching, I realized for the first time I could teach my dream class. I've always been involved in quantitative methods in some way or another, but I

[12:33] could never actually teach it. I could never actually teach it in one semester. I could never teach them all the tools that they need to know alongside what I consider to be important parts. The questions, how to think like a historian, how to interrogate documents like a historian. I couldn't do that at the same time and teaching them where that semicolon would go. It's too much. The syntax is too much. And so now I can teach them how to do that, or at least rely on the AI to do that, while I teach them the more important thing, the incredible skill that we all have in this room of taking information and turning it into compelling stories and arguments. The data is just another kind of document in this vein. In my research, I began to imagine how to build OCR pipelines, optical character recognition pipelines, that would unlock all the bureaucratic archives that, like you, I've seen for years in dusty rooms and had no idea what to do with them.

[13:38] Stacks and stacks and stacks of forms about people from the past that I didn't know what to do. I didn't know how to organize millions of documents at a time. They were abandoned. And now with the different kinds of collaborations, I'm trying to figure out how to bring them back to life with my collaborators as well. Now, of course, like everyone else, I'm also confronted with the question of how to teach reading and writing with a plagiarism machine just so easily available to all of us. And I worried about what our students would learn, but a machine would happily do the learning for them. I was at once incredibly excited. and deeply concerned. And you were too. You should be. You still are. I am. I am every single day. That is why we are all here for the next two days. We're going to spend time thinking about what we can do with these new tools to ask bigger questions about history, not faster questions, but bigger questions that we

[14:38] never could have answered in the past. At the same time, think about how to restructure our classes in ways that don't dilute pedagogy, that hold to our core values. In fact, strengthen what we are able to do in the classroom by focusing on the basic, most powerful questions of history. It is easy to fall prey to nihilism, to think we will be replaced, to think that our history will be written by a machine. but what we do and what we teach is more important than ever before. When I speak to my students, I speak to their fears. I speak to their hopes. They're afraid they're going to be replaced by a machine. And I say, we have struggled for hundreds of years with this question, the relationship of the human and the machine, as you all know. AI is not an oracle. It is a tool. And they tell us that AI slop is destroying truth. Well, we historians have made a discipline out of sussing out

[15:43] the lies in our documents, the biases, the fabrications. We live for that, right? You love to see a bias source. There's something interesting going on when it's not true. AI slop is just another Tuesday for historians. Our discipline's more relevant than ever before. We teach discernment. They tell us AI erases our connections with archives. Well, we historians now have a way to make data pipelines from digital images all the way through analysis to our articles and books. And so more than ever, AI assistance with the code, code which is inspectable, reproducible, deterministic, our evidentiary paths can restore trust. how we went from the original to the argument with code, with all other ways of channeling those data pipelines. We can do this with AI's help in a way we could never do before.

[16:45] What do we teach? We teach evidence. They tell us there's no way to trust the answers given by an AI. Well, we historians have worked with teams of transcriptionists and editors and synthesizers for decades. At least we did when we had funding. We put together vast collections of presidential papers this way. We know how to have layers of verification, adversarial AIs, to maintain provenance, to create narratives together. Whether with machines or with people, we teach collaboration. They tell us the AI can replace your thinking. Well, for generations, we historians have taught our students to read, to write, and most importantly, to question what we thought we knew about the past. No machine can desire an answer.

[17:45] No machine can imagine a new question. No machine can do history without a historian. We teach fundamentally what it is to be free. And now we are going to figure out together what it means to be a historian in the era of AI. Welcome to Baltimore. Welcome to Johns Hopkins. Welcome to the conversation that's the most important conversation we are all doing, both as historians and as citizens and as people, I think, and as humans in the year 2026. I look forward to meeting you all in person and for those online on Discord. Thank you so much. And now I'd like to introduce our next speaker, Marnie Hughes -Warrington.