“Historians On”: AI in Teaching and Research

Daniel Story

Welcome to History in Focus, a podcast by the American Historical Review. I’m Daniel Story, and this is the start of series of bonus content that we’ll release over the next few months. Back at the 2024 AHA annual meeting in San Francisco, historian David Trowbridge sat down with a handful of attendees to talk about topics of particular interest to historians in our present moment. We’re calling this our “Historians On” series, and this first installment covers the topic of AI and its implications for history teaching and research. David spoke with historians Katharina Matro, Jeff McClurken, Kalani Craig, Jo Guldi, Johann Neem, Kevin Gannon, and Lauren Tilton. So without further ado, here is David with “Historians on AI.”

David Trowbridge

During the AHA I spoke with eight historians about their views on AI. As you might imagine, that resulted in many hours of conversations centered on the possibilities and potential threats that AI could present to us, both as teachers and as researchers. I’m David Trowbridge, a historian at the University of Missouri at Kansas City. I enjoyed my conversations with fellow historians,, and I worked to distill about six hours of recordings down to about 50 minutes, divided into two sections. The first section is centered on teaching while the next one is focused on research. Our conversations on teaching brought up chat bots and ChatGPT quite a bit . . . So I thought I would start this episode with a short conversation I had with ChatGPT about some of the concerns expressed:

Trowbridge: Hi, ChatGPT. I’m a historian. I’m a little concerned about you. What would you say to me?

ChatGPT: It’s completely understandable to have concerns about AI like ChatGPT, especially in fields like history, where nuance and accuracy are crucial. What specific concerns do you have?

Trowbridge: Research, analysis, and writing . . . these are essential human endeavors. And they are very hard. What happens to humanity, if there’s a way to generate something that appears to have been the results of these very difficult steps without all of the work.

ChatGPT: You raise a very important point. The ease of generating content through AI could lead to a devaluation of the hard work that goes into research, analysis, and writing.  If we become overly reliant on AI for tasks that require critical thinking, empathy, and deep understanding, we might risk losing those essential human qualities. How do you envision a future where humans can harness the benefits of AI while preserving what makes us human?

Trowbridge: I suppose there would need to be some safeguards. If we get to the point where we are losing our humanity, can we go back to a world without AI?

Hal 9000 from A Space Odyssey: I’m sorry, Dave. I’m afraid I can’t do that.

David Trowbridge

Okay, so everything but that last part was absolutely generated by AI. I did edit it down quite a bit. And I did the same thing with conversations with colleagues, because I ended up with about six hours of recordings at the AHA. After that. I spent another 12 hours trying to distill that down into one hour. The first historian I spoke with was Katharina Matro. She has a PhD in Eastern European history from Stanford and has been teaching high school for eight and a half years.

Katharina Matro

The main thing that history teachers are talking about is plagiarism, cheating with AI. And we started seeing it pretty quickly last year. So we talk about that, and we’ve changed our policies, and we now talk about plagiarism. And then there’s a whole EdTech community that has thought of ways in which teachers can use this to their advantage.

David Trowbridge

You mentioned that you talk to students about this directly.

Katharina Matro

They’re a little bit fearful about using it they are given so many directives by adults in their lives, that this is just another thing that they’re being told not to do, I don’t know that they intellectually think about, you know, why writing yourself is a good thing. I don’t think they buy into it. Reading ability went down during the pandemic, attention spans went down, all of this. So they really struggle with writing.

David Trowbridge

How do you build goodwill and trust with your students that your classroom is a place for learning? That they’re going to be, they’re going to be better for it and that your classroom is also a place where they can trust you to produce work that maybe on its surface doesn’t look as good as what ChatGPT might produce.

Katharina Matro

Yeah, I think, I think the relationship, right, that’s the main thing, really reassuring them that you are, as a reader, interested in their voice. And that you are there to help them polish it afterwards, but, but you’re really interested in their voice and their thoughts. If you are interested in becoming a better writer, let’s do X, Y, and Z the next time you write something for me, but I’m not going to take off points or something. And so I’m hoping that . . . that creates some trust, and also if I talk about, I’m so excited to learn about what’s in your brain, because that’s why I’m here.

David Trowbridge

The next conversation I had about teaching an AI was with three historians simultaneously.

Jo Guldi

My name is Jo Guldi, and I am professor of quantitative theory and methods at Emory University.

Jeff McClurken

I’m Jeff McClurken. I’m a professor of history and American studies and chief of staff to the president at the University of Mary Washington in Fredericksburg, Virginia.

Kalani Craig

I’m Kalani Craig. I’m an associate professor of history and the co-director for the Institute for Digital Arts and Humanities at Indiana University Bloomington.

David Trowbridge

Jo started to us out.

Jo Guldi

I think our friends in digital literary studies are always a little ahead of the curve. And they’re embracing opportunities for creativity in the classroom. I think, you know, history sits between humanities subjects and the social sciences. So this will be more relevant to some classrooms than others. But if ChatGPT is the next generation of a word processor, it’s a tool that helps you write, to imagine, to make pictures, to make films, it can even give you shortcuts for how to code, how to take lots of primary sources and then turn them into a dataset, even though you’ve never done it before. It’s an opportunity to open the door of creativity in the classroom and simultaneously raise the bar for the quality of the work we expect for, from our students. Public historians have been so creative and so fertile with these technologies, like, basically making video games out of primary sources to help us explore, but I get so nervous when there’s a digital project that blurs that line between the fact and the primary source. Because at the end of the day, what separates us from the film studies department is that we know when we started imagining things, and we hold each other responsible for saying what really happened and how we know, we can overcome limits. Difficulties with English as a second language. Overcome the novelty of different ways of writing. It can be an opportunity to raise the bar in the history classroom.

Kalani Craig

I think what I heard in Jo’s response there was also the idea of play. There are these really structured approaches to the way we teach history in the classroom and sometimes play gets lost in that. ChatGPT gives us both the opportunity to do some nicely structured writing support. But there’s also room to play. Students who are thinking about creative projects, who are struggling to articulate how they want to do that creative project, and they have ideas, and those ideas aren’t fully formed. And having a chat partner that isn’t judgmental is a really valuable way for them to play with those ideas, and then, as Jo notes, take them out of ChatGPT and really apply their own thinking.

Jeff McClurken

Yeah, I think I’m particularly excited about this notion of students as creators. I think there’s a, there’s a bit of a potential fallacy here in the way that people are approaching generative AI. And, and that is, I think, they overfocus on the replacement for thinking or the replacement for creativity. And in doing so, their fear blocks them from seeing the potential for the kinds of things that Jo and Kalani are talking about. We have to understand that right now, we’re really looking at the worst version of AI we’re going to look at in the future. Right? This is only going to get better and better. So we have to be careful about structures or changes that are likely to be overcome by the technology. I don’t know how long it’ll take David to edit this, but it’ll probably already have changed by the time he does.

David Trowbridge

Yeah. So I have to add Jeff was right here. It took me a long time to edit this, and sure enough—the technology did get better.

Jeff McClurken

This is a fast, fast evolving area. But, but I think there, right now, the detectors that are out there have lots of false positives and they tend to discriminate against people who are not, do not speak English as their primary language. So, so we need to acknowledge that in the process. But what the Mary Washington Task Force came up with was just some recommendations in terms of academic integrity and ways to try to deal with AI. So, things like scaffolding your writing assignments, right? You can see, as you sort of work through the semester. It’s good practice anyway, if you have time to do it. A little harder to do in those larger lecture classes. But you can scaffold your writing assignments. You can break those things up. You can see the, the evolution of an argument. You can get a chance to give feedback on that. And you’ll get a sense of that final draft. It will not have, you know, if it comes from somewhere that does not match with the previous floors of the scaffold, you know, you’ll know. Run your own prompts through AI. See what kind of responses you get. See so you have a sense of what they are. Use assignments that incorporate personal stories, class discussions, authentic situations, sources and citations. Promote discussion and students sharing about their assignments. Incorporate things from class discussion. Incorporate things from from what’s been going on and have the students share with each other. And then really promote library resources. And then I think the thing that may be the most important about the recommendations from the task force was, was model productive uses of AI tools to help students become AI literate. And, and there’s a—I think there are some faculty who see that as a, as a wrong, like we shouldn’t help them be better at it. Right? But I think that’s the wrong approach. They’re already using it. So if you can, you can provide some of the things that Kalani was talking about, I think, in terms of using it for brainstorming, using it in particular ways. And then I think the thing that I do in my classes is that I tell students they can use it. They just have to tell me how they’re using it. They have to share their prompts. They have to, we have to talk through if they have any questions about whether or not the way that they’re using it is problematic. I think that there is a problem when our assumption is not to trust students—that you set up an adversarial relationship. And I think if you—as you were saying earlier, David—if you can build trust in a class, then you can have a meaningful conversation about how these tools fit in.

David Trowbridge

This brought our group to an important point that we spend a lot of time on. And that is the opportunities for incorporating AI into the classroom aren’t equal. There were so many scenarios from theoretical small, upper division class to a large lecture hall where most of the grading might be done by TAs who are students themselves.

Jo Guldi

Well, I want to say, one of the things that I’ve been hearing at the conference is a lot of concern from junior historians about this very question. And they’re telling me that while they’ve been thinking about GPT proofing their syllabus. So you ask questions at a higher standard of detail and originality, work with primary sources, create a fluent argument and essay. One of the best experiences of this conference so far has been sitting down and talking to Louis Hyman about how he’s using GPT in his classroom. The first day of class, he has been going to GPT and saying, “Answer this basic exam question about the history of capitalism.” And he gets the answer and then he says, “GPT, revise this answer for the following reasons.” And then he prints out that answer and hands it out first day of the seminar, first day of the lecture. And he says, “You’re at college to learn how to do better. We are raising the stakes for the written language and for written argumentation as a culture because we have this technology.” I’ve been teaching code, and working with textual databases, and searching and finding patterns over time for years. But what in over at least the last year, what he’s been teaching is a business history class where the students start by scanning the letters of Samuel Gompers, so they’ve got a primary source base from the archive that not a lot of historians have worked with, so they scan them, then they send those images via GPT to be OCR’d. You used to need to have two semesters with me of text mining for a historical method to do this. But he’s doing it in a series of GPT prompts. And by the end of it, they’re doing statistical regression to find the trends over time, and they’re creating data visualization—”Computer, put this on the x axis, show this, show this line as a dot plot in blue,” and in one semester, they cover basic statistics, they cover text mining as a historical method, they cover what is an archive, and they produce results which create new knowledge and labor history. Now that, that used to be four separate courses a year ago.

Kalani Craig

The collaborative nature of writing with ChatGPT as an author is a good lesson for all of us. But that changes—and here appears me being optimistically naive, I think—in some ways, that changes the relationship that historians have with each other. We struggle with collaboration still, and I think this is one of the positive things takeaways for ChatGPT is that if we can help our students think about ChatGPT as a collaborator where they’ve got to expose its authorship and theirs. Talk about prompt engineering as a skill set that is valuable in the archive, because you’ve got to know what keywords to look for. You have to know how to narrow things down. You have to know how to revise a question. And knowing how to ask a good historical question is the first part of knowing how to answer a good historical question. And so I think that there’s this fun balance between how do we think about it as play, how do we think about it as scaffolding our work, and making it more efficient and effective. Being creative about how we think about chatbots as co-authors or collaborators, but also figuring out how we can acknowledge where those gaps and problems are as historians, because that too is a question about history.

David Trowbridge

I want to bring us back to a couple threads that have been woven through the previous conversations. One of the things that has consistently come up among our colleagues is the value of the work. Reading for content, organizing that information, and then writing and revising. And how you write and revise at the same time. That’s the really hard stuff . . . writing. There’s nothing easier in the world than not writing. And there are few things harder than writing. The questions I keep coming up with is about AI, ChatGPT, and all these new technologies. Is the value of them for our discipline that they’re going to kind of force us to teach the way we’ve always known we’re supposed? And that is that history is a process and we’re going to do it with you. What you’ve described, Jo, is some amazing process-based assignments in a class that could never be replicated in the form that a lot of our classes are in. Because there’s a business model that values the economy of scale of a large lecture hall or a large online asynchronous course. But that efficient model is based around the idea of assessing students solely on a final product. And if that final product can be generated, maybe AI will compel us to have small classes.

Kalani Craig

One of the things that I’ve heard several times—not just from educational researchers but from our own Instructional Teaching and Learning Center—is that students struggle to start projects when they feel like they’re going to fail. So I think the question is: How do we get them past the scary blankness? How can they use ChatGPT to help them get past the scary part, so that the hard work doesn’t seem as though they’re going to do all of it, and then still fail because they don’t know what they’re doing?

David Trowbridge

This point about using all of the potential tools at our disposal to help students was central to my conversation with Johann Neem at Western Washington University and Kevin Gannon at Queens University in Charlotte.

Johann Neem

I think one of the positives is we’re forced to ask ourselves where and how are students learning, and what do we want them to learn, and are some of the things that we used to do the most effective or the most relevant things for our students to be learning to do.

Kevin Gannon

Yeah, to Johann’s point, I think those scholar teachers who have dealt with—I don’t know, I don’t know if “successfully” is the word but with the least existential angst—have been those who have realized that our disciplinary habits of mind that we’re trying to instill with students in our courses—anyway, are actually very well suited to a landscape that is at least generative AI tools. So what does it mean to be critically literate about information? What does it mean to actually interrogate the tools, the methods, the questions that you’re asking? And again, you know, this conversation that we, you know, have been trying to have for years, but has now become more urgent, you know, about assessment, right? What are we asking our students to do? Why are we asking them to do it? And are those things actually the best things for us to be doing?

David Trowbridge

So do you think that AI, despite all the challenges that we’re facing right now, do you think in the end it might compel us to teach the way we’ve always known we were supposed to?

Kevin Gannon

I think AI, in many ways, is sort of chickens coming home to roost, right? We know what it means to do assessment effectively. We know what genuine learning looks like. We know what that process looks like. The problem is, many of us are at institutions where we can’t really do that, because of class sizes, because of that justification. And so all of the things that were, you know, we, speaking broadly, right, are now lamenting, you know, AI might be the end of X. Well, it’s the end of X because we’re at places that are not adequately supporting X. Right. And so how do you know? How do we close the gap between what we know we can do effectively and what we actually have the resources and the bandwidth as individual instructors or departments to do practically?

Johann Neem

I think that’s a really good point. I think one of the things that we know is that the kind of habits of mind and historical thinking skills that we care about require students to have a lot of interaction, a lot of feedback, and in some ways, if the coin of the realm was the academic paper for assessment, you could do that at a certain scale. I mean, evaluating and assessing academic papers is a labor-intensive activity, but there was a certain scalability. But if now the coin of the realm is learning to read critically, learning to speak and have conversations that are thoughtful and critical, you need a human-scaled classroom for that. And so there’s ways in which, you know, the first reaction is the existential crisis that AI will mean the end of the humanities classroom. But the reality is it means that we need actually more and smaller and more intensive humanities classrooms.

David Trowbridge

What are some of the things that you’ve seen in the near term that someone in a situation like that—you’re teaching a large online asynchronous class, you’re teaching a large lecture hall, you’re a TA who’s grading a large number of papers—what can they do now to alleviate at least some of the issues they might be facing?

Kevin Gannon

Well, that’s a great question because you know, in one sense, the answer is not enough, right? And that’s just sort of the reality, but that’s also not a very satisfactory or helpful answer to folks who are in this situation. And what I would say—so there are a few things that we can put into place right away that would be useful. One that I can think of right off the top is: How do we make the process of writing for our students more accessible and one that feels more collaborative and social? You know, we know learning is social, right? So how do we build in peer review in our classes? You know, we know great practice for teaching writing is to conference with students throughout the writing process. So are there ways that we can do that where, you know, maybe we use, instead of holding class, we actually make that day a conference day for individual or small group students to come in and talk about a paper, for example, that they’re working on. You know, so when we, when we emphasize the, the sort of social presence and we emphasize the collaborative and the process aspect of this, we make it less likely that students are going to go to ChatGPT and say, “here’s my paper,” right? So that’s, that’s one strategy. The other thing that I can think of as well is get students experimenting with the tool. The students that I’ve worked with are telling me is, you know, ChatGPT is a thing, but there’s not a lot of specific knowledge about what it can and what it cannot do. And so there’s this perception that, well, you feed it a prompt and it spits out a research paper. As historians, we know how to analyze things critically. So let’s get into this, in order to help our students do that sort of work as well. I think demystifying it and turning it, you know, this is sort of like Wikipedia in that sense, right? It is a resource. It is a thing, but it’s not going to do the work for you in the way that you think it is. I think that is a really important step that we can take.

David Trowbridge

One of the class experiments I read about and I tried . . . it didn’t quite work out for me the way it was supposed to. Generate a few different essays and have your students grade them. And of course, one of those essays ended up being AI generated. And the idea is that the students will be able to recognize very easily that the AI essays are, are very vague and sweeping, and they will grade the honest attempt much higher. Because it brings in specific examples from the reading. I wanted that to happen and it wasn’t my experience. The AI essay got a much higher mark. And the one that was me imitating what I expect a college student to write about this particular essay prompt . . . they, they gave it pretty low marks. And I asked them why. And the answer was pretty clear. This one looks better.

Kevin Gannon

That’s exactly it. But then we have to ask our students, well, why? What looks better? What makes it look better? What are the moves that, you know, the quote unquote writer made or did it make compared to this other sample that made it look better? What are we defining as good writing or smart? I mean, this is a real opportunity for us to have those conversations with students. Like, what does it look like to learn? You know, what does scholarship look like? You know, it’s not just, you know, fully springing out of our heads fully formed like, you know, Minerva, right? And we don’t have those conversations with students enough as it is. I think we do have an opportunity now in an activity like that to have that kind of conversation where if students are ringing in like oh, this is clearly the superior essay. Okay, but you know the citations are made up. And it doesn’t really say anything, although it does say nothing nicely, right? Getting students to sort of critically interrogate, you know, what are, what are the rhetorical moves we make? What does scholarship look like? What does good writing look like? And most importantly out of that, getting students to see themselves as people who can do good writing.

Johann Neem

Building on that, one of the things I’ve noticed since I’ve tried to adapt a little bit—and it’s still a work in progress in my introductory courses in the wake of ChatGPT—is I did do, the decision that people who want to embrace learning how to use ChatGPT don’t do. I decided I was going to go back, I mean actually maybe for the first time in my life, and have in-class blue books . . . I call them reflective, reflections. And I made the questions really about what they’ve learned, and how they’ve learned it, and what are they taking away from the readings. And I got much more thoughtful, much more creative, and much better answers. And I think the students learn more, and I think the reason is when they try to write those formal papers, they’re writing in a voice that’s not their own. The purpose, as I say, this reflection is for you to explain to yourself what you’ve learned and why it matters. And so you’re writing to yourself. And they used language much better than when they tried to sound like someone who had a PhD. And this other kinds of writing that are a little more personal but still require the use of evidence, the formation of an argument. You know, I think one of the things that we have backed ourselves into in the humanities is—not unreasonably, I should say, and not without integrity—is that we teach people how to write and writing is an important skill, and this had a kind of economic justification. But we’re going to live in a world of so much computer-generated information, so much noise, so much mis- and disinformation, that I think that whether or not you’re focusing on writing, teaching students to read texts critically, really unpack them, to learn how to engage with each other, to evaluate and assess ideas—these are both historical skills and civic skills that are extremely important. And they’re time consuming and resource consuming, but at the same time, they’re the most important things we can teach our students right now because they’re gonna be flooded. Our entire world, God help us all, is gonna be flooded with machine-generated information.

Kevin Gannon

And on the purely crassly pragmatic side, employers are telling higher education, we want graduates who can do all of the things that Johann just outlined, right? And so how do we do that? Well, we’ve got a ton of research in the scholarship of teaching and learning that talks about things like small class sizes, relationship building, learning as process rather than product. And the other thing that we can point to is when we create teaching and learning spaces that prioritize those things over simply content delivery, persistence and retention go up. And if you’re in an institution that is largely or even partially tuition-dependent, like almost all of ours are, persistence and retention are the magic words.

David Trowbridge

As we talked about towards the end of our conversation, those magic words do have a potential of unlocking some resources. When I was talking with Jeff, Jo, and Kalani, Lauren Tilton’s name came up several times. So a little bit after the AHA, I recorded a conversation with her. She’s the E. Claiborne Robins Professor of Liberal Arts at the University of Richmond.

Lauren Tilton

One thing that we’ve been doing is using a lot of computer vision methods to look at historic archives of photography. We’ve been taking technologies like computer vision, a type of AI, to look at those photos and find things in them, so that we can open up these collections in new ways.

David Trowbridge

History can not only offer perspective on, on tools and technologies and how they’ve been used, but also we can use these tools and technologies to discover more histories and make connections that we might not have been able to do without, without these tools.

Lauren Tilton

Yeah, exactly.

David Trowbridge

We’re reading a lot and hearing a lot about how how students are using AI to avoid one of the hardest things on earth, which is writing. What can we do?

Lauren Tilton

Yeah. So one of the things I’ve been thinking about is how these technologies are both, can augment and also are challenging how we teach right now. So, one of the big concerns is you give a student an assignment and they just type in a prompt. I think the bigger question for historians right now is what is the goal of a history course? What are we trying to do? And I subscribe to the idea that it’s about historical thinking. Are we asking questions about the past, and are we asking about how we came to our current moment? And do we understand the ways that we can do that? And so one of the things I’ve actually done more of is diversifying the kind of assignments that students do. So I have students who now do podcasts. And they make graphical novels and comics, and they also will might make digital essays or videos. I’ve made a lot more of my assignments multimodal, so that they may engage in that kind of historical thinking. In the ways that they may actually be really excited about engaging with the past. As well, you know, you’ve got tools like Clio, your digital tools, right? Like having students work with these digital technologies. That help them have a voice in the way that they, the stories about the past. I think a kind of older pedagogy, which was sort of the classic five-paragraph essay—this is going to be a polemic comment, but I have found to be a sort of outmoded model for a long time, and not just now because of generative AI.

David Trowbridge

Yeah, I don’t know that the five-paragraph essay ever really worked, but it certainly does seem to be working as well as it once didn’t work. And there are, there are very few instances where I think someone’s going to give you five paragraphs if you don’t grab their attention pretty quickly. How do we build trust in the classroom so that students will give it the old college try and, and, and go ahead and move past the tyranny of that blank page and write like themselves?

Lauren Tilton

Yeah. That’s a great question. One of the things that I’ve done a lot more in my class is a lot more scaffolding for writing assignments. I do a lot more workshops now, and in-class activities where students come in, and they have to identify their primary and secondary sources, and then we workshop those. And they work in groups to talk about what kind of evidence is the source providing, what’s the historiography. Then they come back with another class with an outline for their assignment, and topic sentences, and the ways they want to move through their narrative structure and we workshop those as well. So, one of the things I’ve done is, I agree, is that really working on them finding their voice in their writing, is making it a process that is much more iterative throughout the semester, where we really walk through each of the stages of the process. So it’s the way that I’ve really tried to do that is have a lot more scaffolding in the course, so that students start to trust their own voice and hear each other’s different voices and approaches to it. And the other thing I’ve started doing is just assigning a lot more different types of sources.

David Trowbridge

So that brings me back to the conversation with Jo, Jeff, and Kalani.

Kalani Craig

So I think one of the things that we also want to address when we talk about classroom work is that there are no single classrooms. And student responses to assignments and standards are going to be really different. So some of this, yay, historians and our context is about the contextual structures in which we live and work. And so I think what’s been really valuable for me to think about this is having the departmental chats with people who have classrooms like mine and are thinking through this in very different ways because they have different approaches to teaching, and encouraging more of that conversation. Contact with teaching and learning departments, talking to colleagues who teach your students and figuring out what kinds of values you share, what kinds of values your students share, and how you can fit some of these adaptive play approaches or scaffolded learning approaches that chatbots can afford into your classroom, matters. Because I think we also have to meet our students where they are. We’ve been talking about that a lot, but it’s worth underscoring that where our students are is all different, all across the spectrum, in all of our different classrooms.

Jeff McClurken

I was having a conversation with somebody about ChatGPT and how it was potentially changing teaching and learning. And this person said, well, it sounds like they’ve democratized cheating, and to some extent that’s true. There’s an ability and an access to a set of resources that allows a sort of minimum level of response. And we can critique that response in a number of ways. I think which classroom you’re in makes a big difference about whether or not that it, that kind of work is acceptable. But it’s also not truly democratized, because there’s a paid version of ChatGPT, and there are paid versions of these other AIs that exist—some people will have access to them. Those with money will, and those without money won’t. So I think there’s another level of a kind of digital divide, that we have to reckon with. The other piece of that is that we’ve been talking a lot about ChatGPT; it’s one of the fastest growing technologies. But it is one of many—literally there are over a thousand of these AI tools out there—that go beyond the large language models and the generative AI. There’s a bunch of directories of AI tools out there—you could say there are a couple thousand of them out there. And we need to understand that even if you’re not using one of those formal platforms, you are using AI. All of us are using AI, because machine learning and AI-enhanced tools are already built into many of the products all of us are using every day, right? So whether it’s predictive text on your phone or it’s the search in Google or Bing, or in email and writing systems like MS Office with Copilot and Google Docs with their integrated AI. We’re all already using AI, and so that’s part of why we need to be grappling with it, because the people that we’re teaching, the people that we’re working with already are.

Jo Guldi

One other theme that comes to mind when we’re talking about all of this is, of course, part of our value added as historians is thinking about what’s in the archive and what’s not in the archive. So one of maybe one of the first questions that historians might have for GPT is what’s the text base that’s been digitized? So it’s everything that was on the web, that was web scraped, which includes a lot of code, it includes a lot of popular culture, it includes a lot of books which aren’t necessarily in the public domain. There are legal issues with that. I think we don’t know enough about which journals, what periods of history, are covered. I’ve played around—and this is something that I would do in a classroom—I’ve played around with the question, how does the bot answer questions about Abraham Lincoln? And they do pretty well. Like, you can ask questions about his relationship with his mother. They answer with a lot of detail, because the American book industry loves Abraham Lincoln, loves Civil War history. Ask it about the history of land reform, the history of global land reform, which isn’t something that there’s been a lot of research or writing about since the 1960s. You tend to get a kind of Maoist answer about the successes of land reform in China, which is, which is interesting. It’s coming from somewhere, and you think, where are these documents? What are those documents that the bot has read? And this, that kind of like, archaeology of the technology via the archives, or the archives via the technology. I think this also makes me think about Matt Connolly’s arguments in The Declassification Engine about the disappearance of the archive in the digital age. He says that for US political historians for the 1990s, we have one third of the documents that we had for the 1960s. He says when email arrived, the archives basically became paralyzed.

Kalani Craig

One of the things that in an ideal world we would see are people like us in this room, with expertise in both worlds, talking about user interface design choices that show you where the gaps in the data set are right now. Jeff talked early about the idea that in the probably very near future, a lot of the chatbots will be able to produce citations, but I think the question is more deep than that. It is that any given word in a chatbot response or any given detail in an image generator, like mid-journey. It comes from somewhere and we should be able to mouse over any given word or image or feature in a generative AI response, and see where the data came from, because that’s the only way we’re going to know where the gaps are. And that is a really crucial piece of the puzzle where historians fit into this world of AI chatbot building in a way that I don’t think any other disciplinary faculty can. Like this is our concern, and we really need to claim it.

Jo Guldi

I’m going to put it in a plug for my most recent book, The Dangerous Art of Text Mining. The thesis is that data science needs historical thinking. In particular, it needs to be smarter, and I have a kind of soup-to-nuts approach to what that means. But it starts with the occlusions in the archive that we have the set of questions about what’s been left out, and that’s increasingly vital. We’re at this funny point in time, in which it might be more difficult to find historical sources from the present than from the Middle Ages. That’s a new kind of challenge. And I’m still wrapping my head around what that means for us, and what it means for our relationships with archives and archivists and information schools, and political offices where the money to support archives and research lie. You know, it may be that we have demands that we need to make in terms of digitalizing and preserving the record of the now.

David Trowbridge

And so that’s an important, important point because, you know, the assumption is always that the tools will just get better and better and better. But the data set might go in the other direction. Especially as AI-generated content enters the public web. What can we do to support archivists and other people to make the data set as representative as possible?

Jeff McClurken

First of all, AI is driven almost entirely right now by giant tech companies, giant, very wealthy tech companies, and the platforms and the infrastructures that they have built. And their motivations are not our motivations, right? Their motivations are driven by profit, driven by finance, driven by power and control. As historians and as teachers and as archivists, right, we can and will adapt the tools that they’re creating to the work that we do. We have before with other tools that were not intended for us either. We need to propose the priorities that we think are important. Things like citations are just the beginning, I think. It means that we need to push them to use their vast resources to be able to add these occlusions from the archive into the larger set. Lauren Tilton was interviewed for an article in Technology Review where she talked about how we’ve made this remarkable progress in the ability of AI tools to recognize visual things. But what it recognizes are things from the present, not things from the past. So there is this bias towards the materials and processes of the present. And that’s an incredibly important thing to understand. The same is true for the bias towards bias. If what these tools are doing, is taking the record of the public web, well, it is taking in all the biases that that exist on that web. And so we have to be aware of that as much as possible. If there are opportunities to provide and add to larger archive, it’s our duty to to work with archivists to advocate for those additions.

Kalani Craig

I also think it’s our duty to think about the fact that historians have always dealt with and interpolated into gaps. And I’m thinking particularly of some of the scholarship on formerly enslaved and enslaved Black people. Jasmine Sutton, Marina Myers work on filling in gaps in the life and lived histories of enslaved women who are often not named in the historical record, who show up not as, in many cases, people but as property. There is a vast array of our colleagues who deal with this on a regular basis, and I think this is a real opportunity to incorporate more of that work into what we all do as historians, to learn from our colleagues who are thinking about gaps, because I don’t think we can ever fill all of them. I do think we should try. But I also think that there is a wealth of skill and valuable historiography and methodological innovation from folks who are doing that already. And it’s worth, again, I think, thinking about the fact that historians synthesize things that aren’t fully complete. We know how to combine data sets, we know how to think about what happens in the gaps between those data sets, and that is something that any chatbot is going to really struggle to do, even for, I think, the foreseeable future.

Jo Guldi

I think another thing that’s useful to bear in mind, if we’re going to solve the problem of missing archives, it’s not something that any heroic individual is going to do. It’s something that we’re going to do together as a discipline. Two precedents come to mind that are really useful for this time. One of them is grassroots, one of them is more top down. But they both have collective elements. Scholars of digital literature have been forming these data collectives. And they get together and they make a list of what are the data, The data sources that we would really like to go after, one of them famously was the Penguin Random House Archive of American Novels. And that data set will support all sorts of analyses—it’ll support dissertations on the career of Toni Morrison and inquiries about race and gender and the transformation of the 20th century, and so many more. And so they all got together and decided to pool resources. They wrote SSRC grants. They wrote NSF grants. They wrote NEH grants. And they planned resources so that they would have the tech support they needed and the archival support they needed to host this data and make it accessible so that all of them could get data driven results together. And the result is that they are flooding the journals with analyses. So, it’s a group of historians who we would expect to make a data collective like that available, and I think we need to be forming data collectives with other people who have substantive interests that are the same. The other example is Tomptown, and it’s on a nationwide level. So in Finland and the Netherlands, and I believe Germany, the historians have organized themselves into a group that I believe is called Daria. The Daria projects are historians from 20 departments, and their chairs, and their deans, and they’ve been convening to write, with archivists, to write a letter to the national political bodies and the national archives and the national funding bodies saying, hi, we are the discipline of history, and we have some priorities that we really want you to take seriously and what we are concerned with would, you know, all of our big hopes and dreams or maybe 5 percent of the biology budget. Do you think that we’re worth 5 percent of biology, 5 percent of physics? This is what we need. So they start making a list of, if we’re in Finland and we had to rank the most important archives to digitize, and we want to have respect for temporal diversity, like the Middle Ages get a say, we want to have some respect for geographical diversity, and also for the ethnic diversity and the diversity of languages in the archive. But none of those constituencies of the discipline gets to drive the conversation. They all get represented. What’s the list of archives that matter the most to us? And then, what could we pick? What could we do with that? We’re going to need to pay some archivists, we’re going to need to develop some standards for what is good digitalization, what’s good OCR. Historians can imagine something, a transformation of the archives and the discipline around data with these concerns of the occlusions of the archive in mind. And that is not something that any one historian can do alone. It’s massively beneficial to historians and the cause of research but it takes some real organization. So I think we need to start taking inspiration from those models and thinking about using our organs, like using the AHR, using the AHA meetings, to start having these conversations with each other. It’s legitimately gonna be better answered if all of us participate in the plan and then in the discussion and we can envision a future discipline of history that represents all of us.

Jeff McClurken

That’s awesome. What Jo has described is amazing, and I think at the same time as, as we’re doing that work, the collective work, I think we also need to remember the value the work of individuals who are institutions that don’t have access, or the support, or the right, and who are, but are reproducing act, works of digital history that are making available things that were not available for. I think it’s both a grassroots effort and it’s a collective effort.

Jo Guldi

That’s right. Rubica Rosamon, her masterpiece New Digital Worlds, has all of these stories about Indigenous communities, digitalizing their own history, and all of the privacy issues that come up in that process. But it’s, you know, I think if our Indigenous historians show up with those concerns in mind to speak for those communities and to make sure that they are at the table of those conversations, that’s when the conversation gets really strong.

Jeff McClurken

Absolutely.

Kalani Craig

I think the takeaway is that AI and the questions that surround it are so much broader than most of us think, right? We’re so focused right now in the moment on chatbots, and is that going to result in me having to change my essay assignment? And I think that the outcome of this conversation says there are lots and lots of places where we can intervene as a discipline, where we can intervene as individuals to shape what happens with AI because there’s so many things that are complicit in it and that contribute to how it can or does succeed in some of the ways that we’ve talked about in ethical domains.

Jo Guldi

For the people on the highly technical end of the conversation about GPT, there’s a conversation about how you can get in there in the code of GPT and tweak the imaginative extrapolative factor versus saying, you know, tell me line by line only like near sentences that appear in the text. The Abraham Lincoln bot can only imagine things that he knows about. He can’t imagine space cities of the future in 2040. He can only talk about, you know, the events of his childhood and things that we have letters about. So, you know, being able to have this conversation about what the chatbots can and cannot do under certain circumstances is really useful. But I will say that from the world of text mining, that’s the kind of digital humanities that I actually practice. You know, it’s been very important to me to stay super close to what’s on the page when people in text mining work with computers. We’re not working with these extrapolative technologies that are trying to imagine or create other possibilities. We’re trying to ascertain the truth, so we count words at different periods of time and we want to know what are the words that were used in the 1860s, but not in the 1850s or the 1870s, that only Abraham Lincoln used and none of his contemporaries used.

Kalani Craig

Well, I think there is a nice lead in for me to be selfish and briefly say that, from the perspective of my research agenda, which is about building tools that have historiographic foundations, I think one of the things David has started with is the idea of this as a potential research agenda builder. And there is a value in us thinking about how we can go in and tweak those settings, how we can build our own chat bots, how we can reshape the ways that the chatbot works with large language models and with different source bases, not just for the purposes of addressing bias in the data sets or for the purpose of, you know, limiting the hallucinations, but because there are specific things that we ask as historians that aren’t going to come up in the norms that are built into computer programs, not built by historians.

Daniel Story

That was the first installment of our new bonus series, “Historians On,” with historian David Trowbridge. You heard David in conversation with Katharina Matro, Johann Neem, Kevin Gannon, Jeff McClurken, Kalani Craig, Jo Guldi, and Lauren Tilton on the topic of AI in historical teaching and research. Look for more of these bonus episodes over the next few months. History in Focus is a production of the American Historical Review, in partnership with the American Historical Association and the University Library at the University of California, Santa Cruz. This episode was produced by David Trowbridge. You can find out more about this and other episodes at historians.org/ahr. That’s it for now. See you next time.