Abstract

Song Chen shows how historians can move from sources to network graphs and metrics. Network analysis is a relational approach to data modeling, not only a visualization technique and not limited to ties between persons. He compares the kinship and marriage ties of Chinese prefects in office in the 1040s and the 1210s, and models temple cults and places in the Lower Yangtze Delta as a two-mode network built from 34 local gazetteers. In the second half he tests ChatGPT on extracting tables from a page that has not been run through optical character recognition (OCR) and on running community detection in Gephi, and he warns that the model silently set the modularity resolution to 1.0.

Chapters

Transcript

AI-generated transcript. It was produced from the recording by speech recognition and cleaned up with AI. It has not been edited by the speaker and may contain errors. The captions on YouTube are AI-generated in the same way. Each timestamp links to that moment in the video.

[0:05] Good afternoon. Good afternoon. It's about time, so let's get started. My name is Song Chen, or Chen Song. I'm a historian of Middle Period China. Roughly from the 7th century to 13th century, I teach at Bucknell University. And first of all, I would like to thank Bucknell University, the Institute for Advanced Study, and the Starr Foundation East Asian Studies membership for supporting my travel here. Now, my talk is called Network Analysis for Historians. In the next 45 minutes to an hour, I'm going to talk about four key questions that are largely two different categories. The first category of questions about the methodological and conceptual characteristics of network analysis, I'm going to ask several questions. What is network analysis? What can network analysis reveal that close reading may not? And what are the limits of network analysis in historical research?

[1:06] Now, to address these questions, I'm going to cite and discuss two examples, two case studies that have been done in the past few years in the field of Chinese history in order to illustrate what I think network analysis can and cannot do for us. Then I'm going to move on to the second part and talk about the relationship between so-called AI or generative large language models on the one hand and network analysis on the other. I'm going to talk about what tools historians can use without advanced programming skills and how is AI or generative large language models transforming historical network research. So let me start with the first example by asking questions. start with research questions. As a social historian of middle period China, I'm interested in looking at the social and political changes going on from the 11th century to the 13th century. So my question is broad. How did the political elite change between the 11th and

[2:10] the 13th century? And to understand that question, I'm going to start with a very small cohort of people called the prefects. These were local administrators At any moment, there were several hundred, three to five hundred across the country in that period. So these were important people that governed local society, but at the same time had the authority to report directly to the court. So these are the prefects. So I'm asking about, I'm looking at two cohorts. One is the cohort of prefects who were in office from 1040 to 1049, 1040s. And then another cohort, the other cohort, are prefects who were in office from 1210 to 1219. I'm going to ask, how different were they? So this is based on work that was published now 10 years ago. So we can ask questions in different ways. We can ask, how did these people enter civil service?

[3:11] So on the left-hand side, you have this diagram that shows, well, the great majority of these people entered the civil service not through any hereditary privilege, but through the civil service examination. So a slightly higher percentage of these people entered the civil service in the 13th century through the examination system. But generally, there's not much difference between the 11th century and the 13th century. Or I can ask a different question. How old were they when they were in office? And then we began to see some interesting differences. That is the graph that you see on the right-hand side. We see that in the 11th century, many of these people who were in office, were their 40s, 50s, 60s, and some of them were in their 70s. But once we get into the early 13th century, very few people reached that

[4:13] level of office in their 30s, some in their 40s. The vast majority of them were prefects when they were only in the 50s and 60s and sometimes in the 70s. That basically means the Korea trajectory, more and more challenging for these people. Or I can ask yet a different question. I can ask where these people actually come from. So I can put their places of origin on the map and contrast these two maps. In the 1040s, we see most of these people were from the capital region in the northern part of China. In the 13th century, very few people were from the North China or zero. China, that was because North China was occupied by the Jurchens, so that was not surprising. The new capital, the provisional capital, was in this area. But what was interesting in the 13th century is that the

[5:13] number of prefects coming from the capital region were relatively few. People coming from the capital did not dominate this cohort of prefects. They were more or less evenly distributed across a stretch of coastal prefectures. So we can ask different kinds of questions about this. And this is what we call prosopography, which in the 1970s was defined by Lawrence Stone, this British historian, as the investigation of the common background characteristics of a group of historical actors in history by means of a collective study of their lives. Wonderful study, right? So that means we're shifting our attention from events, institutions, or the prominent politicians or thinkers in history by looking at the common characteristics of a large group of people. But I want to ask yet a different

[6:19] question. This question, here I'm asking, how well connected was each cohort of prefects? through marriage or through blood. Here we see two network graphs. So each dot that you see there is a prefect. And if there's a line, that means these two people, these two prefects were connected through blood or marriage, father-son relationship or affinal ties. And what is interesting here is that we see something that we didn't see before that is more contrasting than the previous few graphs, right? Either the statistical diagrams or the maps. What we see is that in the mid-11th century, the vast majority of these prefects were connected to one another, weaving together a complex web of networks across the empire. By the early 13th century, that kind of a kinship network broke apart.

[7:24] This is what we call network analysis. And this tells us something more about prosopography. If prosopography in the 1970s definition is about looking for common background characteristics and a specific historical population. Now, we're doing a different kind of prosopography that is called by some scholars, like Dion Smythe, as the new style prosopography. in which we're looking at the structure of relationships that define, constrain, and empower each individual in a historical population. And we are looking at how the structure of relationships actually evolved over time. So this is prosopography in a new style because it's informed by what we call network analysis. So what is network analysis then? my first argument is that network and networks are not just relationships

[8:27] we tend to conflate these two terms but we really think about it networks are not the same as relationships relationships or connections or in chinese guanxi are an anthropological constant if you think of the human society there are relationships everywhere right teacher disciple relationships spousal relationships, relationships between parents and children, trading relationships between countries. But each dyadic relationship is not a network. When we're talking about network analysis, what we define as a network is a totality of relationships that are aggregated for analytical purposes. So in this sense, a network is not an anthropological constant that was not given. mental and analytical construct that we define for analytical purposes. Also, we assume that this totality of

[9:30] relationships, this aggregate of relationships, has some sort of structural patterns that can be identified, revealed, and described so that we can understand these different relationships better. So historically, at least in the Chinese case, we know that some sort of historical writers in China were talking, were kind of aware of this, and they would say, well, people had relationships, these relationships matter. And these politicians, they form a complex web of relationship that was like entwined roots and tangled branches. So they have the sense that relationships, dyadic relationships form a web, constitute a web, and that was really mattered for politics and for social action. But your assumption there is that it's so complex.

[10:31] You cannot describe what it's like. So it's so entangled. But the goal of network analysis is premised on the assumption And we should identify and interpret these structural patterns of these relationships. So speaking of structures, well, if you look at, if you flip into some of the pages in the standard social network analysis textbooks, it will tell you why you have these different idealized types, like a star graph, you have a circle graph, a line graph, and more and more. And if we think of these in the real-world cases, they're meaningful. If we think of, imagine that these are representations of email exchanges or business meetings between employees, among employees of a company. So each dot or each node here represents a person, and a line represents email exchanges. And we see that the first graph, the star graph, is

[11:32] highly hierarchical. N is the center, and he's connected to everybody else, and everybody else had only connections to this guy, receiving command from him probably. The second graph is probably more egalitarian. So when we apply to historical data, of course, historical data is much more complex. But it's interesting. Right here, I'm going to bring in the Tackett study of the 9th century political elite. And we're going to contrast that with my studies of the 11th century political elite and the 13th century political elite. It's clear that in the 9th century, there were two different cliques in this marriage network. In the 11th century, it's just a wide sort of web of relationships with no clear clique structures. And in the 13th century, everything fell apart.

[12:35] Now, this is what we mean by network analysis. I don't think network analysis is just about visualization, creating beautiful images. I think fundamentally, network analysis is about understanding the structural patterns of relationships in a human society. so now now my second the second point i want to make is that network analysis networks are not always interpersonal right we're here at the conferences we network with each other it's kind of natural for us to think of networks in terms of interpersonal relationships teacher disciple relationships, marriage relationships, and so on. But I don't think network analysis is always interpersonal. Network analysis is not limited, at least, to the study of interpersonal relationships. Network analysis should be taken as a relational

[13:38] approach to data modeling and data analysis. In other words, it is not defined by what you study, interpersonal relationships or something by how you study it. It's defined by a particular methodology. Persons, places, institutions, texts, events, and any of the entities' words, any of the entities that you can think of, can be modeled as nodes in the network and connected by explicitly defined relationships between these entities. Going back to the foundation of network analysis, which is the very often traced back to that 18th century mathematical problem that this mathematician Euler encountered. That's what's called the problem of the seven bridges in Königsberg, which is a city in Eastern Europe. So basically in this city you have like seven

[14:39] bridges. And Euler was asking this question, is it possible to find a walk through the city of Königsberg the seven bridges once and only once. Now, how do you solve that problem? He said, well, if you want to understand this problem, you have to reformulate the problem in abstract terms, essentially basically transforming it into a network graph in which the nodes are the land masses, different sections of the city, and the bridges are the edges connecting different parts of the city. And then you can use graph theoretical theories to solve the problem you're asking. So here, again, we have a network. But the network is not about relationships between persons. It's a relationship between different sections of a city. So in that sense, my broader understanding of network is that a network is a model consisting of nodes and edges that connect

[15:43] these nodes. and the edges represent their relationships. What counts as a relationship? That depends on your own research question. So because a model is an abstraction, the subject of a network analysis is not confined to interpersonal relationships. Any historical phenomenon that can be meaningfully represented through nodes and edges may be studied as a network. This brings me to my second example, which is a work I published later, like three years ago, as a contribution to a special issue on religions, on network analysis in the field of Chinese popular religion. So the context here is this. So, you know, China is kind of a polytheistic society. There is a many-to-many relationship between places or persons and gods. and deities, right?

[16:46] So people in the specific place worshiped multiple deities. And individual deities, each deity, each god, may also be worshipped across multiple different areas. So as one might expect, places that are close geographically, in geographical proximity, may have had many deities in common, while more distant communities probably had less overlap in their pantheons. The research question that I'm asking here is that based on what deities were worshipped where, can we identify distinct geographical areas that are similar in their religious profile? So because of the data we have, I'm looking at the 12th century through 13th century, roughly that period. We don't have a whole lot of data on this, at least not a whole lot of systematic data. So I'm sort of focusing on this area called Lower Yangtze Delta, which is the

[17:49] cultural and economic center of China at the time, for which we have richer historical sources. So this question, at first sight, might sound like a GIS question. If you are interested in what gods were worshipped where, just find the data and project them on the map. So I am creating like six different maps for six different gods. And we know that some of these gods are more local. Their devotees were largely from a very small geographically confined area. But for other gods, they had a much wider following across a much larger geographical area. So can we see geographical patterns from this? I would say not easy. And if we put them on, put all the, if we conflate all of these six maps together, this is what we get in the GIS map.

[18:53] It's so difficult to see any patterns. And in fact, here, I'm not talking about six deities. In my data, I have dozens, even hundreds of deities, depending on what deities you think are worth studying. So my solution to this is a network solution. Don't think of this as a GIS question. Think of this as a network problem. Because we are not interested in looking at the geographical patterns of a distribution. We're interested in the relationships between places. How similar, how dissimilar are these different places in Lower Yangtze? So what we're mapping here is not a network of persons, but a network that consists of two different kinds of nodes, cults and places, or deities and places.

[19:55] So if this god is worshipped in this place, you draw a line between this god and this place. So here, on the left-hand side of the slide, you see the network, where each green dot is the place where a god is worshipped, and the red ones represent temple cults or deities. And then we can apply community detection algorithm to partition this network into clusters for better understanding. And we can sort of present the results of the clustering algorithm as a matrix where places occupy the rows and the calls occupy the columns. And if the cults were sort of close together in one cluster, that means these cults were more likely than the others to be worshipped in the same set of places.

[20:57] And if the places were placed close together, that means these places have significant overlaps near pantheons. These are not like 100% overlap, but significant overlap. So this is how we deal and model this highly complex data. And then we put these places on the map. And then we began to see, if you see the colors here, each color represents a cluster to which that place was assigned in that clustering algorithm. Then we can see more or less consistent sub-regions of the lower Yangtze delta where religious profile was similar. So in doing all this, what we're doing is that we're not relying on one particular text. We're looking at multiple different texts. And in the second example, I'm taking data from multiple historical sources, local gazetteers from the Lower Yangtze area, 34 in total, looking at only specific sections

[22:01] that dealt with local cults and took the data out. So this is one page of the historical source. and I'm interested only in a few set of information there. The county, where the temple is located. The temple, the name of the temple. And the deity or the cult, the deity that was worshipped in that temple. So that can establish a relationship between the place and the cult. So, in other words, to summarize, network analysis is a relational approach to data modeling and data analysis that can be applied to diverse historical subject matters, from elite marriage networks to religious geography and to many other stuff. A second point I want to make here is that, like many other computational methodologies to historical research, network analysis brings together data that was scattered across different historical sources so that we gain a macroscopic, a bird's eye view of the broader patterns

[23:06] of whatever subject we're studying. Elite marriages, we're not looking at specific like this family or that family, this particular marriage relationship or that particular marriage relationship. And here in the second example, we're not particularly interested in one God and how this God was worshipped and who worshipped it. We're looking at the broader patterns of the elite networks. Now, this is not to say historical interpretation does not matter. Now, in fact, historical information, the historian's judgment should be part of every step of this data modeling and data analysis process. Historians formulate the research questions. We decide what data to use. We decide how to model them. to model person-to-person relationship or to model called-to-place relationship or something else, what algorithms to apply

[24:08] and how to set the parameters when we're applying these algorithms and how to interpret these results. And sometimes when you get into network analysis, you will see when you apply different algorithms, you see different patterns. Some patterns are more meaningful, which means some algorithms are more meaningful than others for your specific research question. But sometimes they are simply providing different angles at the same data. So let me take, for example, in this kind of hypothetical network that people always use as an example of so-called a kite network. We say, which node, or if you think of this as a network of persons, so then which person was more important in this network? Well, that depends on how do you define by a person? How do you define the importance of a person? The more friends that you have, the more important you are, or the more

[25:10] influential friends, well-connected friends you have, the more important you are. Or you may only have two friends in this network, but you are bridging like two separate clusters. So you're a doorkeeper or a broker. So we have different network metrics. that may be used to understand the centrality or the importance of a node in a network. Or we think of this as an example. Suppose this is a network, right? And we want to partition it into different clusters. How do you partition it? One way is to partition this network into four different clusters. So that we maximize the density of connections while the connections between clusters are minimized. But there is a second way of doing this, which is to say, well, in this network, the nodes in the center, the persons in the center, are

[26:13] much better strongly connected to each other, while the blue nodes are in the periphery. They are not connected to each other, and they are connected to the network only through the central nodes, put these blue nodes in a very disadvantaged position. But both could be meaningful. If you think of a real-world case, if you think about this as, again, as a network representing email exchanges among employees of a company with four regional offices, with a manager in Boston, and there's three staff members that may be subordinate to him in the Boston office. New York manager, you have a Los Angeles manager or Austin manager. And here we're looking at a kind of hierarchical relationship. You're partitioning this network into these four regional offices where the staff members had close relationships with their managers, but not close relationship, no

[27:14] relationship at all with the managers or staff members in the different regional office. Or when you're doing this so-called core periphery analysis, you're looking two tiers of corporate hierarchy. You have the managers that were often in meetings and were in conversations with each other, and the staff members were sort of connected to this network only through the managers. So this is, of course, a highly idealized example, but the point here is that very often when you apply different algorithms to a network, you'll find different results. It does not necessarily mean that one algorithm is the other was wrong. What was more interesting is that, how do we interpret these results? And this is the job of a historian. They may tell you different stories, but these different stories are equally valid. So, if this is about network analysis, so how do we study it?

[28:20] If I had been given this talk, like say five or six years ago, I think I would have to spend a lot of time talking about the tools. And very often, talks like this that introduces the concept and usefulness of network analysis has to be followed by a hands-on workshop about Gephi, which is probably the easiest and the most popular among historians, and also, later on, more sophisticated tools. Now, these tools are still useful. integrates a spatial dimension into network mapping, but it offers a very limited sort of functionalities in terms of calculating the metrics and understanding the structure. Gephi, and now you have a Gephi Lite, which is a web-based application now, allows us to sort of visualize the data. And you have more advanced tools like the R and Python packages, the UCINET,

[29:26] which is a very old, but still kind of authority software developed by sociologists. And then we have more specialist tools for very specific purposes. There are textbooks out there. There are tutorials out there from the programming historian, the historical network research community, or tutorials offered by any of these software providers. Now, I would say today things are different. We have to teach this differently. I would say AI can help. AI can help. By AI, I'm using a less accurate and a more narrow definition of AI, which largely means the generative of large language models like GPT and stuff like that. AI can help us at every single step of this network analysis to data formatting, to data visualization analysis, with some limits.

[30:27] So we can use AI to extract the data from historical sources. This, for example, is the same page from the local gazetteers that I showed earlier. So this is an experiment I just did like two days ago, right, when I was preparing for this talk. I did this research that I don't have to use AI, But I just say, hey, if I'm doing this research now, can I use the AI to help me? It turns out to be pretty good. I just give it. So I send a ChatGPT. I send GPT this page, which is a scanned page of the historical document. It's not a modern print. It's a modern reprint, modern photocopy of a historical document. And it's not OCR. and send it to it and the prompt I give it is, please identify all the shrines mentioned in this text for each shrine specified the county where it's located, the deity, and shrine, and on and on.

[31:29] And the output in a few minutes is this structured data table in Excel. So now what this basically tells us is that we're collapsing some of the steps here. I've been doing network analysis for about 10, 15 years by now. So in the past, we're thinking about a sequential pipeline, including different stages. You have a scanned image. Then you transcribe the text. You do OCR, automatic text recognition. And then on the transcribed text, you apply rule-based algorithms, such as regular expressions, to extract the named entities. This is a place. That is a shrine. And this is the name of a person or a deity. And then you extract the relationships. How do we understand the relationship between this place and this shrine and this shrine and this person or deity mentioned there?

[32:30] Now it's different. We will say, hey, just do it. Now, it's different from the user's point of view. It does not necessarily mean that the computer in the back end was not going through these different steps. I think it probably does, but from the user's point of view, it became less important for us to understand the technical details of the intermediate steps. And with one prompt in natural language, it's going to do the data extraction force from scan image to structured data in Excel or CSV format that you can use for analysis. then with all this or if you already have the data in some way you can ask it to link the data so here for example i collect the data so this is my old document in which sort of more or less manually collected this data many years ago by sort of constructing two tables the place and the shrine and the shrine and the second table is the shrine and the deity worshipped in each shrine.

[33:34] So I send these two CSV files to ChatGPT and I ask ChatGPT to combine it into a place-to-deity relationship and they did it. Of course it can make mistakes. So it's important that we always verify its output whether in terms of data extraction or in terms of, in this case, the integration Then the next step is network analysis, right? We need to visualize and analyze this network data, right? In the previous example, what we did was to partition this network into different clusters using community detection algorithms. Now what I did is that I uploaded this data file to ChatGPT. And I said, please import this data set into Gephi, run the community detection algorithm clusters and color the nodes according to the cluster membership and on

[34:37] and on and give me the image on the right hand side is the image that you see you don't see seven clusters as i identified you see like a 20 how many 18 clusters right the reason why is that there are many of these what you basically see is each place is a cluster because each place has some gods that were very to that place that were not worshiped by people in other places. So that was not the kind of work I did. What I did was that these highly specific local gods do not tell us about the connections between places. So I have to exclude them from my analysis. So in other words, if you are just asking ChatGPT to do the analysis, to do the thinking for you, It can't do it. These are choices you have to make based on your research question and based on

[35:40] your understanding of the data and the historical context. So I give it a different prompt. This prompt is revised and much more sort of, it's much clearer, right? It's like remove all the cards that are associated with only one place, convert all ties to unweighted, which basically saying, well, I only care about whether this god is worshipped here. I don't care about how many temples this god had in this place. And exclude three specific cults that I know will worship almost everywhere in this area. So that it's still, and for that reason, are not particularly meaningful for my research question. And this is the new result that we have. So it identified nine clusters. fine, but if we think about this message data, there's always some room between like a six or seven or eight or nine.

[36:41] So we're looking at the patterns of whether one place or one god belong to a cluster or belong to different clusters really depends on how strictly you are defined the closeness of a type within each cluster. You can put this in again. You can ask I can follow up with another prompt asking ChatGPT to present the results as a matrix. So I have a very similar matrix with sort of places as rows and codes as columns, as I have shown earlier. So this is one way of using AI. is almost a hands-off approach. But even in this hands-off approach, if you want to make the analysis meaningful and historically interpretable, you need to tell it how to clean up the data. And that is not a kind of a decision that AI can make for you.

[37:46] Or we can use it in a very different way. We can say, hey, let me do this analysis myself. I don't know how to use Gephi, but I want to use Gephi. People told me that Gephi is important, right? It's the first thing you should try. So I'm going to try to use Gephi, but I don't know how to use it. Please provide step-by-step instructions about how to use Gephi, how to import the data, how to structure the data, how to run community detection algorithms and so on. So in this case, we're still doing the work in a very old-fashioned way, or a software tutor. Or in the third possibility is to use AI as a coding agent, so called Vibe coding. Well, still do the analysis for me, right? But write it in a Python code or R code. The good thing about, and it can write the code for you. So the good thing about writing Python code or AI code, the good thing about using

[38:50] AI as a coding agent is that you have these programming script that you can inspect, try to understand, and rerun to reproduce the results. So it's much more flexible. You have control over the algorithms and the parameters in this process. It's transparent. You can share the script and others. And once you share the script and the data, other people can run the same script on the same data verify or disapprove or refute your results on their machines. And so it's transparent and it's reproducible. So in this process, there's one other thing, right? We need to, so in the earliest steps, we basically send the AI the data, we send the AI the prompt, and the AI gives us the output. What we didn't realize when AI was running these, creating these scripts,

[39:55] when it was creating these images, it has made decisions worse without our knowledge. So as a follow-up, I asked AI, in this script, what parameters are set as default and what parameters do I need to specify? And it tells me that most of this is less important with the exception of this parameter here. The modularity value, the resolution for the modularity analysis for the community detection algorithm is set as 1.0. So basically you can sort of increase or decrease the modularity resolution to reflect your understanding of how close you want the ties with each cluster to be. And that is going to affect the size of each cluster and the number of clusters you identify in the network. So that is a very important parameter. But the AI just did this. It made the decision for you without telling you that.

[40:57] So I think that's very important. However, we want to use AI as just like a one-prompt solution or as a coding agent or as a software tutor, we want AI to tell us what are the decisions it has made for us. So it's very important for you, for us as historians, to inspect and understand the code. We don't have to be a programmer. We don't have to write the code, but we have to understand what the code means. Now, I think there is a big difference between writing the code and understanding the code. Take European history, for example. I believe many historians can understand Latin, but that does not necessarily mean that you can write in medieval Latin, right? The latter is way more difficult. So coding is similar. Programming skills are necessary, but probably writing the code efficiently now

[42:02] became less important than understanding, reading and understanding what the script that the AI writes does. So in the above examples, I would simply say, if we think of digital sort of computationally, these historical research with computational methodologies, so-called digital humanities, as a research cycle that starts with scanned sources, to transcription, to named entity extraction, to disambiguation of entities and their relationships, to data formatting, to network visualization and analysis, to assessment and historical interpretation, almost at every single step, AI has a role there. So the basic workflow does not change, but AI can be integrated into every single step in this workflow to accelerate our work.

[43:04] So, or for some simple tasks, you can do it just with a one -prompter solution. and verify the final results. For example, in this case, I'm sending AI 10 tomb epitaphs, so-called long biographies from 8th century and 9th century China. I send it to him and say, hey, I know these are biographies or epitaphs from the Yan family in the Sichuan, which is a southwestern province of China. Take, so convert this into a family tree. showing their kinship relationships, their marriages and the offices they held. And this is the result I got. And some of the names were not consistent. Usually when historians are reading these historical sources, you have to scratch your head over and say, hmm, so some of the names, so these two names must refer to

[44:05] the same person because they have the same father, they have the same sons. And in this case, ChatGPT is doing a very good job and they say, hey, these two names must refer to the same person. And it resolved this ambiguity very nicely. Now, I actually did this experiment about five months ago, and it was able to resolve one of these sort of discrepancies. Now, it perfectly resolved the two discrepancies. So here in this case, we're jumping directly from transcription to network visualization. We're skipping all of these steps, again, from the user's point of view. The AI was doing this in the back end. So I would like to sort of conclude my talk here with a contrast between what we call digital humanities and AI very briefly about what I think is the most important thing that AI

[45:10] cannot do for us. If we think of close reading, it's like this. This is actually a picture I took of a salt pond in North China. When you get to the salt pond, it looks like this. This is close reading, right? You see sort of the grasses in the water and so on and so forth. Or if you can take an airplane, you can take a helicopter and go all the way up above the ground and look down in that same area, this is what you see. You see colorful salt ponds in this Shanxi province, in this place called Yuncheng. So I think this is what network analysis, or more broadly, computational methods or digital humanities does for us. It's distant reading. It integrates data from diverse sources, and offers us a macroscopic perspective of the historical landscape.

[46:13] AI, I don't think AI is offering something similar. I don't think AI is providing a different approach to analyzing historical data or historical sources. I don't think AI is providing a different perspective at what we are doing. So I would say that digital humanities or computational methods is like a helicopter, right? It asks us to step back from close reading and to see the larger arc of historical change. It changes the way how we see it. It changes the way how we think of history. AI is more like a high-speed train. It's still on the ground, but it's efficient, it's faster. It doesn't change the way how we see history. It changes the way how we think of history, but it changes the way how we do it. It makes it a lot faster, a lot easier. It lowers the technical barriers so that we can focus more, not on the intermediate

[47:15] steps or the technical details, but focus on interpretation and assessment. With this, I'm gonna stop here. Thank you very much for your attention. Now the floor is open to questions. Please. getting answers from different practitioners you know the importance of verifying the attractive or what it would work um and it sounds like the ai enabling more of that style work and so I'm interested when you talk about the important to verifying how do you go about that fine because I've been seeing that you're doing that by hand, you're reintroducing the thing that's slowing down those type of index. That's right. Yep. So I think that really depends on

[48:15] what stage we're talking about. I think I'm supposed to repeat the question, right? So the question is, how do we verify the data? If we're spending time verifying the data, are we actually slowing down this efficiency using large language models. I think this really depends on what we are looking at. If we're looking at a data extraction, extracting sort of persons and place names from historical sources and putting them into a structured data set, in that case, we have ground truth. You can read the sources and check if it is done for you. Now, it's going to slow you down, but what you're getting is you have to do some close reading anyway, process you may discover other problems that you would not realize so this is sometimes when we're sort of hiring like a graduate students to work as a research assistant for us I don't think this is going to replace our own reading of the text now of course it's by

[49:17] saying this I I recognize I'm a historian of middle-period China our sources are more limited I'm not a like a history of a 20th century China in which it's almost impossible for it to read everything case, I think you have a ground truth. You can check the ground truth against historical sources. When sometimes, even if you have ground truth, but it's hard to check every single data entry, then I think we have to sort of settle down with some other techniques. So I've been talking to other conference attendees. One strategy, they were saying it's like a sampling, right? So you You pick a set of data and they check to what extent the recall and the precision, right? How many false positives you got, how many false negatives you got, and so on and so forth. And yet another strategy is to ask different AI agents to do the same tasks and ask a third AI agent to compare these different sort of

[50:21] data sets constructed by Gemini and ChatGPT, and then you focus on the discrepancies. So this is a second strategy. There could be other strategies. For example, when I'm constructing these network data for the kinship. So when I was reconstructing the kinship networks of the 11th century and 13th century elites. I needed several sets of information. I needed to know that they were actually serving in the 11th century. And then I need the relationship between these prefects. And you can use these two different dimensions of data to verify each other. If you say A is the father of B, but they were serving in the government 100 years apart, you know that's wrong. So this kind of sort of you're looking for internal consistency between different

[51:24] dimensions of the data. The data certainly is not applicable, for example, to the called place data set that was talking about. So this is more kind of specific to the different data sets you're dealing with. So I would say multiple agents, sampling, close reading, and some sort of custom device, sort of a custom built sort of verification strategies for the specific data. And these are the possibilities I can think of. Thank you very much. Please. Thank you very much for the fascinating talk. So in comparison, this last slide, thinking about the no shifts in perspective or methodology, in your use of AI, it seems very strict in the instruction. Did you try with a more open-ended, what are some other types of analyses, all-time network analysis? How could I shift my methodology? How could I shift my perspective in this analysis? and if you did try that, what it returned, or if you didn't,

[52:27] I'm interested to type it in now and speak. Yes, yes. And to be frank, nowadays, so these are the work that I've already done. So I'm just trying to test its limits. For the ongoing research, I'm engaging in constant conversations with AI about the kind of methodologies that they think are appropriate for my work. Not all their devices are useful, but some of their devices. For example, one of the projects I'm doing is called sort of language modeling. So basically, you have, I'm looking at the writing styles of different periods and see to what extent writing styles change over time. So one thing you can do is you can build a language model based on an early set of texts. So ask this language model to learn the writing style, and then you use it to predict the writings from later times and see to what extent it can predict and to what extent it cannot. If it cannot, that means there is a significant change, right?

[53:30] So in things like this, so you have multiple different choices, right? You can build a large language model. You can build a more traditional statistical engram model. So I'm in conversation with AI up to the pros and cons. So I think that would be using AI as a colleague that you can have conversations with, especially colleagues with a different domain of expertise is actually very useful. And it's kind of, I think it's more or less in recent years, kind of integrating it into the process of my research design. Thank you. Yes, please. So, taking an example of the temples and the cultures, I'm understanding mathematically how clusters come about, but I'm also wondering, though, what these clusters have in

[54:33] common other than just some sort of mathematical relationship. I mean, is there some sort of explainability beyond this? Do these clusters have common properties beyond just that they're co -located? And, you know, what might we do to explore that? I think, you know, there are other styles of graph analysis that are much more sort of knowledge-based, which look at different properties and things like that that have this kind of explainability around it, but aren't mathematical analysis. Yeah, thank you for your question. So this question is about how do we interpret these different clusters, right? Besides simply saying, well, they have something in common, right? I think I've, now I will say two things here. Now, first of all, the clusters that we identify here is based strictly on data, right? On the data of cult-place relationships, what

[55:35] cults were worshipped in what place without regard of their geographical proximity. So two places are placed together in the cluster simply because they have similar religious profiles regardless of their geographical proximity. And then you put them on a map. And then you see they actually geographically, they cluster geographically. I think that is the most important argument, that is the data-based argument I want to make here, is that we can simply, it is possible for us to use a cop -placed data to understand the geographical clusters of similar religious cultures. And then the second question is that, I think the second part, probably that's also part of the question you are dealing with, is, okay, so through network analysis, we see these patterns, but how do you explain these patterns? What are the historical mechanisms driving the formation of these geographical or religious patterns that we see here?

[56:39] And very often, I think it's not in all cases, but very often, I think it's something network analysis really does not allow us to do. It gets us to see the broader patterns, but it's not really helpful in most cases in explaining, in helping us explain the patterns. In this case, I think there are many different things going on there. Now, one interesting, I don't have the maps here, in the article that I published, I have another map showing the modern sort of dialect zones, modern 20th century dialect zones sort of coincide almost perfectly with this 13th century religious cultural zones. So I would say, well, there's something going on about these religious cultures. Or if you think about the religious process, I think what this graph tells us is that there are two different mechanisms for a cult to spread.

[57:42] One is you worship these gods, and the people living nearby are affected, they're under the influence, they became devotees of the same God. They're converted, right? So we begin to see this God, maybe sort of spread from one place and it spread to nearby places. Now, what is also interesting, which I didn't talk about just now, is we know that some places, like this ones and these ones, right? So these dots are bigger. Bigger means there are more cults there. There are more cults in these places. And they are more colorful. You have different colors. Each color represents a cult that was assigned to a different cluster. So that means these places not only had more cults, but they had more diverse cults from different places. And these places were the prefectural seats. All of the others are county seats.

[58:44] actually controls several counties. So that means the administrative hierarchy of imperial China in the 13th century mattered to some extent. The prefectural seats, some sort of served as a meeting place for the different religious cultures playing an integrating role for the local religious culture. So, but this is based on historical research and based on our interpretation of what we see given our understanding of the historical context, our understanding of religion in China. It's not something that network analysis immediately tells us. So I think what computational methodologies do is to allow us to see the patterns that we couldn't see with just a close reading. But interpretation and explanation remains the job of historians. AI probably has a kind of an advantage in that.

[59:48] In some cases, I can ask AI to provide an interpretation. Yeah, I wouldn't say it's always right. But in some cases, these are interesting interpretations that we as historians should consider. Yeah, please. I would say that in addition to, we still need to do the interpretation, the network analysis and the discontinuities we see in spatial representations, for instance, they point to the existence of variables that we haven't accounted for yet that we need to follow up on. Right. So in this case, right, I see this continuity of the first cluster in the dark blue on the top left and you're right on the left hand map. But and there are a couple of other pockets. Right. Right. I think there is I don't know if this is of the norm of missionaries or particular relationships that are accounted for in the documents we have. But we are seeing evidence there could be relationships that we found more data. Is it helpful to use network analysis in that way to sort of find out what you don't know because there is something? Yeah. Okay, so the question is about whether

[1:00:54] it's possible to use network analysis to find out something we don't know. And so first, this pattern that I'm showing here about the geographical clusters is something I think we didn't know before. We may have some speculation, but we didn't know before. And second is that I think you are sort of pointing out a very important point. Now, I kept talking about sort of patterns and patterns and patterns. But wherever there's a rule, there's an exception. Wherever there's a pattern, there are outliers. So network analysis in this way also highlights these outliers that especially deserve our attention. Now, I haven't followed up on this particular study yet. And as you said, yeah, these blue pockets here and there, they're interesting. there, right? So this is an invitation for further study that can guide

[1:01:55] close reading, and I certainly think that is very important. Okay, I think our time is up, and if you want to stay and you have more questions, you're welcome to stay, and I'll be happy to stay on for a while, or we can see each other in the coffee break. Thank you very much. Thank you.