Abstract

Katie McDonough introduces MapReader, an open source library that treats historical maps as data. A map sheet is divided into patches, an image classifier labels each patch, and a text-spotting step reads the words on the map. She draws on the Living with Machines project, where railway patches from Ordnance Survey sheets were linked to the 1901 census, and on the Machines Reading Maps project, where text from the David Rumsey collection was clustered and explored through random walks over a network of words. She opens with principles on open source tools and community maintenance of research software and closes with open problems in multimodal queries with vision language models (VLMs) and in storing map text in library systems.

Chapters

Transcript

AI-generated transcript. It was produced from the recording by speech recognition and cleaned up with AI. It has not been edited by the speaker and may contain errors. The captions on YouTube are AI-generated in the same way. Each timestamp links to that moment in the video.

[0:20] All right. My mic is on. Great. So I'm thrilled to be here. I'm going to give people a minute to get settled and sorted. After that wonderful lunch. I've lured you here with the promise of talking about maps, which we will, but I'm also going to say a few things. This has been a really, I think, motivating event for thinking about sort of the position between history and DH and AI and this thing that we also now call computational humanities and computational history. So I have a few, I don't know, it's a mix between principles and programmatic statements that I'm, provocations if you will, and I'd really love to kind of hear what you guys think about these. All right, why don't we go ahead and get started. So yeah, so today we're gonna do,

[1:24] I'm gonna give a little bit of a walkthrough. As I've been in sessions, I kind of noticed we were doing slightly less actual hands-on stuff. We'll see. Who in here has used Jupyter Notebooks before? Had a few takers. All right. I'm going to talk for a while. Please interrupt me if you have any questions. If there's something you don't understand, just raise your hand. I'm happy to be interrupted at any time. I will pose some questions to you, too. more of a kind of two-way conversation and I can towards the end kind of point you to the demos and hopefully at least do a little walkthrough of one of the Jupyter notebooks with you guys. So as I said this is both going to be a kind of introduction to MapReader which is a open source software library for analyzing historical digitized map

[2:26] collections at scale or as I like to say data. And maybe I'll just say a little bit about me. I am a historian by training. I've always continued to call myself a historian, but I've had a pretty non-traditional career. I just started at Carnegie Mellon this fall. I was in the UK for about eight years. I've had DH postdocs in a number of institutions ranging from kind of digital book history to a project called Living with Machines, which I'll say a bit more about in London. And I've moved around temporally and geographically, starting in 18th century France, moving to 19th century Britain, and also working with some global map collections. So I think of myself as someone who has specialized a bit more methodologically and kind of thinking about what it means to work with spatial information from the past in a digital environment.

[3:27] And MapReader kind of comes out of that sort of broader research interest. So as I mentioned, I have some kind of programmatic statements to open with. I mentioned that MapReader is an open source software package. Have you guys heard, kind of like, dealt with the difference between commercial and open source? Is that something that is like I was hoping that we would talk a bit more about that at this conference. And so I welcome conversation and questions on that issue in particular. I think that open source computational tools, especially those developed by historians, for historians, or broadly for a kind of social science and humanities research community, are essential for the kind of work that we want to do where we want to have methodological transparency. This is essential. Historians, therefore, we need to be building and maintaining research software as a community. It can't just be a thing that one person does.

[4:28] And I really want to encourage historians to kind of get involved in the broader community around reproducible research software and software maintenance, which is something that scientists spend a lot of time thinking about and committing resources to, which we do need to do more of. Third, when we're using tools like MapReader or any kinds of computational tools and creating data that is derived from library and archive collections or wherever your collections might be, including in oral histories and things like this, this has to be done in conversation with the curators or communities who host those materials. This is really essential. I'm really, really not a fan of the historian walking the archive, taking 20,000 photos, walking away, and never talking to that archivist again. I think this kind of lone scholar approach to computational history, digital history, is something that we really need to kind of work against and work

[5:33] with our colleagues in archives and libraries. On the tail end, when we create this data, we've got models and data that we need to start treating as meaningful publications, right, that count for things in our professional lives, because they are essential to doing computational research. In the UK and Europe, where I've been working, this kind of publication counts, right, in the way that we use that word. And in the US, that has been slower to uptake on that. So there are ways to write data papers as journal articles. You can publish models with model cards, and you could also publish a model with a kind of paper attached to it. I think the Journal of Open Humanities Data, the Computational Humanities Research Journal, would welcome papers that are sort of presenting models and training data,

[6:34] ideally, to go along with it. Cloud might be your collaborator or copilot. But another human, humans, experts in different fields are really going to transform your mindset and have the potential to become, I was being very optimistic here, a lifelong research partner, right? But I think that there's something really fundamental about the experience of being in the digital humanities community for the last 10 to 15 years that's taught me that I learn and I do more interesting research when I'm in conversation with people from other fields. So yes, you might not need them. Claude might be able to do that data visualization or that analysis. But what are you missing out on by not having those conversations with people who can think in a completely different way or very, very, very quickly? So the work that I'm going to be showing you today is very much kind of

[7:36] interdisciplinary collaboration. As I'm going to tell you when I'm walking you through MapReader, you don't have to do what the commercial products tell you to do. And this kind of comes back to that open source question. Commercial products, including those designed as research software, so most of the specific things like ArcGIS, right, all the Esri products, they want you to think about your data in a particular way. you that you need to do analytical tasks that have been defined by a certain community. For example, ArcGIS is really restrictive on what kinds of data formats are input and output, right? And those are not necessarily the tools that historians want to engage with when they're thinking with their sources. But it doesn't mean that they're pointless. I use ArcGIS or QGIS all the time, but they shouldn't be the only thing that we use. Finally, and this kind of came up in a few of the sessions yesterday, you shouldn't

[8:37] be asking AI models like your high-level question. You should be using AI. You should be automating the things that are kind of the non-reasoning tasks that you want it to process large amounts of information. You don't want the computer doing the thing that you love to do. So I hope I'm going to give you an example of what that means. Almost finally, this might be the last number, computational methods, both if they're AI or non-AI, are best suited when you have formalized your unit of analysis. And if you're not using commercial tools or you're not operating with a chatbot, very clear sense of what you're looking for and what that translates to in a kind of computational environment, right? What is a machine-readable piece of data?

[9:38] In the panel yesterday with working with computer scientists, you know, this is a great example of, well, the term example. What is an example? Well, it's an example of the thing that you're looking for. Oftentimes, historians say, well, I'm looking for evidence of this kind of, like, human behavior, maybe. define what the proxy for that is in your primary sources. And coming to figure that out is a learning process that you wanna be in the driver's seat for. So MapReader is a tool that really kind of helps the historian think about what the unit of analysis is. And finally, I think this is finally, I also just wanna remind people that AI isn't just restricted to generative AI. And the focus on sort of agentic and generative AI here makes sense because these are kind of the bright, shiny new things that we're all trying to figure out. But there's really a wide world of tools out there for automating what

[10:42] you do that don't have to mean engaging with generative models. Okay. So as I mentioned, this work is collaborative, both with other historians, but also with data scientists, geographic information scientists, software engineers. I especially want to highlight the work of Rosie Wood, who is the main software engineer at the Turing Institute in London, working with me on MapReader. There's individuals here, teams. There's a lot of people behind MapReader. It is not just me. I want that to be extremely clear. All right, so MapReader started as a kind of research journey on the Living with Machines project, which ran from 2019 to 2023 at the Alan Turing Institute in London. Let me move this down. Is that better? Yeah, okay. And Living with Machines project was funded by the National Arts and Humanities

[11:46] Funding Agency in the UK to be a big experiment, right? what can we do, what kind of new histories of the experience of living through industrialization in the long 19th century in Britain can be told if we use the massive kind of sets of primary sources that have been digitized in the UK already. So census records, newspapers, books, and when they hired me I said maps, let's work with the maps. So the National Library of Scotland not only scanned, but also geo-referenced a huge portion of their collection of ordnance survey maps. So if there are Americanists in the room who are familiar with the USGS topo maps, right? This is kind of the national mapping project in the UK that started in the mid 19th century. There are several scales and several editions of each scale. We ended up working with the six inch to one mile. So this is fairly large scale, but not the largest scale.

[12:49] one edition of the six inch maps is about 15,000 sheets. That's what you see here in the visualization. But one edition of the 25 inch maps is about 50,000 sheets. And so we were thinking about how can we work with this data? It was really important to us to think as historians about what it means to make claims based on thousands of maps rather than just one map. You know, historians, as we all know, we kind of tend to work on like case studies where we have lots of high-resolution, detailed data from a few local archival experiences or printed materials in a cultural history context. Or on the economic history side or quantitative social history side, we have regional and national explorations that are looking at aggregated data, in particular thinking about how people engage with census data. What's really cool about these continuous national scale, scale, as in very detailed maps, is that we can move around

[13:52] in scale. You're not limited to doing only kind of like highly aggregated, high level research or just picking a few case studies. You can move around in scale in a way that is spatially grounded, and I find very exciting. And I think this is the basis for what it means for maps to be used as kind sources is the ability to look at patterns on a national level, but also zoom in and look at what that means at a street or village or county level. The thing that I tend to say that's a kind of challenge for working with maps, and one of the reasons we wanted to move away from a kind of traditional historical GIS approach, was that we didn't want to treat the maps as just the truth about the historical landscape, right? As anyone who has studied the history of cartography knows, right, we don't treat maps like any other historical document as an objective picture of past landscapes.

[14:55] And so, you know, we want to use computer vision and machine learning tools here, not to kind of define the truth of what was on the ground, but to allow us to interpret maps as primary sources in a new way. and so in this sense for me it meant to kind of step away from the paradigm of geographic information systems where you know there's this kind of natural pathway that I have followed and I think many people have followed here where you say okay well I've got a scan I've got a historical map I want to create some data from it you scan it you geo-reference it you create some data about the roads and building footprints and other kinds of features that are present on the map, right? You're doing a kind of data mining, data extraction approach where every single feature gets a very, very precisely defined point, line, or polygon. I wanted to say, like, what if we just didn't do that?

[15:56] What if we stopped kind of imposing this kind of precision on the features of historical maps. And instead, we developed a method that was much more flexible, creative, iterative, and critical in the sense that it kind of encourages the scholar to remember that the map is not an objective document. Okay, so another way of thinking about this is that working with maps as data is not mapmaking in reverse. GIS tools were originally designed as kind of a cartographers or urban planners toolkit. We actually want to have tools that are designed for scholarly research, for historical research on maps, not to create maps or to perform spatial analysis for kind of contemporary problems, right? And so I draw on, you know, ideas from, you know, people like J.B. Harley and John Wood and John Fels, where we talk about acknowledging

[17:00] that the map is not a fact, but that it's an argument, and we need methods that help us unpack those arguments. All right, so MapReader is a kind of alternative to those ways of thinking. At its heart, MapReader does two things. It allows you to use image classification models, and these are traditional, say traditional now. In 2020, we were like, ooh, convolutional neural networks. It's so cutting edge. Now it's really boring, and computer scientists are like, really? You're still using those? But they do the job well. So image classification with CNN models, which means that we're sort of able to assign a label, right, or a concept that's of interest to you, the historian, to a region of a map. I'm going to walk you through that in a second. The second task that MapReader performs is text spotting for maps, right? And so this is really important to me because for a long time, text on maps has been kind of ignored as a really integral part of the information

[18:03] on a map. It's also not been something that we've been able to kind of sort of identify en masse, all the text on a map, and maybe compare that to similar maps. So this is a toolkit for working with maps as multimodal documents, both the visual and the textual information. Any questions so far? Oh, good. I'll keep talking. OK. So the first task, the visual classification of map content, I'm going to be using examples from the Ordnance Survey Maps of Britain throughout here, basically works like this. We take one sheet, which we can call a parent image. We impose a grid on that sheet. The user can define the size of the cell or region or what we call a patch. and the size of patch can be defined in pixels. If your map is geo-referenced, you can also define it in meters. And each of these patches becomes the unit of analysis.

[19:05] So the patch is the thing that you're studying that you're gonna label in a kind of machine learning context to identify something, a concept that you, interested in, which may or may not be something that cartographers have told us we should be interested in on maps. The model itself, as I mentioned, is an image classification model. This means that we're not identifying where in the patch the thing that you're interested in is, only that it's present within the patch, right? So this is a low precision task, which is intentional. We're not trying to find the pixels associated with every tree symbol. We just want to know that a tree symbol is present within the patch, anywhere in the patch. All right, so this is the really crucial part here. This is where MapReader is a kind of tool for thinking with your maps, and it

[20:07] encourages it to be iterative. So in GIS, when you start tracing something in GIS, maps and I'm going to trace all of the roads. If halfway through you decide that actually you want to capture that information differently or you're actually interested in something else altogether, you've got to start all the way back at the beginning from scratch. As a graduate student, I found this extremely frustrating and as a professor, I know I don't have endless money to pay research assistants to do this kind of work. What MapReader allows you to do is to not only kind of change your mind about what your label is, what your concepts of interest are, it also allows you to annotate quite quickly, train models quite quickly, so that starting over from scratch isn't the kind of burden that it is with traditional kind of manual annotation and GIS. But what's interesting here of course is that it takes kind of time to get used to thinking about the patch as the unit of analysis.

[21:09] You want to say okay for us I'm the example of railway infrastructure. We were interested in sort of capturing information about where railway infrastructure was present in 19th century Britain for a broader study about the experience of living near railway infrastructure. But that's a kind of analytical concept that we're interested in as historians. What does that mean on the map, right? What's the kind of machine-readable thing that that analytical historical concept corresponds to. That is what the label becomes. And we have all these kind of arrows here to indicate that this is an iterative process, because it took us several months to kind of land on a set of guidelines for annotating these patches according to rules that both meant that we were able to train a high-quality model and that it corresponded to the thing that we were interested in. So I mentioned we were

[22:13] interested in sort of who benefits from the presence of rail, in particular in relation to who is living near railway infrastructure. And in working with MapReader and Patches, we were kind of rejecting this idea of rail as purely a kind of transport system, right, that allows people and goods to move from one city to the next in a kind of economic history terms. We wanted to think about the impact of rail in a way that was completely different from the kind of way that British historiography has treated it to date. And this was instead to kind of return to the site of rail infrastructure and think about the footprint of rail infrastructure. And the patch really gets at this, right, can think not just about the track as one line, but actually the space that the track kind of is adjacent to, the community that railway infrastructure is a part of.

[23:16] And so the patch level data actually matches up with our historiographical concern for thinking about the footprint of rail and its relationship to the community rather than then kind of focusing on the line as something that moves things around, but which is isolated from its immediate neighbors. So with MapReader, what we were able to do was take those 15,000 sheets, patchify them, and annotate enough training data. Way back when we did this, we didn't know that you don't need 62,000 patches, but I annotated 62,000 patches. really need about 5,000 to 10,000 for a data set of this size. And the patches were defined as 100 meter by 100 meter. And then we identified, as you see in B, the national kind of footprint of railway infrastructure across England, Wales, and Scotland around 1900.

[24:17] And then C, you can kind of see that at the patch level for one sheet. So we use this work, as I mentioned, to connect up with census data. This is work that is now available open access online in the forthcoming Living with Machines book. If you want to hear more about it, I'm not going to go into the whole argument, but just to show here that the real power of this was that we created a national level data set. We were able to link it up because it's spatial data with another geospatial historical data set, which was geolocated residential streets from the British census records of 1901. 1891 and 1891 too. That was work led by my colleague Josh Rhodes, published in Historical Methods, I think. So that's good. And we were able to identify what residential streets are close to high density, medium density, or low density rail space, right? So how many rail space patches are kind of within a certain

[25:19] distance from a street. And then, right, to come back to this around in scale, what you're looking at here is London, but we have this as a national data set, the census streets as well. And we were able to kind of target case studies at the especially high density we were really interested in to figure out, okay, well, in the census records, what's the nature of the households? What kind of occupations do they have? How many live-in servants do they have? Remember, this is 19th century Britain. How many children do they have? What other kinds of information can we garner from census records to help us understand what it was like to live in a neighborhood dominated by rail space? And one of the big findings from this work was that while historians have really argued that the amenity effect of rail was primarily restricted to white-collar professionals

[26:20] who live near a station and can use rail to commute from sort of leafy suburbs of larger homes into the city, across British cities, not just in London. But actually, we were able to point to an amenity for working class people, which is that they can live near potential sites of employment, right? So we found very high levels of employment in trades and occupations related to transport, railway infrastructure, we did a kind of special coding of occupations to identify those that are related to railway work. So again, I'll point you to the chapter for more information, but just to show you this data in action. So the second thing you can do with MapReader is identify text. And this sort of part of the pipeline grew up grew out of a separate project that I co-led with colleagues who are at the University of Minnesota in the

[27:25] knowledge computing lab that was called Machines Reading Maps and so the focus here was on identifying text on maps. Our partners on that project were the British Library and the National Library of Scotland again and the Library of Congress and then we also worked with David Rumsey who has the largest private map collection in the world it's now at Stanford Libraries And he said, I want you to identify all the text on these maps, and we're going to make it searchable on our website. So if any of you have used the davidrumsey.com website where you can search by text, this is where that data came from. This is what it looks like when you put it all on a map. And it's a lot of data. It's a lot of data that's completely new to learn to work with. So not only did we kind of create this pipeline for working with the data, what we're really trying now to do is figure out how do we help people work with this data. So I'm working with Bert Spahn and the All Maps team to think about how we go from like the giant dump of GeoJSONs that we put

[28:27] on, which is open data via the Stanford Library catalog. How do we actually make that easy to explore? So we're creating data visualization interfaces like this, which allow you to kind of bring selections of data together. and do searches and hopefully in the future be able to kind of export kind of curated sets of maps from this large 57,000 collection of maps. All right. Part of this is sort of continuing this mission of thinking about how do we not just create data using open source data, but also continue to kind of contribute to an emerging like open maps ecosystem or set of infrastructures where we're all using the same standards. We're sort of sharing models. We're thinking about annotation together, how we evaluate our data. These are all things that in the kind of emerging world of computational map studies, we're working on together.

[29:27] If you're interested in this kind of thing, I highly recommend checking out the Open Maps meeting, which was convened by Jules Schumann and Vincent Baptiste. And there's gonna be a second meeting this year. So stay tuned for that So I mentioned that text on maps is totally new data. What do we do with it? Well, the first thing is like, how do we structure it? If you have text from one map, you can have it geolocated. You can have its pixel locations. But on a page, we have sentences, and we have punctuation, and we can turn the page. We need to think about what it means to read text on maps. And not just to pay attention to maps as sort of reference sources or text on maps as reference sources for like, oh yeah, that place name that I'm interested in, that's on the map. We want to look at all the text on maps. And there's a lot of unnamed places on maps.

[30:30] So on the Ordnance Survey maps, for example, we have footpaths identified. We have churches. We have schools, pubs. which have their kind of formal business names, right, but just PH for public house. So these are the kinds of pieces of information about a very, very detailed understanding of what is present in a neighborhood that we can work with here, but that are not traditionally, in terms of kind of GI science, treated as kind of formal spatial information. So here is also an opportunity to think about how do we not just treat the text as kind of sort of objective information on the map, but instead to think about, as Harley and others invited us to do many, many years ago, map as discourse. But to really take that quite literally, right? If the map is a text, how do we read it, right? So instead of just thinking about the map as text in a kind of metaphorical sense,

[31:32] we're taking that very seriously here. But it raises this question of, right, of like, what are meaningful sets of text on maps? So experiments that we've done so far involve thinking about how we can cluster around sort of patches, right? So this is a kind of multimodal experiment where we take the text associated with all of the sort of text intersecting with all of the rail space patches, and we cluster it. we take a group of sort of three by three patches, and all the text within that three by three group, which is centered around a rail space patch, becomes a patch group, right? And becomes kind of the equivalent of a sentence. I don't want to call it a sentence, but it's about 21 average words when we look at this across a little case study we did of London. And so you have this kind of set of 21 words that you're looking at as opposed to, you know, you can see there just within one patch maybe two or three words.

[32:35] What this means is if we define, we had a defined number of clusters using a k -means clustering algorithm where we can see that there are certain kinds of patch groups. There is a really important presence of patches that have no text, patch groups that have much in the outskirts of London. So you can see on the left, you have the most present words, so none being no words, then line, great, way. These are associated with railway names. And on the right, you can see that distribution geospatially. Here, you can see a different cluster where the most common words within those patch groups are dock, works, wharf, peer. So I think of this as the docks cluster. the map that this is really associated with the kind of with East London, with the Thames, with the canals and the Lee River sort of here going

[33:36] north. And so this final one here, the extractive industry is also very oriented along the Thames, but both east and west where we can see that there are, you know, kind of gravel, brick, chalk companies that are dependent on river transport. So this was an initial experiment for thinking about how the text allows us to understand the nature of the city, how we can understand this both from a linguistic and from a geospatial perspective. Our second experiment, which is something that is our current work in progress, is to actually say, okay, how could we think about just the text itself? We set aside the patches for the moment. And just think about what it means to structure text in a network. So we gave a lightning talk on this at Computational Humanities Research last year. the thinking behind this is that we wanted to imagine what text is most proximate to other text

[34:39] and in a sense what does it mean to walk around the neighborhood of text on the map. Now we're not following streets and sidewalks right this is kind of as the crow flies but it's still a really kind of way to maintain the spatiality of the text while putting it into conversation with each other. work for every county. And then we used a random walks method to identify a fixed number of nodes that are connected within a walk. And this is an example of what that looks like. This is in Walthamstow in North London. So you can see here that we have a walk that's connecting like Almshouse, Church, Smith, Grange, and Garage. And the idea then is to compute the similarity like we did with the patch groups and clustering, but to compute the similarities between different walks and identify patterns in what words are close to each other more commonly or less

[35:42] commonly. And we'll do this nationwide as well. So where does this leave us? Kind of moving away from the impulse to just try to digitize everything on the map. We want to, I think, as I hope I pointed out, like patch classification, text spotting on maps, these were things that we didn't really do manually in GIS. And I'm really excited about the opportunities for historians to think of entirely new ways of engaging with historical map collections computationally in doing things that aren't just automating what we did manually, but actually doing something entirely new. The third thing is that we're creating data that we don't treat as kind of precise, objective information. I know I sound like a broken record now, but that instead

[36:42] continues to treat maps as propositions, as arguments. and that we want to think very carefully about structuring map data in historically informed ways rather than simply creating files based on one document. So again, doing things that make sense to the historical mindset, not necessarily to the computer science or NLP mindset. These two things feed into each other and we learn from each other. But instead of sort of handing a problem to a GI scientist and walking away and then getting something back, it's in this kind of interchange, I think, that the most exciting collaborations happen. And that's the kind of environment that MapReader came out of. Okay. Yeah, I wanted to end with just some statement about some things that we're still finding hard. And I'd love to have a bit of a discussion about this.

[37:43] So one of the things is that when people think about spatial data, there's still a real focus on place names, on sort of formal toponyms, right? The kinds of things that have records in gazetteers, although not World Historical Gazetteer, which is more capacious. And we really want to think not only beyond toponyms, but beyond tax corpora as sources of spatial information. We want to think about collaborating with the complex document processing community, right? These are the people who are thinking about Frontier OCR. But we need culturally specific models. We need to create training data. We need to think, again, about how we evaluate the data that we're creating. When we think about kind of truly multimodal queries, you know, MapReader right now has two separate pipelines for text and image. What does it mean to start using vision language models for a truly kind of multimodal query where, for example, we can automatically associate

[38:47] text with the symbol or the area that it's describing, right? That's kind of the holy grail here that there are computer scientists working on. But right now, when we do that kind of work, VLMs are just so, have been so overly trained to summarize something that they really struggle to do the kind of precise queries that we're interested in for historical maps. So, work in progress. Again, spatial information from the past is imprecise. The ordinance survey maps I've been showing you, yeah, they were scientifically surveyed. They are pretty accurate. When I talk about precision and not pretending your maps are objective, with OS maps it might seem like overkill. But as soon as you start going earlier in time, as soon as you start thinking about other mapping cultures, you really have to pay very careful attention to this. There's a reason why MapReader is designed to work both with geo-referenced and non-geo-referenced

[39:51] it makes sense to georeference things before the early 19th century. That's what I tried to do when I was a grad student and it was a disaster. I'm sure many of you have tried to do that too. So we wanna develop methods that allow people who are working in these temporally and geographically diverse fields to still engage in computational map studies. This is even more important because not all maps are digitized. More and more maps are getting scanned all the time, but the collection is inherently, for now, skewed to European and North American collections. The more that users develop relationships with libraries and archives and communities elsewhere in the world, perhaps we can develop larger digital collections that become accessible, but sometimes that's not appropriate. Sometimes maps shouldn't be publicly available, right? So I want to acknowledge that there's a lot of complexity here. Finally, while commercial software and models are more accessible, and sometimes, especially at the beginning of a kind of outset of a new task or a new kind of

[40:52] model, they might perform better, but they restrict your conceptual creativity and they restrict your ability to kind of develop a transparent, humanistic, sorry, I'm in the humanities camp, I should have said historical interpretation, which is really what we're trying to enable here with MapReader. I think that's it for now. So I can, oh yeah, question, yeah. Actually I just have to add two. Yeah. Mostly related to the, well first of all, I'm very intrigued by your approach. In my work I use a lot of historical geo-reference map data. With all the imprecisions that accompany with any kind of historical mapping. I'm intrigued by the way you're suggesting about reading these maps in different ways, if I'm representing that correctly. Yeah.

[41:56] Yeah. Yeah. Yeah. and finding about working class amenities. That's the question. And second, and this is kind of, this is, I think, a central challenge in historical analysis, no matter what technologies are deployed, is what, I want to be in common a little bit on the, do you think there's any kind of risk in approaching the reading of maps in these new ways of potentially undercutting what the creators were. How would I put this? Do we have the risk of creating interpretations that run out of the foul of the people who created them? You know, like, that we kind of tear them out? Would it make sense? Let me put it this way. Would it make sense to a 19th century reader what we are saying about the maps? Yeah, yeah. So there's a correspondence. Yeah. I say that we have to be beholden to what they thought to derive from

[43:00] comments. But I'm just wondering about the potential of that. I mean, I think that goes both ways, right, as you've kind of just suggested. On the one hand, we don't want to make of the map something that is inherently wrongheaded because we simply are kind of looking at it anachronistically. Totally. That's why I think let me kind of switch over here. to this, so you can kind of at least show, see this kind of in action while I'm talking. Yeah, it's going. But the, I think the flip side of that is that there are more people, and I'm sorry, I'm gonna like slightly throw some of my GI science colleagues under the bus here that are not trained to understand these complex historical documents, which is not their fault. They're not historians. I get it. And they treat these images as if they're like

[44:06] the equivalent of OpenStreetMaps data. And so a footprint is a footprint and a road is a road. I mean, that's very, very kind of, you know, objective. And so a lot of my work has been actually thinking about how do we work with computer scientists who are, you know, driving a lot of the development of these models, often more so for remote sensing imagery, but maps are like the side use case there. And so it's like, how do we kind of intervene in that research to say, hold up, not only is remote sensing imagery something that we need to think in the same way that we think about photographs critically, but that maps themselves are really complicated documents that we need to have some sort of understanding of the cartographic production in order to work with appropriately. So I'd like to think that when I give a longer workshop here,

[45:08] there's a big emphasis on saying like, if you're working with MapReader, you should be having lots of conversations with the curator. You should be talking, you should be sort of learning the annotation part of MapReader, which you're kind of getting into here, right? So this is a Jupyter notebook going through the patch classification task where, so you kind of created your patches. These are your patches. Then you're going to start annotating in a second. And it's in that process of annotating that not only are you refining that label, right, That sort of finding the meeting ground between the machine, readable, visual information and the analytical concept that is driven by your historical knowledge. But it takes time to figure that out. But it also allows you to spend time with the maps. It allows you to spend time with these patches and get to know that map. And I think that's really essential here.

[46:11] So I hope that kind of answers the second question. The first question is really important here. So for the text on maps, like I said, it's still early days. But I think the work that we're doing with the ordnance survey maps, so kind of in a 19th century British history context, there's been a lot of interest in thinking about trade directories as sources of very granular information about what is on the ground and kind of allowing us to study business histories. There's another chapter in the Living with Machines book about the bicycle industry, which uses trade directories as a source. But this text on maps is, for these maps, geolocated information about not just businesses, but also civic life and residential life. and I'm really excited about what it means to kind of tell us about, in a sense, the,

[47:13] we can come up with kind of profiles of communities driven by an understanding of what was in the built and natural environment as far as that is labeled by the OS surveyors. And that is a way to access a kind of intersection of information that is very rare in any other source. And so I like to think of the map as a kind of synthetic gathering place for information that it's not the only source we're going to use. I think that's really important. I'm not kind of positing that you can kind of work with maps and ignore all your other sources. the map, it's a way of bringing together information that otherwise would not be spatialized and would be really divided in totally different archives. And the other thing, right, is that that means that from the map, it opens up

[48:14] avenues to kind of simultaneously work across scales and work across across social, economic, environmental, technology, histories from the same starting point. And I find that really exciting. As someone whose work kind of spans those fields, you know, in my own career, it's been really powerful to think that from this one source, I can be working on an environmental history project with UK national parks about the complexity of like vegetation around certain and be doing this work about extractive versus rural-based industries with the text in London. So I hate being that person and saying, we're still working on it. It's such an annoying DH thing to do. But for us, the first thing has been to make some arguments about the structure of text before we get to what those structures then allow us to say about

[49:20] the environments that we're working on. And I think that's an important contribution here. It's more of a methodological contribution still at this stage, but until we get beyond just having this gobbledygook of text, remember I showed you that global map of the Rumsey stuff? That data set is so big that you can't open it. You can't download it and open it. To create that image was extremely complicated. There is no way for me to actually open that up. And so there's a whole piece of work, right? And just like, how do we visualize this data? How do we make selections from it that are intelligent? How do we make this useful for people? How do we kind of think about the nature of text on maps in ways that allows us to then answer questions about cities and regions and nations and internationally, right? And what's interesting too about that is it allows us to think both about cartographic processes of including text on maps, but also about what that text reflects on the ground in an appropriate way.

[50:27] I'm sorry, I didn't mean to make it like, what have you found? No, no, it's good. I'm curious, just to sort of, have you thought about doing something with the symbology? Yeah. Because some of those structures, I'm a little less, it's been many, many, many, many, many years since I've ordered survey maps, but I do a lot with USGS maps. Yeah. They share commonalities, and one is just the symbology itself of structures that even if you didn't know that, if it doesn't say PH, you know that's a church. That's new construction. Yeah. maps because of the different coloring. Has that been something that MapReader is on your queue to try to take on? Is it a symbology as well? Yeah, it has. So, sorry, I'm trying to find the right thing, but this is just one example. So, I mentioned the UK National Park. So, with the national parks, there's this huge desire for having the ability to push the kind of baseline for types of land cover and land use in the UK back beyond 2000, which is what basically the

[51:32] government is using right now, which is crazy. And so we're trying to create a kind of historical land use and land cover data set by doing image classification of all the different vegetation types. So, you know, one of the things that often happens in a MapReader workshop is that MapReader, kind of like a portal into starting to think about working with maps computationally. There are lots of other computer vision tasks out there that might be better suited to the task, the questions that you have, and the specific kind of information you're interested in. And if you really wanted to count tree symbols, then object detection is the way to go. But a tree on a map is not a tree on the ground. And so once you sort of know that, then you say, well, do I need to invest in a more, well, you know, a more expensive task to find all the symbols when I could just say, does this

[52:37] patch contain tree symbols? So that's kind of what I'm getting at when I say, do we really need to be super precise? Do we need to use image segmentation, object detection? for the question that we're asking? I think for most historians, not all, for most questions, not all, the patch level classification is sufficient. And it's a lot less onerous compute-wise, which means that MapReader can be run locally. So we do a bunch of HPC stuff, but MapReader can also be used locally You could also use MapReader just to manually annotate. In the notebook I showed you earlier, you could just keep annotating until you annotated all of your patches by hand. You don't actually have to then go train that model. So there's a lot of flexibility built in here, depending on the level of engagement

[53:37] with machine learning or AI that you want to have, the size of your collection, or things like that. Yeah? I was wondering for the search tools, did you try to do any sort of semantic searching? If I search bar, what would like? And we didn't calculate embeddings for word similarity in that data set for that particular purpose. so yeah it's even like you can do multi word searches but there are some restrictions there you can add some fuzziness but that's sort of the level of complexity so far yeah it would yeah the thing that David has implemented now is just that you

[54:39] can query the collection with a chat bot assistant so that's kind of the direction or a slightly more complex query if you had that. But it would be nice, but it was nice. Do you think the chat one will work? Do you feel like? I don't use it a lot myself. But I think, well yeah, David was saying recently that it's getting used quite a lot, you know, thousands of uses a week. So I think it's having an impact. you know one of the things I didn't say today was that the Machines Reading Maps project initially right was about making sure that there's a two-way street between researchers and libraries and creating this data we're creating research data but we're also drastically improving the discoverability of maps and while we couldn't implement that kind of like front-end search experience for the

[55:42] Library of Congress and the National Library of Scotland because you can't change that software overnight, or even on a two-year project, with the rumsey .com website, we could implement that kind of new search interface. It's a big problem, and I know there's many other people in the room who have dealt with this, right? Like, when you create what, in a library context, might be considered as metadata, right? The thing that's going to help you search the collection might be all this text, but where do you put that? There's nowhere to store in current library system software. Where do you put it in the notes field? There's like an average, on the Rumsey collection, there's an average of a thousand words per map. So it's a really complicated software engineering and storage problem on the library side, which a lot of brilliant sort of library and archives people are thinking about, but implementing that is really hard.

[56:43] So we just published these as data, open data sets, and try to hope that just having the data is like the first step towards a future in which collections are more discoverable. Yeah. Yeah, thanks. Hey, sorry to rob a bit late. So you already covered this, just send me on my way. It's okay. But Columbia, a mapping store from New York, has a nice job. We were doing some work, but with street names and creating vectors to create anchor points for georeferencing maps, have you all experimented much with auto-generating anchors for georeferencing on georeferenced maps? Yeah, we did some of that on Machines Reading Maps for Sanborn fire insurance maps, but the kind of uncertainty of the points that were generated

[57:47] was bigger than the map sheet itself. So that was not highly successful. Also, the knowledge base that they were using to link historical names, street names, or any kind of named entity on the map, too, was... Gosh, what was it? DBpedia, which just doesn't match up with historical places. So having better historical gazetteers, thank you, Ruth, is a big part of improving that particular methodology because you can identify features on the map that are likely high-quality linking candidates, but if you don't have a good knowledge base to link to, then it's all going to go wrong very quickly.

[58:50] Yeah. What's made a big difference manually is your referencing stuff for us. it was with Atlanta, where some of the interstates are in that intermediate map. So, you know, index maps. I wonder if you pull points off of that. The individual map sheets, if that would be much higher match rate. Yeah, I haven't played around with that stuff in a while. I'm always really interested to see the latest efforts. And it is better. Like we're going to get there soon with automatic geo-referencing but it really depends on the scale of the map, how old the map is. Like I think it's going to work well for for Sanborns and maybe large-scale maps like these OS maps but these are done now so we don't need to do it again. But you know for other for smaller scale stuff it's it's just a mess and for more historical stuff it doesn't make sense. So yeah I think it really is going to be

[59:53] series dependent. Yeah. Time, okay. If anyone is interested in hearing more about, like, how do I do MapReader myself, please reach out to me on Discord. You can email me. There's a Slack channel, which, which, which, which, which, there is a QR code for that I will find in a second. and maybe there's not. So many slide decks, you guys. Let me know otherwise. Thank you so much for coming. It's been great. Thank you. Yeah, there's the Slack QR code if you're interested.