Open Context launched in late 2006, roughly 17 years ago. Since then we’ve worked on some 180 project datasets now in various stages of completion. Currently, we have 134 projects complete enough to include in our main project index.
Each of these publication projects required dedicated and sustained commitments of effort. This work includes: data cleaning; extraction of records from source databases for transformation and loading into Open Context; metadata documentation; and standards annotation. At the same time, we needed to develop and maintain the software that supports these workflows, including user interfaces, search features, and integrations with digital preservation services. This necessary effort means that Open Context publishes only a tiny fraction of the archaeological data that our community routinely generates. Despite this limitation, it’s still worth exploring possibilities of what can be done with this relatively (for this domain) large and diverse archaeological dataset.
Enter AI…
AI has captured a great deal of attention over the past few years, especially with the launch of ChatGPT. Computationally oriented archaeologists have also experimented with AI for everything from detecting archaeological sites from remote sensing data to detecting relationships between people and organizations involved in antiquities trafficking.
Training AI models requires large datasets. But developing large datasets is hard to accomplish in archaeology because individual archaeological datasets tend to be one or more of the following: (1) small, (2) complicated, (3) created by people with little formal database training, (4) idiosyncratic and inconsistent, and/or (5) not shared. Open Context helps address some of these issues by providing a platform where publishing small, individual datasets contributes to building a larger, integrated database. Over the years, Open Context has grown to a non-trivial scale of more than 2 million records.
Open Context Image Data and AI
2 million records may sound like a lot, but only small slices of this overall corpus can be useful for a given AI training project. We wanted to explore if we could use image and artifact data published by Open Context for AI and machine-learning.
Of Open Context’s 2 million records, around 77,000 are image records that describe artifacts. Each image is associated with artifact records describing styles, types, materials, and other characteristics. Most of these descriptions are inconsistent because they come from several different projects, each with different recording protocols. However, Open Context also annotates data with shared vocabularies as part of the editorial and publishing process. For example, the Getty Art and Architecture Thesaurus (as well as other metadata) provides more consistent description of this 77,000 image corpus.
Since this image corpus shares some common elements of description, we can (hopefully) train a machine-learning model to associate patterns in images with textual descriptions of artifacts. This can be used to enhance image search in Open Context and possibly even support AI image recognition services. It would be very cool to enable “reverse image search” features that allow you to upload an image (say of an unusual object discovered in excavation) to find possible comparanda that could aid with the object’s identification. There are probably many other applications that may also emerge.
However, some caveats need to be considered!
Those 77,000 images are not evenly distributed in time and space. The vast majority describe artifacts from excavations and surveys in Europe or Southwest Asia. So it is a heavily biased sample dataset.
Moreover, the images themselves vary. Some images are actually digitized ink drawings and some photographic images document highly fragmented and weathered sherds collected in surveys. Some are more “museum style” and lack scale bars. Some have scale bars, but have objects displayed against a variety of backgrounds, and some have backgrounds entirely removed so objects appear to float in a blank void. A human may regard some of the variations in imaging as incidental. But will a machine model get sidetracked by these irrelevancies?
Starting our Experiments
Open source tooling for AI and machine-learning now make more casual experimentation easier for relative newcomers (like us) to this space. In fact, we got inspired to start these experiments because of some very informative blog posts by Simon Wilson that provided a great “how to guide”. A couple of weeks ago, we generated a data extract from Open Context. We’re now at the stage of “plumbing” together some of these open source libraries to fine-tune an off-the-shelf AI image model (called CLIP) using the image data extracted from Open Context. Our colleague Shawn Graham is madly scripting different experiments to see what may work. He’s made some real progress too! Please stay tuned, or follow our progress directly here at this Github repository and here. [Update 2023-10-11, Shawn blogged about his experiments here]
If we can make some progress with fine-tuning CLIP for archaeological image recognition, how could we build on this work? Could we expand it to include data from other sources beyond Open Context? How difficult would it be to compile comparable data from archaeological data repositories like tDAR or the Archaeology Data Service (ADS)?
Museum Data too?
Also, some museums offer open data, including image data. How can we use machine learning with all of these data sources? In some ways, museum data may look very different from “field data” curated by Open Context. The data in Open Context mainly documents (let’s be honest) mundane ancient trash. In contrast, objects that enter into museum curation are typically much more special and rare– heavily biased to represent impressive works from elite, mortuary, or ritual contexts. Sometimes what we find in excavation may have once looked like those museum pieces, but they are a shadow (or a fragment!) of their former selves. Could an AI learn to recognize these as similar?
Considering the history and motivations behind museum collecting also raises important ethical questions. Many major museums have large and geographically diverse collections because of legacies of colonialism. Is it right to use digital representations of these “ill gotten gains” to fuel AI models? What kinds of governance and data sovereignty issues need to be addressed first?
These are all important questions to explore. AI feeds on data. Lots of data. Commercial interests are already heavily investing in these technologies. Archaeologists and other cultural heritage professionals need to engage in AI debates and voice our perspectives and ethical concerns in this rapidly developing sphere.
