Open Context has grown into one of the world’s largest open access databases of archaeological data. It now houses nearly 2.4 million records from over 200 data publications, including 215,000 images, maps, documents, 3D models, and GIS layers. This growth has required us to make important performance improvements so that the user interface has acceptable speed and responsiveness. Backend database updates enable Open Context to continue to grow and respond to searches and other queries quickly and efficiently.
Needless to say, the growth of AI has led to profound changes in the information ecosystem of the Web. Many users now discover content from Open Context through using ChatGPT and other AI services. In that sense, AIs sometimes act like search engines by directing users to relevant and unique information in Open Context.
However, unlike search engines, AIs have a voracious demand for training data. This demand has led to increasing levels of traffic. AI software agents constantly request data from Open Context. These software agents (bots) often ignore our directives and request too much data too quickly for our systems to handle. The unsustainable level of traffic made by AI software agents degrades performance and can lead to crashes, much like a denial of service cyber-attack.
Until Internet governance catches up with this new reality, we find ourselves in an arms race to defend Open Context from AI Web-scrapers. Fortunately, the new security measures we installed in 2025 seem to work for the time being. Open Context is currently no longer overwhelmed by requests from AI bots. However, this experience highlights how the challenges and opportunities afforded by AI technologies will surely occupy much of our attention in 2026!
