1. It's All One Big Opportunity
Joanne Cook, Senior IT Support and Development Officer, Oxford Archaeology/OA
Digital
Abstract:
Location runs all the way through archaeology- from the position of an individual find to the
distribution of settlements in the landscape. We do mobile GIS, augmented reality, and high-end
spatial analysis, and it's all open source. Going open has not only helped our company survive in
very difficult circumstances, but has provided new opportunities for innovation, improved
efficiency, and helped us better meet our company goals. We prove that it's possible to run a
professional company in this way, but also to innovate and succeed in an economic downturn. Our
experiences may be of value to other companies faced with a similar situation- can you also turn
adversity into opportunity?
Introduction:
In this paper I'm going to work through a hypothetical archaeological excavation, explaining how
we achieve, or are trying to achieve, the various steps with open source software. I will talk about
the challenges and misconceptions we have overcome, and the opportunities and benefits we
have gained from our approach. I will try and place these challenges, misconceptions and
opportunities into a more general concept, and highlight ways in which I believe other companies
can learn from our experiences.
What's this archaeology thing about- is it like Time Team?
The formal definition of archaeology is the study of past societies through their human remains.
The purpose of archaeology is not only to tell us about our ancestors, but to provide insights into
how we got to where we are now. For example, how did our ancestors deal with (or cause)
warfare, climate change, or epidemics? Archaeology allows us to put our own lives in context- we
are who we are, and live in the society that we live in, because of everything that has gone
before.
A few common misconceptions remain about archaeology. Firstly we don't (often) encounter
dinosaurs, and secondly, unlike on Time Team we usually take considerably more than three days
to study a site. We usually require more than a single piece of pottery to reconstruct an entire
Roman villa too! Thirdly, archaeology is no longer generally the pursuit of elderly retired
gentlemen antiquarians or adventurers, but by highly qualified professionals, often with post-
graduate degrees in their chosen specialism.
In the UK, archaeology is generally undertaken as part of the planning process prior to building
work. Some level of archaeological intervention, be that a simple desk-based investigation of
historic maps through to a full-scale open area excavation, has been a legal requirement of the
planning process since 1990. The majority of archaeological work is undertaken by commercial
units, via a highly competitive tendering process overseen by the county councils, although some
university-based research also takes place.
Archaeology therefore forms part of the complex chain of contractors, sub-contractors and
deadlines that make up the average construction job. Though a very small part of the process,
delays in the archaeological phase, such as from the discovery of something important or
unexpected, can lead to massive delays and therefore penalties later in the scheme.
Consequently, as a result of the tendering process and of integration in the overall construction
AGI GeoCommunity '10: Opportunities in a Changing World
'Innovate - Connect - Succeed'

2. scheme, archaeologists need to work to extremely tight budgets, to strict deadlines, in a harsh
environment, and to liaise and share data with many other contractors. Not only that, but the
type of data we gather and use is extremely varied. As a result, our IT requirements can be quite
demanding, but we are a niche software market, unattractive to traditional software or hardware
manufacturers.
Oxford Archaeology are the largest commercial archaeological unit in Europe. We have 400 staff,
spread over three offices in the UK, and two in France, and are called on to provide expertise at
an international level, in countries as diverse as China, Iraq, Turkey and Tibet.
The requirement for openness
Our data is often the only record that remains of a site, once it has been developed. Our
understanding of the site is necessarily incomplete, due to the limited size of the excavation
trenches, and it is not uncommon for findings to be contradicted by later work, often decades
later. At the very least, later work may place our findings into a wider context. Consequently our
primary data must be kept in perpetuity, so that later researchers can access it, and can repeat
the analytical processes that led to the final conclusions. With a paper archive, this is relatively
straightforward. However, with digital data and devices, the situation is quite different. Either it is
necessary to maintain devices and packages capable of reading the data, and spend time curating
the archive, ensuring that it remains readable, or to choose open, published formats and
programs that will always be readable. The long-term impact of using a non-standard format,
readable only by a particular software package or device, however suitable that is for the
recording task, will be to render our archives useless or inaccessible. Moreover, for the analytical
process to be truly repeatable, at any later point, the algorithms used should be accessible, or the
process is a “black-box”, repeatable only by researchers with the same version of the same
software running on the same operating system.
The Oxford Archaeology “Open Ethos”
Our approach to the requirement for openness outlined above is called “open archaeology”. This
has three parts:
• Open access to our data,
• Open standards for file formats,
• Open source for our software.
As part of this, we are trying to create a work-flow for excavating a site that uses only open
source software.
The archaeological process
In broad terms, the process of excavating an archaeological site has the following phases:
Excavation, Analysis and Publication.
Excavation
Initially, trenches are laid out across the site with professional-grade survey equipment. Within a
trench, buried features manifest themselves as changes in colour and texture of the soil, known
as contexts. Every context is planned (drawn to scale) and recorded before being removed, and
an average excavation may contain several thousand contexts. Artefacts are generally recorded in
relation to the context in which they are found, as that relationship is one of the key ways in
which the development of a site is understood during the analytical phase of the work. Previously
this process would have been almost entirely paper-based, with site records often being
measurable in metres of paper rather than number of sheets.
The search for a device for digitally recording data on site, to replace the paper archive, has been
going on for decades. Approaches to the device problem include electronic paper, ruggedised
AGI GeoCommunity '10: Opportunities in a Changing World
'Innovate - Connect - Succeed'

3. laptops and pdas. Devices must be capable of surviving in a dirty, wet environment, preferably be
operated by cold or gloved fingers, and have good battery life. Old favourites include Psion series
5s, which can run for days on standard AA batteries and will work inside a plastic bag, but these
are now difficult to acquire, technologically limited, and only solve part of the site recording
problem.
Our aim is for a digital recording system that collects tabular data in a database, spatially locates
it, and allows photos to be attached. The database needs to synchronise with a server-based
master version, needs to support multiple users, and also allow device-based data storage in
areas where there is little or no data signal. Having trialled a number of devices over the years,
we have discounted single-function devices in favour of multi-function devices such as pdas and
now smart phones, to keep our overall costs down.
One device that seemed originally seemed very promising was the openmoko phone. This is a
totally open source phone (both hardware and software), with built-in GPS, accelerometer, USB
host capabilities, a very readable screen, and a robust bump-proof body. Unfortunately for
openmoko, the revolution in Android phones pretty much took away their target market, and the
phones are no longer being developed.
Our current target device is a Motorola milestone phone. We issue these to most of our staff, for
use as their phone, and as a digital recording device. Getting good enough batter life from the
milestone is a challenge, but we are investigating solar charging options and it is relatively easy to
provide spare batteries. Bump proof cases, and waterproof bags protect the phones on site. The
increase in cost of the phones themselves (actually not as bad as it seems as part of a company-
wide contract) is offset by savings in data collection costs, both on site and later in the analytical
phase of the project. There are many additional benefits to this approach. Smart phones mean
staff can check their email on the move, which is important as many of our site staff may never
visit one of the main company offices. This helps them stay engaged and feel part of the
company. They are also a conspicuous sign of company investment in our staff, which again helps
with engagement and morale and ensures that staff take good care of them- all important factors
in the present economic climate!
A promising data collection program is Epicollect (http://www.epicollect.net/), that meets most, if
not all, of our data collection requirements. Staff can take geospatially enabled photos of all
features and finds, and immediately attach them to their database entry. This package is being
paired with gvSIGmini (http://confluence.prodevelop.es/display/GVMN), which provides a feature-
rich GIS client. The master database is in PostgreSQL, spatially enabled with PostGIS. Full
enterprise level database functionality ensures our data is secure, and structurally robust.
Analysis
This phase is often the longest part of an excavation. It begins with the transfer of all the records
gathered on site into a database. When site archives are mainly paper-based, this process can
take months, and often errors are introduced simply due to typographic mistakes. When a site
archive is created digitally in the first place, this phase of work can be much reduced. Specialists
then use the archive to reconstruct the developmental sequence of the site, examine the finds,
and place the site in the context of it's wider geographic and historic surroundings. Often this
phase of the project is a high-level analytical process. Examining thousands of sherds of pottery,
or pieces of animal bone for example, requires statistical techniques. Increasingly, geospatial
analytical techniques are used as part of this process. For example, when examining prehistoric
sites where the evidence is mainly artefactual rather than physical, the changes in density of flint
scatters across a site can help pin-point areas of specific activity such as tool manufacture or
animal skinning.
AGI GeoCommunity '10: Opportunities in a Changing World
'Innovate - Connect - Succeed'

4. Previously, Microsoft Access was used as the site database, with specialists (who are often self-
employed and working remotely) being sent their own copy of the database, leading to currency
problems, and a total lack of control over the data structure being used. However, replacing
Microsoft Access completely is not advisable as staff often have many years experience in using it
to analyse data, and are not familiar with alternatives. Our current system is to use Microsoft
Access as the front-end to a PostgreSQL database. In this way, we maintain control and security
over the data, we can ensure all staff are seeing the most up to date version, can control who has
the ability to delete or change records at a very fine-grained level, but also allow staff and
specialists the familiar interface to the data.
Geospatial analysis was previously done using ArcGIS, but the cost of licensing and training meant
that few staff had regular access and the skills to use it. Until current versions, it has also not
been possible to connect to PostgreSQL databases without yet another expensive plugin. We have
now migrated almost entirely to Quantum GIS and GvSIG as our desktop GIS packages, both of
which have arguably a more powerful set of geospatial analytical tools (GRASS and Sextante
respectively), and natively connect to PostgreSQL.
Publication
The final phase of a site is to check that all of the conclusions are logically consistent, and bring
everything into a single coherent narrative. The report of the excavation is published either as a
basic site report (commonly known as “grey literature”, held by the County at a Historic
Environment Record (HER), or as an academic monograph, or even sometimes as a “popular
publication” aimed at the general reader.
The various levels of publication have different requirements. “Grey Literature” reports can be
created in a word processing package, with illustrations generated in the GIS as necessary.
Monographs and “Popular Publications” require a full desktop publishing package. We are in the
process of converting to Open Office for our word processing/spreadsheet needs, and for some
cases, to Inkscape and GIMP for creating illustrations.
Challenges
It would be naïve (and false) to suggest that there haven't been any challenges along the way-
and some still to overcome.
• Changing the work-flows of a company of approximately 400 staff, many of whom have
been doing their job for decades is not easy. It is necessary to win hearts and minds, and
to get engagement at all levels. Simply making a financial argument is not enough, as
people see the process causing them more work, or forcing them to learn new skills, and
abandon old ones (the flip-side of this is that it highlights the difference between real skills
and simply knowing which button to press).
• Misconceptions of staff, clients and other contractors that the open source approach leads
to bad data. We have encountered a complete lack of understanding of the difference
between software and data at all levels of the archaeological and construction process.
Often specific software packages are mandated in tenders for work, when what is actually
required is a specific software format. At the very least this requires us to word our
submissions extremely carefully in order to avoid losing work because we do not use
ArcGIS!
• Problems are often blamed on the open source software where previously they would have
been blamed on something else (or not blamed at all). Software crashes all the time,
everyone knows this. However people tend to be a lot more critical of open source software
than it's closed source compatriots. The reason for this is unclear- it may simply be due to
AGI GeoCommunity '10: Opportunities in a Changing World
'Innovate - Connect - Succeed'

5. the perception that open source software is somehow not as high quality as proprietary
software (it can't be, otherwise it calls into question why one would pay for the proprietary
alternative). Furthermore with packages such as Open Office, every formatting issue is the
fault of the software. Every previous incident of this kind with Microsoft Office is
conveniently forgotten.
• Changing work-flows does need an investment of time (and therefore money). From
software installation to staff training, this process has taken time, and has been criticised
as a result. However, many of these so-called “additional costs” are comparable to those
that would be entailed by moving from (say) one version of Microsoft Office to another, or
from ArcView to ArcGIS. Many of these costs are also offset by the lack of licensing costs,
and by efficiencies that re-examining the work-flow brings.
• Dead ends. Particularly with the search for a device for digital recording on site, there have
been a number of false starts. The openmoko phone is an example of this- although since
it is possible to run the latest Android operating system on them, they make excellent
development devices!
• Limitations of the software. Some of the open source packages we use do have rough
edges, as they have been designed to meet a particular need and time has been spent
improving the functionality rather than the polished interface. Unfortunately this leads to
an undeserved impression that the software is somehow not up to scratch. GvSIG is an
example of this. In it's native form it is a Spanish package, and the translation to English is
not as good as it could be, but functionally it is a superb piece of software. Part of our task
in choosing it as one of our desktop GIS packages has been to improve the translation.
Tools without a GUI, such as ogr2ogr, are shunned because of an innate fear of the
command-line (perhaps a throwback to the DOS years). Quantum GIS and GvSIG both
lack the ability to create really high-quality cartographic output, but we circumvent that by
exporting the basic figure as a PDF or SVG and finishing it in InkScape. Initially this
process added to the amount of time required to create a figure, but as staff familiarity
with the software has improved this is no longer an issue.
• Lack of open source cad packages. While a traditional site suits a GIS-based approach, for
some of our work, such as building survey, a CAD-based approach is more suitable.
Furthermore many of the engineers that we work with on road schemes and other large-
scale construction projects use CAD rather than GIS. While there are a number of open
source alternatives, we are yet to find one that fulfils all of our requirements, and keeps up
with the annual change in the DWG format.
Opportunities
• Greater staff engagement. The elimination of licensing costs and restrictions has meant
that we can roll the software out to a much larger proportion of our staff, and afford to
train them. Working through issues, learning a new package, and helping to develop work-
flows have caught people's enthusiasm, and made them more committed members of
staff, considerably better at their job.
• Consultancy. The open source geospatial market is still small, and lacks any major players
to dominate the market. Oxford Archaeology's open approach has garnered enough
attention that we are called upon to provide support and advice to other companies looking
to incorporate some open source software into their work-flow. We are of course living
proof that it is possible to work in a high-pressure collaborative environment and to use
open source software, with only a small number of hiccups (see above).
AGI GeoCommunity '10: Opportunities in a Changing World
'Innovate - Connect - Succeed'

6. Our consultancy arm doesn't just provide support for open source software, however. For
example, we are using the new technologies that the company is investing in, such as the Android
phones, to investigate augmented reality applications, for doing historical “virtual tours” based on
medieval maps and archaeological evidence.
• Cost savings. There are enormous financial savings to be made with an open source
approach for many companies, although these can be hard to quantify due to the costs of
implementing the new software, training new staff, providing support, and so on.
• More control- no licensing changes. For educational charities the proprietary software
market is becoming a lot more strict about licensing. Over the last couple of years, many
software providers have changed the terms of their licensing agreements to bar
educational charities from receiving a discount. It is a moot point whether or not this is
fair, but at the very least it leads to a large increase in the software budget, often at very
short notice- simply to continue using the software that they have. Moving to open source
software does at least ensure that this will not happen in the future.
Conclusions
For Oxford Archaeology the open source approach has been, and continues to be a success,
helping us to survive in a very difficult industry during an economic downturn. It has also led to
many positive factors, such as more engaged and skilled staff, and opportunities, such as our
flourishing consultancy business. We do not believe that our approach will work for everyone, in
fact our position, even within the archaeological industry, is unusual. However, we do believe that
many companies in different industries can learn something from our experiences, particularly in
the current economic climate.
For example, large-scale change of software and work-flows might seem untenable, expensive,
and difficult for staff. However, it is an opportunity for a truthful assessment of what functionality
is actually needed- as prior assumptions about how much of a particular package are used, and
why, may be wrong. Are staff skilled in achieving tasks with a particular software package, or do
they have a true understanding of their area of expertise, with flexibility and transferable skills? If
the former, then even upgrading to a later version of the same software may be costly.
Furthermore, true ownership costs should be taken into account when considering a move to a
different software package, and it's fair to say that these are often under-estimated when open
source software advocates promote their favourite software. However, the cost of remaining with
key proprietary packages who perhaps change file format every couple of years, or who demand
better and better computer specifications as a minimum, should also be considered.
Our key advice is that people should question their software use, licensing, costs and
requirements to ensure that they truly get the best value, both from the software, and their staff.
In many cases, an honest assessment of requirements may lead to a decision to investigate open
source alternatives. Our success over the last few years proves that it is possible to use open
source software in a high-pressure environment, doing advanced analysis, and we would like to
see other companies reap the same rewards.
AGI GeoCommunity '10: Opportunities in a Changing World
'Innovate - Connect - Succeed'

7. Our consultancy arm doesn't just provide support for open source software, however. For
example, we are using the new technologies that the company is investing in, such as the Android
phones, to investigate augmented reality applications, for doing historical “virtual tours” based on
medieval maps and archaeological evidence.
• Cost savings. There are enormous financial savings to be made with an open source
approach for many companies, although these can be hard to quantify due to the costs of
implementing the new software, training new staff, providing support, and so on.
• More control- no licensing changes. For educational charities the proprietary software
market is becoming a lot more strict about licensing. Over the last couple of years, many
software providers have changed the terms of their licensing agreements to bar
educational charities from receiving a discount. It is a moot point whether or not this is
fair, but at the very least it leads to a large increase in the software budget, often at very
short notice- simply to continue using the software that they have. Moving to open source
software does at least ensure that this will not happen in the future.
Conclusions
For Oxford Archaeology the open source approach has been, and continues to be a success,
helping us to survive in a very difficult industry during an economic downturn. It has also led to
many positive factors, such as more engaged and skilled staff, and opportunities, such as our
flourishing consultancy business. We do not believe that our approach will work for everyone, in
fact our position, even within the archaeological industry, is unusual. However, we do believe that
many companies in different industries can learn something from our experiences, particularly in
the current economic climate.
For example, large-scale change of software and work-flows might seem untenable, expensive,
and difficult for staff. However, it is an opportunity for a truthful assessment of what functionality
is actually needed- as prior assumptions about how much of a particular package are used, and
why, may be wrong. Are staff skilled in achieving tasks with a particular software package, or do
they have a true understanding of their area of expertise, with flexibility and transferable skills? If
the former, then even upgrading to a later version of the same software may be costly.
Furthermore, true ownership costs should be taken into account when considering a move to a
different software package, and it's fair to say that these are often under-estimated when open
source software advocates promote their favourite software. However, the cost of remaining with
key proprietary packages who perhaps change file format every couple of years, or who demand
better and better computer specifications as a minimum, should also be considered.
Our key advice is that people should question their software use, licensing, costs and
requirements to ensure that they truly get the best value, both from the software, and their staff.
In many cases, an honest assessment of requirements may lead to a decision to investigate open
source alternatives. Our success over the last few years proves that it is possible to use open
source software in a high-pressure environment, doing advanced analysis, and we would like to
see other companies reap the same rewards.
AGI GeoCommunity '10: Opportunities in a Changing World
'Innovate - Connect - Succeed'