Computing, GIS and Archaeology in the UK

Once in a while, Oxford Archaeology get called upon to do some really big archaeological projects, like road schemes and airport expansion, that cover huge areas, go on for years, and generate loads of data. We love these, because it’s not very often that you get to look at whole landscapes- how multiple prehistoric villages interact, for example, rather than tantalising snapshots where you have to play “join the dots”.

Not only do we generate loads of data (thousands of artefacts, records, environmental samples, photos etc), but we ask difficult questions of them, based on statistical analysis of finds distribution, travelling salesman algorithms, best-path analysis. I think you can call this “real GIS”, though often people are surprised to hear that archaeologists use GIS at all.

As part of our ongoing “open approach“, and to prove that we put our money where our mouth is, we are now trying to do all of this in open source software rather than using the “standard” proprietary packages.  I’ve blogged previously about how pleased I was with the integration between QGIS and PostgreSQL, and how easy it was to manage large amounts of data without regress to proprietary packages, well  our current large project needs 3D analysis and large amounts of imagery manipulation, and again we’re finding that the open source tools out there do the job splendidly. Furthermore, we have a choice of tools, so if one approach doesn’t quite work the way we expect or want, then we can choose another. Now that can’t be bad, can it?

So, in brief, we’re using Quantum GIS and GvSIG pretty much interchangeably for our desktop GIS. All the vector data is in PostgreSQL. We use the QGIS Grass plugin to get data into a sensible format for 3D display and analysis in Paraview and Visit. We’re mosaicing up aerial photos using GDAL tools, and using Geoserver to publish everything to people who just need read-only access, and a direct connection to PostgreSQL for those that need to edit. We’ve developed a workflow for creating high-quality cartographic output by exporting to Inkscape, and the next step is a project website with links to our database and a nice openlayers map. Simples!

A couple of weeks ago was the OSGIS 2010 conference at the Centre for Geospatial Sciences at the University of Nottingham. This was expanded on the previous year’s event- it was spread over 2 days with workshops on the first day and the conference on the second. We also held the 2nd AGM of the UK OSGeo local chapter after the conference (a shameless attempt to get as many people as possible to attend).

The workshops went well- I reprised my “Databases and Web Mapping the Open Source way” workshop using the OSGeo Live DVD in the process. If you last tried Live DVDs a few years ago and think they are going to be unworkably slow and clunky then think again. The Live DVD team have done a very good job with the OSGeo version too!

As is usually the case when you’re involved in running an event, my memory of the actual papers on day two is somewhat hazy. Stand-out themes were TinyOWS and GvSIG in my book. The GvSIG team in particular were there in force, and very impressive.

The local chapter AGM went well, although I’ve committed my first mistake in calling it the local chapter, as we decided that was a bit too hells-angels, so we’re now called OSGeo:UK (branding updates to follow). Perhaps due to my (even more) shameless emotional blackmail in my introductory talk, there was a lot of new interest. We now have a mailing list of over 100 people for a start. The mailing list is actually the best place to go for my quick round-up of the AGM- check out the archive for more details. In particular though, if you are involved in promoting any workshops using open source software and you want some publicity then get in touch as we want to get a good list together of all the events that are taking place in the UK.

I followed up events in Nottingham with assisting at another two-day workshop in Leicester, at which I repeated my “databases and web mapping…” talk yet again, alongside talks on spatial statistics with R, and QGIS/GRASS. My talks for all these events are up on slideshare if you’re interested, and will follow in the downloads section of this website when time permits.

I’m helping to teach at a workshop on open source GIS at the University of Leicester in a couple of week’s time. As usual, this means running around trying to get all the software that we need installed on the university computers. As usual, what the course organisers think will be OK, and what the IT department think will be OK, are two different things!

We’d rather not use a LiveDVD, as we want the students to work in the environment which they are used to- in this case windows. We don’t really want to saddle the IT department with lots of PostgreSQL and Apache configuration, so we’re going to try using Portable GIS (gulp).

The current iteration (as available here), has an out of date version of Quantum GIS, doesn’t contain the statistical package R, and also contains a bunch of other stuff that we won’t use on the course (MySQL, GvSIG etc). The beauty of open source cross platform software though, is that rolling a customised version, containing just what software we need, and the latest versions, was quite easy.  Adding in the data, and even the course notes, will be straightforward (once we’ve written them, of course!). Installation is simply a case of copying everything onto each pc, and the students can take it all home with them when they are done.

The open source license is not the big deal here, despite the title of the post, but the added benefits that it brings are pretty cool. Free software means we can run the course without having to worry about buying licenses, or sending students home with limited demo versions afterwards. The cross-platform nature of the software means most of the donkey work is done in human readable files that can easily be edited to work in a portable fashion, and finally, because it’s open source, we can do that legally.

I think that’s all pretty cool…

(Small Plug) If you like the idea of using Portable GIS for a course, but don’t fancy customising or preparing it yourself, then get in touch!

I went to the Open Knowledge Foundation conference, OKCON in London a few weeks ago, and have been meaning do a review of it ever since. Whilst little of what I saw had a direct relevance to what I do, it was invigorating to be in a room with a whole bunch of people with imagination, who believe knowledge should be free to anyone, and who basically like to disrupt the status quo.

A few quotes:

[2009 was] the year open data went mainstream (Rufus Pollock)

The threat [to the record industry] is not piracy, but obscurity (Glynn Moody)

What would happen if every school had a reprap? (Ben O’Steen)

One paper that was quite relevant to me was “Dig the new breed” by Anthony Beck, about opening up archaeological data. There are related articles and mailing list posts here and here. I feel uneasy about some of the details (it’s not lethargy or ethics that dictates the data we do or don’t release, it’s money) but in general it’s a no-brainer. We’re doing our bit with our Eprints library, but we’re at an early stage with getting reports on there.

The only problem I see is figuring out who to lobby- I would add the developers and the county-level curators to the list as well as the actual archaeological units. I know of specific cases where developers would not provide the money for making the results of an excavation public access, and also where a regional Historic Environment Record would not allow “their” data to be shown on a web map.

Otherwise, I particularly enjoyed Glynn Moody’s paper on The Post-Analogue World- focussing on the “plight” of the record industry, struggling to cope with the transition from analogue to digital, and Ben O’Steen’s paper on Making the Physical from the Digital. Bookbinding, repraps, MP’s expenses and Cory Doctorow all in one talk. Can’t be bad!

Many of the talks are available to download here, and OKFN have working groups for both archaeology and geospatial data if you’re interested. I hope to have more involvement with both, and to investigate links between OKFN and OSGeo, now I have some time and mental space. More to come…