Computing, GIS and Archaeology in the UK

Once in a while, Oxford Archaeology get called upon to do some really big archaeological projects, like road schemes and airport expansion, that cover huge areas, go on for years, and generate loads of data. We love these, because it’s not very often that you get to look at whole landscapes- how multiple prehistoric villages interact, for example, rather than tantalising snapshots where you have to play “join the dots”.

Not only do we generate loads of data (thousands of artefacts, records, environmental samples, photos etc), but we ask difficult questions of them, based on statistical analysis of finds distribution, travelling salesman algorithms, best-path analysis. I think you can call this “real GIS”, though often people are surprised to hear that archaeologists use GIS at all.

As part of our ongoing “open approach“, and to prove that we put our money where our mouth is, we are now trying to do all of this in open source software rather than using the “standard” proprietary packages.  I’ve blogged previously about how pleased I was with the integration between QGIS and PostgreSQL, and how easy it was to manage large amounts of data without regress to proprietary packages, well  our current large project needs 3D analysis and large amounts of imagery manipulation, and again we’re finding that the open source tools out there do the job splendidly. Furthermore, we have a choice of tools, so if one approach doesn’t quite work the way we expect or want, then we can choose another. Now that can’t be bad, can it?

So, in brief, we’re using Quantum GIS and GvSIG pretty much interchangeably for our desktop GIS. All the vector data is in PostgreSQL. We use the QGIS Grass plugin to get data into a sensible format for 3D display and analysis in Paraview and Visit. We’re mosaicing up aerial photos using GDAL tools, and using Geoserver to publish everything to people who just need read-only access, and a direct connection to PostgreSQL for those that need to edit. We’ve developed a workflow for creating high-quality cartographic output by exporting to Inkscape, and the next step is a project website with links to our database and a nice openlayers map. Simples!

I’m helping to teach at a workshop on open source GIS at the University of Leicester in a couple of week’s time. As usual, this means running around trying to get all the software that we need installed on the university computers. As usual, what the course organisers think will be OK, and what the IT department think will be OK, are two different things!

We’d rather not use a LiveDVD, as we want the students to work in the environment which they are used to- in this case windows. We don’t really want to saddle the IT department with lots of PostgreSQL and Apache configuration, so we’re going to try using Portable GIS (gulp).

The current iteration (as available here), has an out of date version of Quantum GIS, doesn’t contain the statistical package R, and also contains a bunch of other stuff that we won’t use on the course (MySQL, GvSIG etc). The beauty of open source cross platform software though, is that rolling a customised version, containing just what software we need, and the latest versions, was quite easy.  Adding in the data, and even the course notes, will be straightforward (once we’ve written them, of course!). Installation is simply a case of copying everything onto each pc, and the students can take it all home with them when they are done.

The open source license is not the big deal here, despite the title of the post, but the added benefits that it brings are pretty cool. Free software means we can run the course without having to worry about buying licenses, or sending students home with limited demo versions afterwards. The cross-platform nature of the software means most of the donkey work is done in human readable files that can easily be edited to work in a portable fashion, and finally, because it’s open source, we can do that legally.

I think that’s all pretty cool…

(Small Plug) If you like the idea of using Portable GIS for a course, but don’t fancy customising or preparing it yourself, then get in touch!

There’s a fantastic article in today’s Guardian (via Computer World Magazine) about the International Intellectual Property Alliance, who say that countries advocating the use of open source software should be put on a “Specialist 301 list” (ie a trading watch list) because open source “weakens the software industry” and “fails to build respect for intellectual property”.  The IIPA is an umbrella group for organisations like the RIAA and the MPAA, who are of course well-known for their open-mindedness and forward thinking.

Go read the articles, I’ll wait…

As well as the obvious lack of understanding of, well, anything, this is all quite bizarre. It implies disapproval of almost the entire internet infrastructure, as well as the big (US) players like Google. It shows a complete lack of understanding about companies (in the US) making a revenue from an open source business model, or even including open source software within their offerings. And how you can claim that the open source licenses somehow harm intellectual property is beyond me.

Oh well, I guess that puts the British Government on the watchlist, as well as the US government, and presumably the IIPA need to put themselves on if they use Apache at all…

This week I have actually been doing some real GIS work for a change, rather than going to meetings, writing bids, writing reports, fixing computer problems and showing other people how to do stuff. I think this is the first time in approx 2 years that I’ve done this, and I was pathetically excited about the prospect at the beginning of the week.

It has also been an opportunity for me to really put my money where my mouth is, regarding using open source GIS, since last time I did some real analysis it was with the Redlands offerings. So, I loaded up PostgreSQL and PostGIS, and Quantum GIS with the Grass plugin and Shapefile to PostGIS Import Tool (SPIT), and wrangled half a million polygons of historic landscape data into submission (ie merged, dissolved, reclassified, cut, pasted and cleaned).

I have a confession to make. It was easy! It was quick! I hardly had to go near the command line (with the exception of creating indices and merging tables in postgis).  OK, I had a few crashes (mainly python errors in windows) and I had to try a couple of different approaches to get my dissolves and merges to work, but I would expect that with any program dealing with large amounts of data.

I’ve been evangelising about open source GIS for a number of years now, but until now I’ve had to take other people’s word on the performance aspect. It’s always nice to get your own personal confirmation about something (albeit in a totally un-scientific, non benchmark sort of way), and even better, to have it exceed expectations.

So, to all you developers out there- thanks!