Computing, GIS and Archaeology in the UK

On Friday I attended a one-day conference at the e-Science Centre at Edinburgh University, entitled “Maintaining Long-term Access to Geospatial Data“. The quote above came from one of the speakers and, to me, it was the key point that came out of the day. Having said that, some interesting themes emerged that align nicely with ongoing discussions amongst the geospatial crowd.

Firstly- the overwhelming trend in all the discussions was a move towards a service-based decentralised architecture for geospatial data storage and sharing. This requires good and standard metadata, and of course a transparent Spatial Data Infrastructure (SDI). The title quote refers to the fact that we haven’t really suceeded in either of those two areas yet, for data sharing now, let alone 50 or 100 years into the future.

Steve Morris, from the North Carolina Geospatial Archiving Project (NCGDAP) said that the establishment of an SDI should be seen as a catalyst for future discussion, and presumably experimentation/innovation within the geospatial community. I find this encouraging- as trying to create something that suceeds right from the word go is daunting, if not totally off-putting. In a later discussion, James Reid from Edina asked whether we should be trying to create global or national SDIs, or whether more localised or sector-based approaches might not be better. At the risk of over-using a phrase, should we be Thinking Global, but Acting Local? Given recent discussions about the lack of success of many current SDIs, perhaps this approach might be the way forward. Certainly I would like to see both an archaeological SDI, which could be global or national in scope, and a UK SDI focussing on datasets with national coverage. In particular, I would like to see the establishment of an SDI as an aim of the OSGEO Archaeology Special Interest Group (follow the link if you’re interested).

Clearly a good SDI is totally reliant on good metadata. Without going into a long discussion on the merits and pitfalls of metadata, there is a tendency to overdo the amount required, making it unlikely that people will produce any at all. There is also a tendency to invent entirely new standards, making it more difficult for data producers to know what they should be complying with. Tony Mathys, again from Edina described something emminently sensible in his discussion of the Go-Geo! Project, which is the creation of sector-based profiles, all of which contain a small number of required core elements, and additionally only those elements that are relevant in that sector. Again, I would like to see something like this created for archaeology, particularly in the UK, but I don’t want us to reinvent the wheel when we could create an “archaeology profile” from existing standards.

Finally, there was a suggestion that producers of geospatial data are generally “temporally impaired”. In other words, we rarely consider future uses or interpretations of our data as we are concentrating on fulfilling a particular objective in the present. We rarely maintain archives of old versions of files, and our metadata tends to refer to a snapshot in time. At OA we try and get around these issues by maintaining an archive of geospatial files, when storage allows it, and we try to keep a “lab notebook” detailing the work process (and hence any changes made) throughout a project. Storage space is an issue though, as is educating people about the need to do this.

Overall, this was a very useful conference to attend (the food and facilities were great too), and it provided a very useful insight into how larger institutions such as the UK Higher Education bodies are addressing issues of data sharing and archiving. For OA, the main thing is to keep moving forward, get our own SDI running smoothly, get sharing data with our colleagues elsewhere, and produce the right metadata to go with it all. Phew!

Update 16th November: There’s a report and presentation of the conference available.

In my spare time/lunch times I’m in the middle of a major project at the moment, to update our site database. Without going into the gory details of how it ended up in three separate, totally unlinked databases, it is supposed to document the archaeological sites we’ve worked on since the 1970′s, and to help with the administration of project archives, the location of finds within our finds store and so on.

For a long time I’ve wanted to sort this out, pull everything into one place, display the site locations on a web-based GIS and so on. This is not rocket science, but the data had ended up in such a bad state that I couldn’t see past fixing that before getting to the good and fun stuff. Attempts a few years ago to use Mapserver as a mapping interface were abandoned because the positional data for the sites was wildly inaccurate (we seemed to do a lot of work in the Scilly Isles, which are as close to 0, 0 on the British National Grid as you can get), and connecting to Microsoft Access was quite difficult and unstable. If anyone using a Windows XP machine tried to open the database, my map would not display. None of those problems are Mapserver’s fault of course, but you can’t roll out a map that has inaccurate data on it, and might not alwats work.

So, after a few years of muttering about this, I have finally bitten the bullet. I am working, in stages, towards an integrated database for all of our finds and archive information, in a PostgreSQL database, with both an Open Office base front-end for querying, and a web-based map interface. I’ll blog about each stage in this process, starting with integrating and data-cleansing in Microsoft Access.

The title of this post refers to the incredible feeling of achievement that I had when I got all this data together, followed by the difficulty I’m having moving that data into PostgreSQL. More later…

The science might be a little dodgy (No more methane at all? What about all that decaying plant matter?), but I think this image is great. You’ve got to wonder what alien archaeologists will make of our heavy metals and particularly stubborn plastics though, in a time as far ahead of now as we are ahead of the palaeolithic…

(Goes off in a fit of navel-gazing about the lack of permancy of anything…)

There have been a few heads-up over the last week about 52°North, who have just announced an initiative for geospatial open source software. Of course, we’ve heard this before, but this time ESRI are on board.

So, we have Autodesk supporting OSGEO, and ESRI supporting 52North’s initiative (it needs a snappy title or acronym). The approach seems different though, because Autodesk chipped in right from the word go by open-sourcing one fork of MapGuide, which they freely admitted was not a core product. It appears however as if ESRI are not looking to open-source one of their existing products, but to help to develop new technologies, currently in the areas of Sensor Web Enablement (SWE), Security and Digital Rights Management (to quote from the press release).

Personally, I can’t decide if the big guys are squaring up for an opensource fight, or if we’re just seeing much more involvement in the opensource movement. Following on from the recent discussion about the use of opensource alternatives to ESRI’s main product lines, and further back, ESRI’s decision to include support for PostgreSQL as a database, hopefully this is all a sign of a gathering momentum. At first I thought this 52 North initiative was a bad thing- a dilution of focus, but maybe it’s just a sign that the snowball’s getting that bit bigger. More products, and some healthy competition has to be good, right?