Computing, GIS and Archaeology in the UK

In that serindipitous way that rss readers work, two posts came to my attention over the last couple of days. The first was from Gavin, about problems that occurred when the South African Government failed to keep control of the source code on two GIS programmes that they had developed. When contracts end, or funding dries up, if you don’t have complete control over your programmes then you might as well start rebuilding them now.

Today my fellow trouble-maker and open source advocate Joseph reported back on a Freedom of Information request that he made to English Heritage, the leading heritage body here in the UK, regarding the long-term sustainability of their (proprietary) Intrasis site recording software. I quote:

“Intrasis is developed by the Swedish National Heritage Board
(Riksantikvarieämbetet), this organisation has a long-term commitment to
the continued use and development of this software package. Our present
contract is for four years, and the use of the package will be the subject of
a post-implementation review after two years.”

So, lucky we only need to keep the records for 6 years then. After all, it’s not like they are the only record remaining of the archaeology that has now been dug up. Oh, wait…

(For those of you not from the UK- you might not know about the super-fab WhatDoTheyKnow site. It’s for submitting Freedom of Information Requests to UK public authorities)

This is all extra serindipitous (how often do you get to use that twice in one blog post?) because I’m trying to write my talk for the AGI 2008 conference at the moment. I want to try and get across that there are just as many, if not more, “risks” in a proprietary software solution as in open source. I suspect that government agencies aren’t ever really going to be taken to task over these decisions and this potential waste of public money unless we can get open source software understood at the highest of levels. Well, it’s a challenge…

A couple of people have remarked to me that one reason that they are not prepared to make the open source jump is the lack of support.

Well, last week I wrote about a mastermap importer that I had found. I said I would like to be able to merge the huge numbers of shapefiles produced into more manageable chunks. I got a lot of good advice as to ways I might be able to do that using gdal, which is really handy, BUT I also emailed the developer about the same problem. Two days later he released a new version of the software with that functionality built in.

To me, this highlights one of the big advantages of open source. People want to make their software better, and are willing to listen to suggestions. Small firms or one-man bands don’t have the inertia of the major firms and can respond to requests quickly and easily. To the user, that means flexibility and control. In other industries, that would be seen as an advantage, not a disadvantage, so why not in software?

I’ve been following the “Timmy’s Telethon” debate with great interest, as, I’m sure, a lot of people have. I’m a little surprised that there hasn’t been more comment from different blogs about it, but there we go.

The whole debate struck a chord with me though, for the simple reason that I am involved in the process of trying to change the mindset of an organisation from a reliance on closed source software to one where open source alternatives are used, where possible and appropriate. The trouble is, some people don’t care about the difference as long as they have the tools to do their job, or they don’t understand the fundamental philosophical difference between free beer and free speech, or both. Furthermore, a number of people have asked me what the business model for open source is and why it matters to us as an organisation whether we embrace an open philosophy or not.

Both Timmy’s original comment, and Paul’s spirited responses are really useful to me as a way of highlighting the contrasting points of view, and articulating the reasons why we (particularly as archaeologists with our remit of preservation by record) should embrace the open source philosophy. However, from my point of view, this article (via my colleague Lucian) shows exactly why we should be avoiding the big guns at all costs. It’s old news now, but I’m going to show it to all my doubting colleagues who are wondering why we make so much of a fuss about it all!

This is an idea that seems to come up every so often- what GIS programmes can you run from a USB stick. Well it appears that the list has just got longer. I’m probably the last person to realise it’s possible to do this, but I was really pleased to see that both GRASS and PostgreSQL can now be run from a USB stick, along with QGIS, XAMPP (inc Mapserver, OpenLayers, Tilecache), and FWTools.

Before I go any further, I should state that my aims for creating a portable GIS are not so much having a production GIS setup on a stick, because obviously performance and storage are an issue, but it might be useful for demonstration purposes, and I do feel as if the take up of these opensource tools might be more if they were easier to install and came in one handy package. So what I really want to achieve is a single portable package, probably a zip-file, that can be downloaded and unzipped onto a Window pc (they can learn about the joys of linux once we’ve won them over to the basic concept of opensource). No separate downloads, long installation and configuration processes, just a zip file. I do also like having portable packages on my main computer, as if it breaks it’s a lot easier to reconstruct the setup!

The light-bulb moment for me was when Leif told me that he had seen GRASS working on a USB stick using Msys at the recent CAA conference in Berlin. This intrigued me, so I looked into it and it works quite nicely- as long as you use the same drive letter to mount your USB stick to all the time. I still have to figure out how to figure out the drive letter and change the paths, probably in a batch file. Put simply, you download Mingw and Msys onto the stick, follow the instructions for downloading GRASS, edit the install_grass.bat file to change the drive letters, and off it goes.

PostgreSQL was something I had looked into a while ago, been told it wasn’t possible, and given up on. However, the recent 8.2 release appears to have dropped the requirement for a dedicated non-administrative user to run it. If you download the windows binaries, without the installer, you can set it up whilst logged on as the standard user. When you next run PostgreSQL from a different machine, you just have to use the same username as on your installation pc, regardless of whether that user exists on the pc that you are currently using. Once you have PostgreSQL installed, you can install Postgis into the same directory.

Currently there are a couple of gotchas. Most of these programmes require you to set environment variables, which might not be allowed on the particular pc that you are using, and require some command line experience. I still haven’t quite figured out how to get GRASS to work if I change the drive letter. Also, PostgreSQL does indeed run very slowly on a USB stick. And, I haven’t fully explored this to make sure there isn’t a hidden trap somewhere, or security holes that might be exploited. Having said that, if you want a package that can be unzipped onto a pc with a minimum of fuss then I think it’s almost there.

What’s next? Documentation, as always, and the permission of the various programmers that it’s OK to do this. Oh, and comments please, on what else we might include, or words of wisdom if you’ve already tried it.