Computing, GIS and Archaeology in the UK

Does anyone else see the irony in these two statements, which fortuitously popped up in my feed reader one after the other?

It’s so depressing when people talk about phasing out paper maps. It seems to be part of a general feeling that all maps are for is getting from point A to point B on the road. Does your sat-nav point out interesting things you maybe ought to detour for? (I’ll probably get a wry smile from Mr Archaeogeek at this point, who knows I love to buy paper maps of new areas and sit down all evening just looking at all the sites on them. Particularly archaeological sites. He’s very long-suffering). I guess sat-nav just doesn’t have that same sense of history.

A couple of people have remarked to me that one reason that they are not prepared to make the open source jump is the lack of support.

Well, last week I wrote about a mastermap importer that I had found. I said I would like to be able to merge the huge numbers of shapefiles produced into more manageable chunks. I got a lot of good advice as to ways I might be able to do that using gdal, which is really handy, BUT I also emailed the developer about the same problem. Two days later he released a new version of the software with that functionality built in.

To me, this highlights one of the big advantages of open source. People want to make their software better, and are willing to listen to suggestions. Small firms or one-man bands don’t have the inertia of the major firms and can respond to requests quickly and easily. To the user, that means flexibility and control. In other industries, that would be seen as an advantage, not a disadvantage, so why not in software?

Well, I guess I shouldn’t really put it off any further…

The very first release of Portable GIS is now available for download here. Phew! While you don’t have to register to download, it would be nice if you did, so I know who is interested and can contact about updates. No spam, I promise.

For people that don’t read the readme: this is not cross-platform, it’s for windows only. Unzip it and move the folders inside to the root of a USB stick, then double-click on the menu exe to access the programmes.

Edit: don’t forget to run the setup programmes for those that need it to change to the correct drive letter.

Note the big disclaimer: Be nice- it’s working as far as I can tell, but there might be issues and I’ll try and address them as soon as possible but this isn’t my full-time job.

Note also that very soon I’ll be moving the hosting and maintenance of this to somewhere else, which is why there are different email addresses in the package. Use those, or contact me here, it’s fine…

Moving to open source for our GIS packages continues to throw up some interesting challenges- and solutions. In the UK, the most common way for Ordnance Survey vector data to be delivered is in mastermap format, which without going into the technical details is gml, usually served in gz files. Lots of them. There is an additonal add-on for ArcGIS that converts this into a personal geodatabase, but initially there didn’t seem to be a way of doing this using open source.

Along comes the mastermap importer. This does exactly what it says on the box- imports mastermap data into ogr formats such as shapefiles, or into a postgresql database. It’s free for personal or non-commercial use and is a really great programme. I haven’t begun to explore any of it’s more esoteric options, but for simply converting 78 gzipped mastermap files into shapefiles it can’t be beat, even by the commercial importer add-on for ArcGIS.

The only problem I have is that 1 gzipped mastermap file translates to 1 folder containing 8 shapefiles, one for each component of the mastermap data (annotation, topographic area, point, line, etc). So if you have 78 gzipped files then you have 78*8 shapefiles to deal with. I would like to see some way of merging the matching shapefiles together automatically if possible. The only option I have so far tried is geomerge, which works well, but you still have to go through the folders picking the shapefiles you want in the first place. Surely there must be some way of iterating through the folders finding all the shapefiles with the same initial part of their name ( area shapefiles all start with MM_TopographicArea_, for example) and merging them together?

Note that I haven’t tried the postgresql options yet. If they allow you to append all the data of each type into one table, then we’re sorted as I can import them back out into shapefiles if I need to.

The next thing that I have to figure out is how to apply standard Ordnance Survey symbology to the result. That’s a tricky one…