Computing, GIS and Archaeology in the UK

Just as we are settling down with the transition to open source, it would seem that a lot of other people are at least considering their options along the same lines, for whatever reason. This thread on the osgeo_discuss list, started off as a question about the value of open source to individual’s careers, but rapidly morphed into a discussion about replacing the ESRI packages. This seems to have lead to a general consensus about the areas where the open source packages do well and do badly.

The consensus seems to be that the open source packages are great for the behind-the-scenes and/or web-based work, assuming a relatively technically literate user, but that cartographic capabilities are sadly lacking at the moment. Furthermore, if you’re looking for one single replacement, then you’re out of luck- but you can assemble a toolkit that will get you all the capabilities you need, in a modular fashion. This does require a change in mindset- but it’s a lot more flexible and hopefully more powerful.

I hope that this recent upswell of interest and consensus about where the open source packages need improvement will help focus development in those areas. This does seem to be happening: this page is about plans for a cartographic library that can be shared amongst different packages. Sign up to show your support!

PS- if you’re in the UK you can still come along to our first UK OSGeo meetup at Stansted tomorrow (Thursday 1st) at 4.30. It’s free to attend, there will be some talks to start with (better finish writing mine), then we’ll sit down and try and figure out what we want a UK local chapter to do. There will be tea, coffee and biscuits too, so what are you waiting for!

It would appear that ESRI (UK) were as surprised as we were when they received notification of ESRI (US)’s decision to tighten up the license agreements, and they are negotiating on behalf of all of us educational charities/associate sites to get us a little more leeway. It has to be said that the email we received from them, which I based my blog post on, contained nothing about this whatsoever, and basically told us we had till the end of March to pack our metaphorical bags, but we are grateful to them for their efforts.

However, the point still stands. By using closed source software, we are all subject to the whims of software vendors, who can change the license agreement to stop people using their products whenever they feel like it.  So it has never happened to you? I’d have said the same thing before 4pm yesterday…

UPDATE: We’ve had some more feedback about this from ESRI UK- see my other post for details (though don’t get too excited because nothing *really* changes)

So, our move to open source gets a boost today, from an unexpected quarter. In what can only be described as a noble act of self-sacrifice, ESRI have told us that as an educational charity we are no longer allowed to have an educational discount for using their software and, not only that, our license codes will cease to work at the end of this month. So, we have 3 weeks, with the Easter holidays in the middle, to extricate ourselves and our ongoing projects from ArcGIS and into something else or find the many thousands of pounds to buy the full licenses for all of our staff.

I should explain something about the nature of “commercial” archaeological units in the UK, to those that don’t know much about them. We are usually not affiliated with universities, so we are not educational establishments as such, although we are an educational charity. We exist to fulfil a legal remit to study the archaeology of an area before it is developed. We also undertake educational projects on behalf of English Heritage, the government’s body in charge of cultural heritage. The whole reason we exist is to further the archaeological and historical understanding about the country we live in.

Previously, my biggest gripe has been that we can’t get academic discounts for data, unlike our colleagues in universities or local government. Despite the fact that our job is a legal requirement, we have to pay through the nose for the mapping that we need to do it. Getting hold of geological data, or anything that might allow us to look deeper into the area we’re studying is often out of the question. But at least we had the software to work with.

I can’t figure out what ESRI hope to achieve by this. It clearly affects many other educational charities apart from ourselves, across many different industries. Do they think that we all have secret pots of cash hidden under our desks and that we’re just going to throw up our hands and go “fair cop guv, we’ll pay the full cost now, here’s 7,000 pounds per user”? Do they seriously think that 3 weeks is enough time to get the money or rebuild all of our work in other packages?

Well ESRI, in case it’s not clear, we’re not going to buy your full versions. It’s unlikely that we will ever use your software again, and you’ve made it much easier for us to openly campaign for open source solutions throughout our own industry and other related sectors, and anywhere else where people are concerned about getting screwed over by software vendors.

So, we have to part company. It was nice knowing you for a while, but you’ve changed and I just don’t think we’re compatible any more. Now, where’s QGIS…

Firstly, if you’ve seen this post before, apologies. It got lost in a previous re-organisation of the blog and I thought it was worth re-posting…

You can get free contour data for the UK from the Scottish Mountaineering Club website. It’s based on public domain NASA data, cleaned up by the club and made freely available. It comes in garmin .img format, in several zip files, which roughly represent areas of the country. See their web page for an explanation of the areas.

Edit: Leszek has some great alternatives methods and options for data- check the comments.

1. To convert the img files to something (more) useful, the best programme I’ve found is gpsmapedit. You need to register it to get the full functionality. This costs 42 euros (umm, about 32 pounds at the moment) but allows you to use the programme on up to four computers.
2. In gpsmapedit, click “open” or “add” and load up all the img files you want to convert. Note that they will be merged into one big file so for the sake of your poor computer, choose only the areas that you need.
3. Go to file/export/mapinfo file, select level “0″, this includes the minor contours (down to 10m) as well as the major contours (100m)
4. When it has finished exporting, you can shut it down. If you’re using mapinfo, then skip to step 6.
5. In ArcGIS, crack open ArcCatalog, and load the “Arcview 8.x tools” toolbar. Move your mapinfo files to a location on your computer that has no spaces in it’s path name and a short path. No really, do. It will crash otherwise. Choose the “mif to shapefile” converter. Alternatively, you could do this with ogr2ogr. See the website for usage.
6. The resulting file (mapinfo or shape) is in wgs84 projection, with the height values in feet, so for use in a British National Grid environment you need to re-project it. The height values are in the attribute field “label”, so add another field for holding the converted height and use the tool of your choice to convert the values in the “label” column to metres.

Note that this data is good for broad-based terrain modelling only. If you need something more accurate you’re going to have to buy it…

If you have any alternatives to this method let me know in the comments!