Computing, GIS and Archaeology in the UK

I guess a lot of people will have seen the article on slashdot pointing to an article on a Stanford/Harvard paper on how businesses can win against open source software/technology. I don’t want to get into a debate about how the authors are in fact the spawn of the devil, as you can read the slashdot comments for that. Personally, I started off being slightly disappointed by a number of points that they made, and then quite up-beat about the prognosis for open source as a result.

Why disappointed? Firstly, the article (which is of course a summary of a paper and not the actual paper, so possibly a mis-interpretation) confuses free software with open source, again. Lazy. It assumes that the only reason people will choose an open source product is because it is free, and not because it gives the users better control and freedom from licenses etc. Secondly, they argue that particular types of software are popular because so many people use it (circular argument?, even more lazy?). Apparently this leads to less issues with training, and incompatibility. If this was the case, would Microsoft have had so many issues when they changed the interface for Vista, or Office 2007, and the default file format for Office? Ditto Autocad with their file format in every new release? Consumers are more intelligent than that, and these things do have a big impact on how easy it is to upgrade software.

Why up-beat? Well the point has been made elsewhere that this paper only exists because open source software is seen as a threat. Not only that, all of the strategies outlined in the paper are reactive- how can businesses deal with this threat by using aggressive practices. Even making better, more compatible software is seen as a reaction to this threat, rather than the modus operandi for the company. It’s disappointing that improvement is only seen as a reaction to a threat, but on the other hand if companies are forced to improve their product, and make it more interoperable, everyone wins.

All in all I think it’s disappointing that consumers are given such short shrift in this article, and as always we have this lazy assumption that the open source argument is all about price. It does, however, show that open source software can, and does, have the power to affect the proprietary market, and I find that very encouraging indeed.

<rant>

Like others, I recently received a comment about an upcoming competition on  my “about” page. Like others, I feel that this is a curious way of advertising a competition with large monetary prize. Possibly unlike others, I also feel quite strongly about being contacted in this way. Don’t get me wrong, I think it’s a very worthy competition, and had the advertising policy been better thought through I would be all for it.

I do feel quite strongly about only posting my own personal views on my site. I do occasionally get asked to post on particular subjects/sites/products, and in general I won’t, unless it fits with something I was going to say anyway. I also feel quite strongly about posting on subjects that have been covered ad nauseum by other people, unless I have something different to add.

All in all, that makes me quite tetchy when I receive comments like this. By not using the contact form, it means I have to decide whether to get quite stalinist with my comment deletion policy, or to give people some free advertising. Furthermore, by posting to multiple blogs, our collective readership get stung with the same old post multiple times.

I guess this is just all just a request for people to learn a little blog-etiquette- make the effort to contact people in the correct way, and don’t use blog comments to advertise!

</rant>

Two Mondays ago I came into work in the morning to find one of my windows servers no longer booted. The short version  is that all the data was fine, but the windows partition had got itself corrupted. I now have a linux server, and know more about samba and winbind than I ever thought I would need to. No big deal, you might say, but it has been a learning experience for me, and I’m very grateful to my colleagues for their patience whilst I dropped everything else and flailed around in the dark trying to learn the intricacies of samba config from scratch.

I do have one thing to say… if you’re using winbind, remember that it can take at least 20 minutes from startup to work properly. If I’d known that a couple of days ago I’d have posted this much sooner, if you get my drift!

Things that have crossed my path over the last few days that are worth a look:

• Duh everyone has heard about this one already- Radiohead go open source and geospatial. If I liked their music, they’d be my favourite geeky band for sure. I’m certainly going to use this as light-hearted example of fun GIS things in courses.
• Leif finally posts again with a very thought-provoking article on misrepresentation and whether archaeologists should provide support to the military in conflict situations. It’s quite long, but worth hanging on for, and as archaeologists we need to think about this kind of thing if we are to be taken seriously and have any chance of protecting the cultural heritage of areas under conflict.
• Leif (again) and Paul and Tom from Wessex Archaeology finally have their paper on computers in archaeology publically available. Go read!
• There’s also a couple of interesting white papers floating about. Activestate have one that debunks 10 myths about open source for businesses. I think it could be more positive about open source, and in particular the strength of the community and the lack of vendor lock-in, but (let’s be honest here) they are selling support. Finally, Microsoft have a beginners guide to map projection- it’s a nasty aspx link so apologies if it dies. It’s a shame that it’s not available as a pdf (not that you can’t save it as one, of course).

Moving to open source for our GIS packages continues to throw up some interesting challenges- and solutions. In the UK, the most common way for Ordnance Survey vector data to be delivered is in mastermap format, which without going into the technical details is gml, usually served in gz files. Lots of them. There is an additonal add-on for ArcGIS that converts this into a personal geodatabase, but initially there didn’t seem to be a way of doing this using open source.

Along comes the mastermap importer. This does exactly what it says on the box- imports mastermap data into ogr formats such as shapefiles, or into a postgresql database. It’s free for personal or non-commercial use and is a really great programme. I haven’t begun to explore any of it’s more esoteric options, but for simply converting 78 gzipped mastermap files into shapefiles it can’t be beat, even by the commercial importer add-on for ArcGIS.

The only problem I have is that 1 gzipped mastermap file translates to 1 folder containing 8 shapefiles, one for each component of the mastermap data (annotation, topographic area, point, line, etc). So if you have 78 gzipped files then you have 78*8 shapefiles to deal with. I would like to see some way of merging the matching shapefiles together automatically if possible. The only option I have so far tried is geomerge, which works well, but you still have to go through the folders picking the shapefiles you want in the first place. Surely there must be some way of iterating through the folders finding all the shapefiles with the same initial part of their name ( area shapefiles all start with MM_TopographicArea_, for example) and merging them together?

Note that I haven’t tried the postgresql options yet. If they allow you to append all the data of each type into one table, then we’re sorted as I can import them back out into shapefiles if I need to.

The next thing that I have to figure out is how to apply standard Ordnance Survey symbology to the result. That’s a tricky one…