Computing, GIS and Archaeology in the UK

Well, at last it’s OK for me to tell people that Oxford Archaeology now has a WFS server that is accessible from the outside world. The address is:

http://mapdata.thehumanjourney.net/cgi-bin/mapservwfs.cgi

It’s a standard MapServer setup, and at the moment contains static data about the sites we have worked on over the last thirty years. This is still a work in progress and there are a whole bunch of things I would like to improve (but at least it’s up and out there):

1: As I said, it’s static data. The aim is to get our main databases into PostgreSQL (I’ve talked about this before, and it’s not an easy process to convert messy, historic, access databases into PostgreSQL). In some circles there is a question as to whether we should actually use live data. There are sometimes issues with people looting archaeological excavations, and we don’t want to make that any easier…
2: You may have noticed that I said databaseS (plural rather than singular). Oxford Archaeology only took over the Lancaster Office a few years ago and we are still working towards merging our core systems (along with upgrading them all to sensible, robust OpenSource platforms where possible). The problem with having two databases is that in many cases the fields are not directly compatible, so to get the data out in the shortest possible time I simply included the elements that were common to both, and I will work towards getting more information out.

3: At the moment, there is no fancy front-end to this. I have two candidates in mind for frontends, and I’ve talked about them both a fair bit. They will serve two different purposes, in other words for internal and external use. Externally I’m working towards using OpenLayers, although this might mean that I have to convert or re-project all of our data that is in British National Grid format into WGS84 so I can use something like Google Maps as a backend. Not a problem, I just haven’t done it yet. OpenLayers will give me a fairly basic, but nice looking interface that works in a way that people are familiar with from Google Maps and other sites. It is easy to install and can be built into any web page, so it can be embedded in our corporate site and not look out of place. Internally I want to use MapGuide OpenSource, as it has advanced functionality and a fairly slick style built in (I could use Mapserver and build a front-end myself but this seems like the best approach). However, as my last few posts have explained, I am having quite a lot of trouble compiling this on our platform of choice, so we’ll see.

4: There is only one layer so far. We’ll work on this, but often have licensing issues with our data, so we’ll have to check that out first.

Anyhow, enjoy.

The decision to move towards opensource software is one that more and more organisations are making, and there are many reasons for doing so. We are moving along the route slowly, but surely. Most of our back-end and infrastructure software is now opensource, whilst we are still investigating alternatives to the mainly closed source desktop packages.

Today, I read a series of posts about making the difficult choice between different opensource solutions. It wasn’t clear on Paolo’s blog why his company were making the move, yet that is a key factor.Â� It boils down to balancingÂ� ethics, finance, and control, but the decision will be different for everyone. This is absolutely as it should be, if we value our freedom of choice! There does seem to be a lot of muddying the water at the moment though, confusing free, as in beer, and free, as in speech. Even the BBC are doing it. Last week in an piece on the usefulness of opensource software to businesses, they segued into a interview about how useful Google Maps was. Beer. Not Speech. It’s even more difficult to make the right choices when people are confusing the debate…

Bill Dollin’s question was about why people would choose to release something that was free, yet closed source in today’s environment. I think this is part of the same problem, with people confusing “free” with “open”. Because that distinction is not clear, companies like Google can jump on the bandwagon, and get a lot of good press without having to go to the trouble of exposing their code for all the world to see. I look forward to a day when the meanings of “free” are not confused like this.

The other element that got me thinking though was Paolo’s mention of ZigGIS, which he is working on at the moment. ZigGIS is a free and opensource product, but it relies heavily on closed source proprietary code from Microsoft and ESRI. This raises some philosophical questions but also some difficulties for Paolo, as he cannot delve into ArcObjects to find out why something is going wrong. Bridging products like this, PGarc,Â� FME ,et al are absolutely vital though, for several reasons. Firstly, people who have made the choice to stick with proprietary GIS for the time being can still use a free open source product for data storage rather than investing the large amount of money required for the alternatives. Secondly, it allows much greater flexibility of access, because such a large range of products can work with the data. This means people can adopt a single standardised data stack without worrying about the tools people will beÂ� using to access the data with. Thirdly, it allows people to tryÂ� out opensourceÂ� alternatives without needing to abandon their familiarÂ� proprietary desktop GIS.Â� Rather than expect people to dive into unfamiliar opensource products with no lifeline, they help make the process a little easier. Good luck with ZigGIS Paolo, and the move to opensource!

�

I have made a little more progress with evaluating the various free options for accessing PostgreSQL/Postgis database tables from ArcMap. I have to confess that some of the problem was down to my own lack of experience with Postgis!

The issue that I had with PGarc was that it would fail with an error if you had deleted tables from a database. It turns out that this is because deleting tables using the PostgreSQL “DROP TABLE” syntax does not remove it’s reference in the “Geometry_Columns” table. This must be removed as well- either separately or by deleting the whole shebang in one go using the Postgis DropGeometryTable function. Since PGarc uses “Geometry_Columns” to populate its list of available tables, it fails at this point. RTFM next time, archaeogeek!

There were a number of people in the google group with similar problems to me regarding ZigGis- namely that it would not display some layers. This turned out to be a problem with projections, or the SRID. If a table had an SRID of -1, in other words no projection set, then ZigGis could not display it. This bug has been resolved in the latest release.

My opinions of the two packages haven’t really changed. ZigGis has nice configuration files, and works directly with the Postgis table, whereas PGarc relies on DSNs, and creates a temporary shape file on your hard drive to work with. However, the clincher for me is that using PGarc you can select data and query it, whereas with ZigGis, I could not do that.

In the comments to my previous post, JeffÂ� kindly pointed out the trial version of Safe Software’s FME (Feature Manipulation Engine), which does include support for PostgreSQL databases. I haven’t had chance to try it out yet, and we are trying to work towards open-source solutions to our problems but I will evaluate it and post on my experiences.