Computing, GIS and Archaeology in the UK

The decision to move towards opensource software is one that more and more organisations are making, and there are many reasons for doing so. We are moving along the route slowly, but surely. Most of our back-end and infrastructure software is now opensource, whilst we are still investigating alternatives to the mainly closed source desktop packages.

Today, I read a series of posts about making the difficult choice between different opensource solutions. It wasn’t clear on Paolo’s blog why his company were making the move, yet that is a key factor.Â� It boils down to balancingÂ� ethics, finance, and control, but the decision will be different for everyone. This is absolutely as it should be, if we value our freedom of choice! There does seem to be a lot of muddying the water at the moment though, confusing free, as in beer, and free, as in speech. Even the BBC are doing it. Last week in an piece on the usefulness of opensource software to businesses, they segued into a interview about how useful Google Maps was. Beer. Not Speech. It’s even more difficult to make the right choices when people are confusing the debate…

Bill Dollin’s question was about why people would choose to release something that was free, yet closed source in today’s environment. I think this is part of the same problem, with people confusing “free” with “open”. Because that distinction is not clear, companies like Google can jump on the bandwagon, and get a lot of good press without having to go to the trouble of exposing their code for all the world to see. I look forward to a day when the meanings of “free” are not confused like this.

The other element that got me thinking though was Paolo’s mention of ZigGIS, which he is working on at the moment. ZigGIS is a free and opensource product, but it relies heavily on closed source proprietary code from Microsoft and ESRI. This raises some philosophical questions but also some difficulties for Paolo, as he cannot delve into ArcObjects to find out why something is going wrong. Bridging products like this, PGarc,Â� FME ,et al are absolutely vital though, for several reasons. Firstly, people who have made the choice to stick with proprietary GIS for the time being can still use a free open source product for data storage rather than investing the large amount of money required for the alternatives. Secondly, it allows much greater flexibility of access, because such a large range of products can work with the data. This means people can adopt a single standardised data stack without worrying about the tools people will beÂ� using to access the data with. Thirdly, it allows people to tryÂ� out opensourceÂ� alternatives without needing to abandon their familiarÂ� proprietary desktop GIS.Â� Rather than expect people to dive into unfamiliar opensource products with no lifeline, they help make the process a little easier. Good luck with ZigGIS Paolo, and the move to opensource!

�

I have made a little more progress with evaluating the various free options for accessing PostgreSQL/Postgis database tables from ArcMap. I have to confess that some of the problem was down to my own lack of experience with Postgis!

The issue that I had with PGarc was that it would fail with an error if you had deleted tables from a database. It turns out that this is because deleting tables using the PostgreSQL “DROP TABLE” syntax does not remove it’s reference in the “Geometry_Columns” table. This must be removed as well- either separately or by deleting the whole shebang in one go using the Postgis DropGeometryTable function. Since PGarc uses “Geometry_Columns” to populate its list of available tables, it fails at this point. RTFM next time, archaeogeek!

There were a number of people in the google group with similar problems to me regarding ZigGis- namely that it would not display some layers. This turned out to be a problem with projections, or the SRID. If a table had an SRID of -1, in other words no projection set, then ZigGis could not display it. This bug has been resolved in the latest release.

My opinions of the two packages haven’t really changed. ZigGis has nice configuration files, and works directly with the Postgis table, whereas PGarc relies on DSNs, and creates a temporary shape file on your hard drive to work with. However, the clincher for me is that using PGarc you can select data and query it, whereas with ZigGis, I could not do that.

In the comments to my previous post, JeffÂ� kindly pointed out the trial version of Safe Software’s FME (Feature Manipulation Engine), which does include support for PostgreSQL databases. I haven’t had chance to try it out yet, and we are trying to work towards open-source solutions to our problems but I will evaluate it and post on my experiences.

This post is a discussion of my experiences with a couple of ArcGIS connectors for Postgis. To me, a reliable Postgis connector would be so useful, as we simply don’t have the resources (or the inclination) to purchase the equivalent proprietary products. We are also looking for a solution that will provide a flexible backend for all sorts of interfaces, such as MapGuide Open Source, UDig, and so on, alongside ArcGIS.

The two connectors for ArcGIS that I have tried so far are Ziggis and PGarc. Neither of these are what I would call production-ready yet, and have some problems but they are being actively developed so there is hope!

Ziggis:

I spent a rather frustrating day yesterday trying to install the new release of Ziggis and thought it was worth blogging the steps I had to take to get it installed as I found information on this to be confusing and sparse. Digging around in the ESRI discussion forums and such like seems to suggest that my problems will be relevant to the subset of people who don’t have things like Visual Studio installed, and for all I know will not be relevant in ArcGIS 9.1 or later. The short version is as follows:

1. Ziggis requires the Microsoft .net Framework and Arcobjects .net Assemblies to work,or the install will fail.
2. To get the Arcobjects .net Assemblies, you must have Microsoft .net version 1.1 installed before you install ArcGIS and you must choose the custom install of ArcGIS Desktop (should be disk one).
3. .net version 2.0 won’t work for ArcGIS 9.0 (don’t know about later versions).
4. If you already have ArcGIS installed before you install .net then you will need to do a full uninstallation of ArcGIS, reboot, install .net, and then install ArcGIS. Trying to cut corners and running the repair/reinstall/modify option won’t work.

Once I had it installed, I had an extra button in ArcMap for adding layers from a postgis table. Ziggis using configuration files (.zig files) for providing the information about connecting to the database. There is an example file in the program folder that can be modified using a standard text editor. You add new layers by navigating to the appropriate zig file for the database you wish to connect to, then type the table name that you want to add. This is a slightly frustrating step because you have to know which table you want, ie by firing up some other Postgis/PostgreSQL administrator first. A method of browsing for available tables would be a bonus.

Unfortunately, so far I haven’t managed to get anything to display on the map! The layer appears in the table of contents, and I can open the attribute table, but the geometry column contains the value “no geometry” despite being able to display using a different connector (PGarc). The next step is to try and enable the debugging and logging options in Ziggis to establish the problem.

PGarc:

PGarc appears to be in a state of flux at the moment. Some development is ongoing as there are discussions on the postgis mailing list about it, but the website version has not been updated in some time. A few months ago I was given some updated code in the form of an ArcGIS project, and this version at least works for me- with an important proviso (more later).

PGarc installs itself as an extra toolbar with two buttons, one to add new data from a database, and one to save data back into a database. To connect to a database you need to create a system dsn with the connection details and provide the name of that file, along with your username and password to PGarc. PGarc does list the available tables, from which you can pick one and add it to your map.

Saving data back into a database seems to work, although it only saves the visible extent (so you have to zoom out to the extent of a layer before saving it).

My biggest problem with it is that if I have deleted a table in my PostgreSQL database, then PGarc cannot connect to it because it is still looking for the deleted table. I don’t yet know if this is related to the dsn, PGArc, or PostgreSQL.

Comparision:

Both projects are at an early stage and as you can see I have had problems with both of them. PGarc seems to have the best feature set at the moment (you can select features, which is not possible, afaik, with Ziggis) and save data back into the database. However, the idea of text-based configuration files is more convenient than a system dsn, as they can be easily transferred between computers. I need to investigate the geometry issue with Ziggis more thoroughly, and will keep following the discussions, and then I should be in a better position to comment.

There is at least one other connector available, but I didn’t get further than downloading the instructions! Furthermore, ESRI have intimated that Postgis connectivity will be available in a later release of ArcMap. This should be a good thing, but there are concerns as to how it will be implemented. Watch this space…