The UK local chapter of OSGeo are bidding to host FOSS4G in the UK in 2013! For those of you that don’t know, the location of the main FOSS4G “meeting of the tribes” conference moves around from continent to continent each year, and 2013 is Europe’s turn. We want to dove-tail the event with the UK’s main annual geospatial conference, AGI Geocommunity, giving developers a great chance to showcase their work to a new audience, and for UK GI people to see all the fantastic work the open source community has been doing.

Archaeogeek is now migrated to Octopress! There are a couple of things still to sort out, such as some old broken links, a non-default theme and comments on older blog posts. Hopefully the pharma hack that has been polluting my google search results for a while now will disappear as the site is reindexed…

There are a few stuctural changes. I’m no longer asking people to register for downloading Portable GIS for a start. I had planned to use a mailing list to notify people of updates but never got much further than creating a table of names. It was useful for counting how many people download the file, but I can manage without knowing precise numbers. I’m no longer offering a contact form- there are multiple ways in which people can get in touch if they need to, and the most common thing people used it for was spamming me with SEO requests anyhow!

It has been a fun process- and thanks to James Fee for giving me the impetus to go ahead!

The rest of this post comprises some notes on actually doing the upgrade, not-so-subtly giving me a chance to try out some markdown syntax!

I’ve talked previously about wanting to move Archaeogeek from Wordpress- well I’m about to start that process. I’m expecting some disruption as I feel my way with Octopress- blimey even getting 7 years of blog across intact is going to be fun! So, apologies in advance for feedburner-style reposts, broken links/comments/entire sites.

In the process of making this switch, I’m going to strip out a lot of the plugins and additional bloat from the site. After all, the primary point of it is a place where I can get my thoughts across, and have people comment on them. Everything else is extraneous really.

See you on the other side, hopefully with a non-infected sparkly new slimmed down Octo-site!

Over the last couple of weeks, a few people have asked me the same question, which is (to paraphrase) “what’s the difference between Portable GIS and OSGeo Live or USB GIS?”. You get asked something once, and that’s fine, but more than that and it’s worth a blog post!

The main difference between the two options is that OSGeo Live and USB GIS are bootable disks. You plug the USB stick (or DVD) into your computer, reboot, and you’re presented with a fully configured Linux environment, with all the software ready to use.  While you’re in that environment, you can work as normal, but when you reboot, you’re back to your own OS. These options are cross-platform- they will work with windows, linux, or mac- as long as your machine can accept booting from a Live DVD or USB Stick.

Portable GIS is different. It is designed to work within windows, and only from a USB stick. Download the excutable, install it onto a suitably quick USB stick, open the menu application, and you’re good to go. You have access to all your standard windows resources, and can save your data either to the stick (if you have space) or to any other drive on your pc. The limitation is that it will only work within windows.

I think there’s a place for both options. Live DVDs/USB Sticks are great for a training environment where you want everyone to have exactly the same setup, where it looks identical, and has all the same data and passwords. I use the OSGeo Live DVD all the time for training, and give copies away liberally at events. I haven’t tried USB GIS yet but knowing the guys at Faunalia it’s bound to be good.

Portable GIS, on the other hand,  is good when you want to work on the same software at home as you do at work, in the operating system that you’re familiar with, and you want to transfer your data between the two. I think it will also work in a lot of locked down environments (though I don’t guarantee anything, because sysadmins can be wily creatures).

So, where are we at with the new version? I’m getting together the various packages and trying to find my notes about the secret sauce that makes them all run portably. Expect it to be a bit stripped down- there was some repetition of components last time, and I’m more interested in something that will run on a smaller disk and take less time to download and install. I’ll keep you posted!

Earlier this week I did a couple of presentations for the Association for Geographic Information (AGI) Welsh Group, along with my colleague Matt and a couple of people from Dotted Eyes, another company doing a lot of work with Open Source here in the UK. I did an introductory presentation on open source and the OSGeo “stack”, and then one demonstrating the capabilities of Quantum GIS. You can see my slides here and here. Be warned, the QGIS talk was actually a live demo, so the slides are just emergency screenshots in case the live demo crashed and burned! (It didn’t crash and burn, which was nice!)

We had a short round-table set of introductions before kicking off the talks properly, and it was really interesting to see what people hoped to get out of the day. A lot of the people there were very keen to identify desktop GIS alternatives, whereas in the past the discussion has tended to be about the database and web components. People are now relatively familiar with using PostgreSQL/PostGIS, and Mapserver/Geoserver, and usually OpenLayers or something similar. This battle has been won, I think, whereas for desktops it’s a different matter.

It’s quite clear now that Quantum GIS is a perfectly acceptable dekstop GIS package. I don’t accept any of the FUD that still goes around about its capabilities or stability. The people at the AGI Cymru event were definitely impressed with what it can do these days. Some of them had tried it a few years ago when, let’s be honest, it wasn’t so good, and are now prepared to take another look, which is a big win. However, in the majority of cases there was concern about how you’d go about deploying a totally new GIS package in an organisation where users may have worked with their current package for decades or more. This, to me, is not an open source issue, though it’s often portrayed as one. This issue will be a problem every time a new piece of software is deployed, be it open or closed source, and indeed if a new version of an old favourite has a re-design (Microsoft Office, I’m thinking of you).  I’m not sure how we get around this really, other than by understanding the motivations and concerns that people have, and trying to work within those limitations. Softly, softly, catchy monkey…

So, what else did we talk about? Linked data, both consuming and producing. Strategies for deploying open source within an organisation. IBM’s advice on Open Source in 2006 and whether or not it still holds true (yes, mostly). A good day!

It has been a while since I posted, and while I’m probably the only person bothered by that, I thought I ought to put an update together!

The blog has been going for over 5 years now, and in that time, in the UK, the attitude towards open source software has changed completely. There are now a lot of people who “get” open source, and are quite vocal in combating the FUD which we still unfortunately see from time to time. I don’t see the point in posting if all I’m going to say is “I agree with that person over there” so I have refrained from posting unless I feel very strongly about something.

Then there’s wordpress. Quite frankly, I’m sick of it. I’m sick of fighting a losing battle against hackers, despite my best efforts. I find it hard to engage with my blog when I feel like I can’t really trust the software. So it’s time to move platforms- though finding the time to do that with minimum disruption is proving a little daunting!

Finally, and here’s the good news and the only really “geo” part of this post- I am about to kick off an upgrade of portable GIS. No, really! For a while I wondered if it was worth the bother as it has got so much easier to deploy all the packages on windows, with the awesome OSGeo4W, and the plethora of bootable USB options if all you want to do is try the packages out. However, portable GIS is still downloaded plenty of times a month, and I occasionally get plaintive emails asking for it to be updated. Not only that, but we’re organising some training courses on open source GIS and quite frankly portable GIS kicks *ss for that purpose. So, watch this space!

There has been a lot of fairly excitable posting recently about the continuing rise of OpenStreetMap, and how it’s now being used in place of Google Maps,  in particular since Google started charging for data. People have been talking about how ”authoritative” crowd-sourced spatial data can be, and to be honest, I’ve found that the discussions seem to have missed the point a little bit. For me at least. So- here’s a few of my personal thoughts about OpenStreetMap and why it will be a while before I will consider it authoritative at least.

Firstly, a caveat. I love OpenStreetMap data and the whole ecosystem of “stuff” that has evolved around it. I contribute occasionally, though not as often as I’d like to. I’m pleased that it has forced the “traditional” data providers to reconsider what they offer, and perhaps to raise their game. I think it’s one of the main reasons why the idea of “open data” is so well known that even my Gran knows about it.

However, in the process of trying out WalkingPapers (my favourite addition to the OpenStreetMap ecosphere), I printed out the area around my house- in the centre of Lancaster. Not London, I admit, but still in the centre of a city. My road wasn’t on there. Read on, before you give up in disgust, saying “pah, you should just correct any errors you find”. How did I know my road was missing? There wasn’t an unfinished stub, with a note saying “I went home for my tea and I’ll finish this section tomorrow”, it was as if my road did not exist. I only knew my road was missing because I live there, and I have no readily available tools to help me judge the accuracy of an area that I am not familiar with. So, the idea that the end-user should correct any errors they find doesn’t really hold because you’re not always going to know that there’s an error!

I am aware that the traditional data providers do not produce totally error-free data. However, I would expect Ordnance Survey (for example) to have a workflow for their surveyors that doesn’t allow them to simply give up when they get bored or have to go home for their tea, or perhaps not survey a street because they don’t go down it very often.  I can also go to the site and find published information on positional accuracy if I need it. I guess I’m equating “authority” with trust, or reliability, and the issue is less about what’s there, than what’s not there.

When OpenStreetMap is being used as a static base map, by which I mean a backdrop to the information your map is really about, this is less of an issue. But, if like me, you come from a discipline where maps are collections of data to be manipulated and analysed, then you need to be able to trust your data a little more, or at least have a quantitative understanding of what the error is likely to be.

I know that this is not an easy question to answer, and I know people are trying to figure out ways of answering it. I also know it’s not that important for a lot of people, but I would like to see a more nuanced debate- I think that would be better for OpenStreetMap in the end.

Caution- this post won’t make you a pgRouting guru, but it will allow you to get pgrouting up and running on Ubuntu 11.10 and have some data on a map in approx 20 minutes.

1. Follow the instructions here to install PostgreSQL, PostGIS and pgRouting. Note that the instructions are for PostgreSQL 8.3 but they work just as well for PostgreSQL 9.0, just go with whatever version you want and substitute the package name accordingly.

2. If you’re using PostgreSQL 9.0, you might hit an error when loading the driving distance functions, as seen here. There’s a fix, which  may or not have been committed by the time you try this- if not then you need to apply it by replacing two altered files and recompiling/making and installing pgrouting (repeat the “compile pgRouting” step in the link in step 1).

3. Go here and download a selection of osm binary (pbf) files.

4. Follow the instructions here to download and run osm2po. Note that the documentation is currently available only in german. However, the example given in the link will work to get you a working demo, and it’s quite easy to follow the examples. Note also that you need a java runtime environment to use this- so if you don’t have one then download the jre of your choice using your favourite software package manager.

If this whets your appetite and you want to learn more, then there’s an excellent workshop available on the pgRouting site, along with lots of useful howtos and tools.

This, of course, only scratches the surface of pgRouting, but it just goes to show that thanks to all of the great links I’ve listed above, it is possible to get a working demo going extremely quickly.

If you end up doing a lot of work on mapserver and tilecache behind corporate proxy servers, you’re likely to hit a couple of snags, when the proxy thinks you’re trying to do something evil, when really all you want to do is seed a tilecache or look at an external WMS server. Fortunately there are a couple of useful workarounds, which I record here for my own sanity as much as anything else!

Mapserver

Mapserver has some great documentation on how to use proxy authentication when requesting data from an external web server, but unfortunately if you’re still on Mapserver 5.6.x, and on windows, you are likely to find that some of the directives don’t work. When you test your mapserver layer in a browser, you get a blank screen, and the following error message in your logs:

HTTP request error. CURLOPT_PROXYAUTH not supported. Requires Curl 7.10.7 and up

Needless to say, the later versions of mapserver (5.6.6) are using Curl versions greater than 7.10.7 so we’ll ignore that part of the message and concentrate on the CURLOPT_PROXYAUTH part. A quick google will inform you that this must be explicitly compiled into mapserver, and as far as I can tell, none of pre-compiled windows binaries for 5.6.x have been compiled with this option.

Enter cntlm, which is designed to stand between your application and your corporate proxy server, providing proxy authentication on the fly. To use it with mapserver, install the latest binaries and follow the configuration hints on the home page. The basic process is to run cntlm from a command line to establish the type of authentication that your proxy server uses, then fill in the supplied cntlm.ini file with the appropriate values for your configuration. If you don’t know these, you can easily find them by opening firefox or similar and  looking at preferences->advanced->network->settings.  You’ll need the URL of the proxy, the port, and the type.

Then, assuming that cntlm is installed on the same server as mapserver, your map file needs only the following layer metadata parameters (in addition to anything else you have defined):

wms_proxy_host: localhost

wms_proxy_port: specified in cntlm.ini (default is 3128)

wms_proxy_type: http or socks5 etc

With luck, once you have cntlm configured correctly, your external WMS layer should display just fine!

Tilecache

If when seeding your tilecache you receive “bad gateway” errors, or inexplicably corrupted tiles, it’s possible that the tilecache_seed.py script is also being blocked by the proxy server on your network. Somewhat counter-intuitively (at least it seems that way at first), this can be easily circumvented by switching off the proxy settings in Internet Explorer on the machine you’re running the script on (and switching back on once the script completes successfully). Note that it has to be Internet Explorer- you’ll need another approach if that’s not installed.

The reason for this is that tilecache_seed.py uses the python module urllib2, which uses ProxyHandler to handle proxies (who would have guessed). This will attempt to autodetect proxy settings, and one of the options it checks is in Internet Explorer (I guess this equates to a registry setting somewhere but we don’t go digging in the registry without our vaccinations up to date and all our affairs in order, do we?). A longer term solution might be to extend the tilecache_seed.py script to explicitly set up a ProxyHandler with no proxies defined, as in this example.

September is clearly conference month in the geo world. FOSS4G in Denver, which I didn’t go to, was closely followed by the AGI GeoCommunity in Nottingham, which I did. I participated FOSS4G vicariously, following twitter and starring lots of posts in my google feed, and it’s only now that I’m having chance to catch up on them all and assimilate them. Seems like there was a lot of soul-searching going on, about the future of the organisation, the conference, and the geospatial industry in general. Seems also like there was a really good vibe to the event, perhaps related to a really good choice of venue, which kept people together for socialising, networking and whatever. Perhaps these two factors are related? Get a bunch of conference-goers together over some geo-beers, and no doubt a lot of discussion will take place.

One of the key questions that seems to have come up as a result of the success of this year’s event has been whether or not there should be an annual North American event alongside the global event. Personally, not being in North America, I can only see this from the international perspective, and to me it looks bad. Diluting the FOSS4G brand has to be a bad thing. How many of the big players will choose to come to the international event when they “know” everyone “important” is going to be at the North American event? However, from the perspective of raising cash for the foundation, perhaps it’s a good idea because it’s probably more commercially viable. I don’t know, but I kind of wish the idea hadn’t been brought up…

With my new directors hat on, I’m sure there’s going to be a lot more discussion about this, alongside the organisational changes necessitated by dissolving the Executive Director’s post. I’m not going to go into that at all, but I would just like to say that I think Tyler did a fantastic job for OSGeo, particularly on the international scene- here in the UK he “was” OSGeo and things won’t be the same without him.

Over to the AGI GeoCommunity event. Having been on the working group helping to organise the event this year, again I had a different perspective on things to normal. Being on the working group is quite easy in the run up to the event as the AGI team do most (nearly all) of the hard work. However, you’re expected to help with exciting things like bag packing, chairing streams, and generally being around and visible throughout the two days. No sleeping during not so interesting talks! Attendance was good in the end, after a rush of late registrations, the new venue (the East Midland Conference Centre at the University of Nottingham) was a big success, and in the end even the last minute substitution of two plenary speakers (for perfectly good reasons) didn’t upset things!

I chaired the session on Open Source and Open Data on day one, which included some really worthwhile papers. Standing out (a couple of weeks after the event) are Antony Tuffour’s paper on Open Source Software Stack and Standards- Integration and GI for Everyone- about introducing a FOSS web mapping system at the London Borough of Hackney, and Mark Iliffe’s talk on When Gov 2.0 doesn’t exist- Mapping Services in the Developing World. I’ve seen Mark’s talk a couple of times now and it’s always inspirational. All the papers are available on the AGI website, so go take a look.  The famous soapbox event in the evening was won by my new(ish) boss. His talk was in reaction to the great FUD debate from a few weeks ago, excellently shown up for the absurdity that it is at the time by Paul Ramsey, and again here. Catch the YouTube video while you can!

Other things of note include the relaunch of the Technical Special Interest Group again. It’s easy to be cynical about this, as even I have now been around the AGI long enough to have seen the previous attempt lose momentum and stall, but enthusiasm is everything, and there is a lot of interest in the group as a relatively neutral umbrella under which both proprietary and open source vendors can talk techy without the sales pitch. Look out for events towards the end of the year.

There was plenty of soul-searching at the AGI GeoComm as well as at FOSS4G. While the two organisations are quite different, the same basic questions come up around providing best value, what the organisation is actually for, and how best to retain sponsorship and income in difficult financial times. Let’s just hope everyone pulls through in the end…