I went to the Open Knowledge Foundation conference, OKCON in London a few weeks ago, and have been meaning do a review of it ever since. Whilst little of what I saw had a direct relevance to what I do, it was invigorating to be in a room with a whole bunch of people with imagination, who believe knowledge should be free to anyone, and who basically like to disrupt the status quo.

A few quotes:

[2009 was] the year open data went mainstream (Rufus Pollock)

The threat [to the record industry] is not piracy, but obscurity (Glynn Moody)

What would happen if every school had a reprap? (Ben O’Steen)

One paper that was quite relevant to me was “Dig the new breed” by Anthony Beck, about opening up archaeological data. There are related articles and mailing list posts here and here. I feel uneasy about some of the details (it’s not lethargy or ethics that dictates the data we do or don’t release, it’s money) but in general it’s a no-brainer. We’re doing our bit with our Eprints library, but we’re at an early stage with getting reports on there.

The only problem I see is figuring out who to lobby- I would add the developers and the county-level curators to the list as well as the actual archaeological units. I know of specific cases where developers would not provide the money for making the results of an excavation public access, and also where a regional Historic Environment Record would not allow “their” data to be shown on a web map.

Otherwise, I particularly enjoyed Glynn Moody’s paper on The Post-Analogue World- focussing on the “plight” of the record industry, struggling to cope with the transition from analogue to digital, and Ben O’Steen’s paper on Making the Physical from the Digital. Bookbinding, repraps, MP’s expenses and Cory Doctorow all in one talk. Can’t be bad!

Many of the talks are available to download here, and OKFN have working groups for both archaeology and geospatial data if you’re interested. I hope to have more involvement with both, and to investigate links between OKFN and OSGeo, now I have some time and mental space. More to come…

So, we Brits are getting all excited because the Prime Minister gave a big speech about how Britain is going to be all fab,  broadbandy, and “totally, like Web 2.0, man”  soon. And all this data is going to be given away- look here’s some of it now. We’re all going to have instant access to Government services, free Ordnance Survey data, and a shiny Institute of Web Science to stick all the bits together.

Now, of course it’s a total coincidence that this is a nice positive announcement, and there’s to be an election shortly, for which we don’t yet know the date. Let’s not go there, since this is not a political blog. Let’s also not contrast this with the Digital Economy Bill, fast-tracking it’s way through parliament on somewhat dodgy grounds as we speak, that threatens to take a lot of this nice shiny stuff away if you’re so much as caught within a mile of an mp3 that you didn’t buy from ITunes. Let’s not talk about carrots and sticks, or nice cops and nasty cops or other such comparisons.

Instead, let’s concentrate on the data.  They will, on April 1st, release “a substantial package of information held by ordnance survey freely available to the public, without restrictions on re-use”. The consultation only finished 5 days ago! How can they possibly have had time to analyse the consultation responses yet? Personally, while I look forward to the release of the data, and it is a massive win for the Free Our Data campaign, I would prefer that they gave the consultation responses some respect by analysing them in detail, or at least pretended to read them properly.  I told someone off for assuming that the decision had already been made, and that the consultation was a waste of time, but perhaps they were right.

The rest of it, I quite like, though I’m yet to see a Government IT project really work properly. We’ve got (amongst other things):

• “core reference datasets that contain the precise names and co-ordinates of all 350 thousand bus stops, railway stations and airports in Britain”
• “an inventory of all non-personal datasets held by departments and arms-length bodies – a “domesday book” for the 21st century”
• “information on each set of data including its size, source, format, content, timeliness, cost and quality”

All sounds good. I could get excited by this, but I just want to wait and see what happens, shall we, and not take our eye off the other balls?

A nice short guide on how to start making your data open from Open Data Commons. This shows how simple it really is.

Not wanting to miss out on the whole discussion about data formats, I was surprised to see people give up their control of their data quite so easily, as this comment and following post seem to suggest that we should. Imagine if we ceded so much control to the other people that sell us products. Software companies are only glorified shopkeepers, in the same way that people who sell us televisions and cars are. However (if we have any sense) we don’t allow car salesmen to dictate where we drive, or television salesmen to dictate what we watch. We have allowed software vendors to lull us into believing that they are allowed to dictate what we do with our data. Sorry, but no. That’s what the open approach is all about- remaining in control of our data. At the risk of seeming overly dramatic, anything else is apathy.

I’ve been meaning to post for a while on Oxford Archaeology‘s Open Archaeology Project, also known as our “Open Ethos”, then what do you know, Joseph posts about it and says it so well that I might just as well repeat his post verbatim. I won’t though- then you might go and read the other blog, and wander around on the internet for a while finding out interesting new things . Pop back here when you’re done, if you like!

In short, “Open Archaeology” comprises three strands: open standards, open access, and open source. We see this  as the only logical way of fulfilling our remit as a commercial archaeological organisation, and an educational charity. Our job is to record the cultural remains that are damaged or destroyed by development. Our remit is to make those records available in perpetuity, to anyone who wants to see them. At the end of the day, pretty objects in museums are of little use without the background information that gives them context and fires the imagination.

While the three strands are not the same thing (as they are often made out to be), open data is useless without open standards and open software. Open software is useless without open data and open standards. You get the idea.

I’ve recently been asked if our commitment to open source software was a financial decision, or an ethical one. It’s both, I suppose, but our Open Archaeology project should show you that it’s part of a bigger thing than simply saving money on licenses.

Wessex Archaeology are releasing their impressive gallery and flickr collection of photos under a creative commons license, according to Tom at Past Thinking. This is really good news, and shows that the idea of open access is catching on in British Archaeology.
I’m going to submit an abstract for FOSS4G07 on the idea of “openness”, be that in terms of data or software, in UK Archaeology, and would welcome any comments or examples of use from readers.

Last night I was at the Free Our Data? discussion at the University of Manchester, running as part of the ESRC Festival of Social Science 2007. This was interesting, not least because I have been thinking about this debate purely in terms of geographical data, yet other types of data bring other issues and concerns. The question-mark is important, as it represents the crux of this evening’s debate. Should public sector data be available for free, or freely available?
The session was recorded and webcast by the University, so I won’t try and summarise the entire debate, but there were many thought-provoking issues raised by the panelists.

Charles Arthur, Technology Editor for the Guardian fervently believes that digital data should be freely available at the point of use as the costs of dissemination, reproduction etc are virtually nil. Some costs should be borne by increased taxes, but that the benefits to the economy created by free access to the data would outweigh that. Not sure I agree with the first point, that data dissemination doesn’t cost anything, but I can certainly see at our small-fry level we could charge less for the work we do, and do a better job, if we weren’t incurring an overhead for the cost of licensing data.

Jill Matheson, Director of Census, Demographic and Regional Statistics at the Office of National Statistics had three basic and eminently sensible points: That the value of data is in it’s use; that protecting confidentiality is paramount; and that there’s no such thing as free data, only hard decisions as to who pays for it. Jill then went on to say that the more people use data, the better the Quality Assurance Process.So- if we want more people to use the data, then let’s make it free. However, I’m not suggesting a “Statistipedia” approach, as that kind of editorial model would not be appropriate! Jill argued against Charles’ assertion that data can be disseminated at no cost, saying that making data accessible is what costs, rather than making it available. On the subject of confidentiality- I initially wondered if that was a bit of obfuscation, as once statistical data is anonymised confidentiality is obviously not an issue. I then wondered, however, how I would feel if my street was classed badly on the basis of some demographical analysis. That would be personal to me, but at what stage or scale does the data become safe- at the level of a zone, a town or city, a county?

Duncan MacNiven, the Registrar General for Scotland highlighted the way that they are making a lot of information freely available North of the border. However, I was intrigued by his argument for charging for some types of data but not others. He argued that demographic and census data should be (and is) free at the point of use, subsidised by the state, but that genealogical data should not be. Why should Scottish tax payers subsidise people in Australia looking for information on their family history?

I find this difficult to agree with. As a tax payer in England I subsidise a lot of things that I am not interested in, such as healthcare for smokers. As a tax payer in the North of England, I am subsidising the cost of hosting the Olympics in 2012 in London. As an archaeologist, I have seen jobs cancelled since “we” won the Olympic bid, because developments have been curtailed or stopped and the money channeled into the Olympic development. Furthermore, once the data is out there in the digital realm, it’s available to anyone, no matter what country they are in.

Duncan Shiel, Head of Strategy at Ordnance Survey, was always going to be on the least popular side in this debate, but he did make a good point that having good quality data is more important than free data. I agree with this completely and sometimes worry that too much focus on the cost takes away from sensible debate, and leads to poor quality solutions or ways around the problem. Duncan’s next point though, was something I would have liked to challenge, had there been time. He said that the private sector should add value to public sector data. Lovely. So the private sector should pay to use the data, but should then give something enhanced back to the public sector for free? I don’t think so!

Finally Peter Elias, Professor of Labour Economy at the University of Warwick and a Strategic Advisor to the ESRC said that in an ideal world data for publicly funded work for the public good, at a publicly funded institution such as a University, producing results that are going to be publicly available should be able to use data freely. That seems sensible, but not always economically viable.

I went away thinking that commercial archaeological units, in the UK at least, are in a difficult position. We are not academic, in the sense of being part of an educational institution, although some of us are educational charities. However, the work that we do comes from a piece of planning law- PPG16, which requires that the majority of development in this country has some level of archaeological assessment undertaken. We have a duty to preserve the archaeology, preferably in situ but by record if necessary, and to retain it’s context for future generations. Good units will create an academically rigorous report on their findings, often subject to peer review in a reputable journal. And yet, we have to pay through the nose to use public data, unless we happen to work for English Heritage, or can persuade the developers that they should lend us their data for the duration of the project.

End thoughts- it’s a very complicated and rich debate, and perhaps geographical data is the easiest to resolve as it doesn’t have confidentiality issues. Both sides are quite entrenched, but at least the discussion is happening.