Once in a while, Oxford Archaeology get called upon to do some really big archaeological projects, like road schemes and airport expansion, that cover huge areas, go on for years, and generate loads of data. We love these, because it’s not very often that you get to look at whole landscapes- how multiple prehistoric villages interact, for example, rather than tantalising snapshots where you have to play “join the dots”.

Not only do we generate loads of data (thousands of artefacts, records, environmental samples, photos etc), but we ask difficult questions of them, based on statistical analysis of finds distribution, travelling salesman algorithms, best-path analysis. I think you can call this “real GIS”, though often people are surprised to hear that archaeologists use GIS at all.

As part of our ongoing “open approach“, and to prove that we put our money where our mouth is, we are now trying to do all of this in open source software rather than using the “standard” proprietary packages.  I’ve blogged previously about how pleased I was with the integration between QGIS and PostgreSQL, and how easy it was to manage large amounts of data without regress to proprietary packages, well  our current large project needs 3D analysis and large amounts of imagery manipulation, and again we’re finding that the open source tools out there do the job splendidly. Furthermore, we have a choice of tools, so if one approach doesn’t quite work the way we expect or want, then we can choose another. Now that can’t be bad, can it?

So, in brief, we’re using Quantum GIS and GvSIG pretty much interchangeably for our desktop GIS. All the vector data is in PostgreSQL. We use the QGIS Grass plugin to get data into a sensible format for 3D display and analysis in Paraview and Visit. We’re mosaicing up aerial photos using GDAL tools, and using Geoserver to publish everything to people who just need read-only access, and a direct connection to PostgreSQL for those that need to edit. We’ve developed a workflow for creating high-quality cartographic output by exporting to Inkscape, and the next step is a project website with links to our database and a nice openlayers map. Simples!

I went to the Open Knowledge Foundation conference, OKCON in London a few weeks ago, and have been meaning do a review of it ever since. Whilst little of what I saw had a direct relevance to what I do, it was invigorating to be in a room with a whole bunch of people with imagination, who believe knowledge should be free to anyone, and who basically like to disrupt the status quo.

A few quotes:

[2009 was] the year open data went mainstream (Rufus Pollock)

The threat [to the record industry] is not piracy, but obscurity (Glynn Moody)

What would happen if every school had a reprap? (Ben O’Steen)

One paper that was quite relevant to me was “Dig the new breed” by Anthony Beck, about opening up archaeological data. There are related articles and mailing list posts here and here. I feel uneasy about some of the details (it’s not lethargy or ethics that dictates the data we do or don’t release, it’s money) but in general it’s a no-brainer. We’re doing our bit with our Eprints library, but we’re at an early stage with getting reports on there.

The only problem I see is figuring out who to lobby- I would add the developers and the county-level curators to the list as well as the actual archaeological units. I know of specific cases where developers would not provide the money for making the results of an excavation public access, and also where a regional Historic Environment Record would not allow “their” data to be shown on a web map.

Otherwise, I particularly enjoyed Glynn Moody’s paper on The Post-Analogue World- focussing on the “plight” of the record industry, struggling to cope with the transition from analogue to digital, and Ben O’Steen’s paper on Making the Physical from the Digital. Bookbinding, repraps, MP’s expenses and Cory Doctorow all in one talk. Can’t be bad!

Many of the talks are available to download here, and OKFN have working groups for both archaeology and geospatial data if you’re interested. I hope to have more involvement with both, and to investigate links between OKFN and OSGeo, now I have some time and mental space. More to come…

Making History, on BBC Radio 4, had an article yesterday on the plight of commercial archaeology in UK during the recession. It made for some sobering listening.

To understand what all the fuss is about, you need to get over the idea that archaeologists are all volunteers, either school kids or retired, or perhaps have a nice research job in a University. Some are, and some do, but there’s a whole bunch of others who are trying to make a living from archaeology, in the same way that they would with any other job.

So, the Institute for Archaeologists, the professional body for archaeologists in the UK, have been profiling the profession since 2002, looking at the number of practising archaeologists, the average wage, etc. In 2007, at the peak of the housing boom, there were 7000 practising archaeologists, of which approx 4000 were working in commercial archaeology, with the rest working in academia or part of the planning team in councils. Since then, nearly 700 jobs have been lost, equating to 10% of that all-time high, and it’s fair to say that there will have been more losses in the commercial sector than in academia or councils. On the radio show yesterday they were suggesting up to 1 in 5 commercial archaeologists but I can’t find any figures to back that up.

The reason for this downturn is that commercial archaeology is now tied so tightly into the planning regime in the UK, and is funded almost entirely by developers, so if there is no building work going on, then there is no archaeology being done…

Hopefully things will start picking up as the government try and inject stimulus money into infrastructure, but it’s going to be a difficult year for everyone.

…Actually no one who commented on last week’s post, though thanks for all the great suggestions!

In the end we went for ZooOS, suggested by Jeremy Ottevanger on the Antiquist Mailing List (though the capitalisation is all ours). We like it because it’s got mixed etymological roots, coming from both Greek (zoo =  greek for animals) and Roman (os = bone in latin) but also because it has that essential Open Source ring to it. It’s pronounced “zeus” by the way…

Now we have a name, we also have a launchpad site!

Thanks again for all the suggestions- we think we might borrow one or two of them for other projects, so watch this space!

We are about to start work on an open source database for recording animal bones on archaeological sites, but we can’t think of a name for it! So- crowd-sourcing and all that- I thought I’d open it out to people to come up with suggestions (clean and polite only or we’ll be terribly disappointed with you). I guess we might even be able to scrape together a prize for the one we choose, if you like second-hand conference schwag! You’ll certainly get credited if we choose you so your name will live on for posterity. How about that?

So- suggestions in the comments, or you can contact me using the form, whatever you like…

I’ve been meaning to post for a while on Oxford Archaeology‘s Open Archaeology Project, also known as our “Open Ethos”, then what do you know, Joseph posts about it and says it so well that I might just as well repeat his post verbatim. I won’t though- then you might go and read the other blog, and wander around on the internet for a while finding out interesting new things . Pop back here when you’re done, if you like!

In short, “Open Archaeology” comprises three strands: open standards, open access, and open source. We see this  as the only logical way of fulfilling our remit as a commercial archaeological organisation, and an educational charity. Our job is to record the cultural remains that are damaged or destroyed by development. Our remit is to make those records available in perpetuity, to anyone who wants to see them. At the end of the day, pretty objects in museums are of little use without the background information that gives them context and fires the imagination.

While the three strands are not the same thing (as they are often made out to be), open data is useless without open standards and open software. Open software is useless without open data and open standards. You get the idea.

I’ve recently been asked if our commitment to open source software was a financial decision, or an ethical one. It’s both, I suppose, but our Open Archaeology project should show you that it’s part of a bigger thing than simply saving money on licenses.

In that serindipitous way that rss readers work, two posts came to my attention over the last couple of days. The first was from Gavin, about problems that occurred when the South African Government failed to keep control of the source code on two GIS programmes that they had developed. When contracts end, or funding dries up, if you don’t have complete control over your programmes then you might as well start rebuilding them now.

Today my fellow trouble-maker and open source advocate Joseph reported back on a Freedom of Information request that he made to English Heritage, the leading heritage body here in the UK, regarding the long-term sustainability of their (proprietary) Intrasis site recording software. I quote:

“Intrasis is developed by the Swedish National Heritage Board
(Riksantikvarieämbetet), this organisation has a long-term commitment to
the continued use and development of this software package. Our present
contract is for four years, and the use of the package will be the subject of
a post-implementation review after two years.”

So, lucky we only need to keep the records for 6 years then. After all, it’s not like they are the only record remaining of the archaeology that has now been dug up. Oh, wait…

(For those of you not from the UK- you might not know about the super-fab WhatDoTheyKnow site. It’s for submitting Freedom of Information Requests to UK public authorities)

This is all extra serindipitous (how often do you get to use that twice in one blog post?) because I’m trying to write my talk for the AGI 2008 conference at the moment. I want to try and get across that there are just as many, if not more, “risks” in a proprietary software solution as in open source. I suspect that government agencies aren’t ever really going to be taken to task over these decisions and this potential waste of public money unless we can get open source software understood at the highest of levels. Well, it’s a challenge…

Two Mondays ago I came into work in the morning to find one of my windows servers no longer booted. The short version  is that all the data was fine, but the windows partition had got itself corrupted. I now have a linux server, and know more about samba and winbind than I ever thought I would need to. No big deal, you might say, but it has been a learning experience for me, and I’m very grateful to my colleagues for their patience whilst I dropped everything else and flailed around in the dark trying to learn the intricacies of samba config from scratch.

I do have one thing to say… if you’re using winbind, remember that it can take at least 20 minutes from startup to work properly. If I’d known that a couple of days ago I’d have posted this much sooner, if you get my drift!

Things that have crossed my path over the last few days that are worth a look:

• Duh everyone has heard about this one already- Radiohead go open source and geospatial. If I liked their music, they’d be my favourite geeky band for sure. I’m certainly going to use this as light-hearted example of fun GIS things in courses.
• Leif finally posts again with a very thought-provoking article on misrepresentation and whether archaeologists should provide support to the military in conflict situations. It’s quite long, but worth hanging on for, and as archaeologists we need to think about this kind of thing if we are to be taken seriously and have any chance of protecting the cultural heritage of areas under conflict.
• Leif (again) and Paul and Tom from Wessex Archaeology finally have their paper on computers in archaeology publically available. Go read!
• There’s also a couple of interesting white papers floating about. Activestate have one that debunks 10 myths about open source for businesses. I think it could be more positive about open source, and in particular the strength of the community and the lack of vendor lock-in, but (let’s be honest here) they are selling support. Finally, Microsoft have a beginners guide to map projection- it’s a nasty aspx link so apologies if it dies. It’s a shame that it’s not available as a pdf (not that you can’t save it as one, of course).

Via the seasite mailing list, this article about the difference between underwater archaeology, salvage, and treasure-hunting really got me thinking.

I started off as a diver, then a marine archaeologist, and often came into contact with the strange point of view that if you find something underwater, like something from a wreck, it’s OK to prise it off and take it home to display proudly on your wall, yet you’d hardly go and break the wing-mirror off someone’s car. A friend became an archaeologist for Odyssey, the controversial marine salvage firm, and was ostracised amongst the academic community for his decision, yet he just wanted to put food on his children’s table, and felt that it was better for Odyssey to have some archaeologists trying to do things right than none at all.

Clearly, the difference between archaeology and treasure hunting or salvage is not clear, and is often quite emotive. This latest article, though, makes me feel even more uneasy about this strange blurring of lines. From what I can tell, treasure-hunters felt it was their right to steal things, even when visited by the police and the coast guard.

I don’t think this happens all that often on land, though we have occasionally had to hire 24 hour security for our sites to stop metal-detectorists coming in and digging things up. And sometimes to stop feral children getting in and setting fire to things. But that’s a different story…

The only conclusion I can come to is that people think it’s OK to do this sort of thing underwater, or at sea, because they don’t think they will be seen or caught. So, people behave themselves only because they think they will be censured by their peers, not because of any inherent sense of right or wrong. How depressing…

I’ve spent the best part of the day musing (whilst working, obviously) about this article, on rivers as archaeological artefacts. It’s a really good article, on how we perceive rivers in archaeology, given their pesky tendency to change course, texture, size and so on.

It suggests that we tend to think of rivers as primarily natural features, part of the landscape and full of nice ducks and fishes. Or, in archaeological terms, we treat them as environmental features and subject them to a barrage of scientific techniques designed to test their sediments, so we see them as nothing more than receptacles for more interesting things that have fallen in, such as bugs, seeds, people, boats, etc. We don’t tend to take notice of them as cultural features in their own right, as we would a road, or even a field.

I’m not sure that I totally agree that rivers get ignored as cultural features, after all I’ve spent the best part of two years on projects where the river was a fairly central feature. However, I do think we could perhaps do more by them, give them the credit they deserve, after all in prehistoric times (and more recently in some areas) they were the primary transport network, and have always been social and political boundaries, supplies of food and drink, and the location of most of the industry.

With my database-head on, I think we need a method of describing them in a way that embraces their flexibility and dual nature. They are never just going to be one thing at a time, which makes it difficult to use standard methods of classification. To classify a standard archaeological feature, such as a mill, we would tend to say that it had several phases of use that follow on from each other, but at any one time it has one primary function. Obviously this isn’t going to work for rivers! Another approach would be to split a river up into sections, and define a different use for each section. These would need to overlap so that you could show multiple uses.

I’m not going to take this much further right now, but it has made me think a little more about rivers and how we look at them, and I will try and apply this next time I get one in a project…

So, here’s to rivers and all that sail on them/fall in them/use them/draw pictures of  them/write stories about them.