So, it’s 5 years since OSGeo was formed- that’s pretty cool! Spurred on by this post, I thought I would say a little bit about my involvement with OSGeo, and also rather clumsily segue this into an announcement about my impending change of job. It’s true- after years of not really thinking of myself as an archaeologist any longer, but rather ‘someone who works in an archaeological unit”, I’ve finally gone and got myself a real job. Anyhow, more on that later.

5 years ago, I was playing around with mapserver, because I wanted a way of serving data and some shiny web maps and we had no money to do it with. I sucked at it big time, with no programming experience, and barely any command line experience. My only linux knowledge consisted of messing around with a Sharp Zaurus (the best pda I’ve ever had), and I only really equated open source with “free as in beer”. I remember the excitement on the mapserver mailing list when OSGeo was announced, along with the open sourcing of MapGuide, and I also remember quite fancying the idea of a whole conference about this cool new stuff when FOSS4G was announced.

Since then, there have been so many lightbulb moments for me, such as getting the real “point” of open source, getting more involved with OSGeo via starting the UK chapter, breakthrough moments when I have really started to understand a particular program or language, getting over my fear of public speaking by talking about open source at conferences, teaching colleagues and strangers how to use this software, and getting consultancy jobs designing applications that use it.

Through that time, I’ve seen (heck, we’ve all seen) OSGeo go from strength to strength, and become a truly global organisation. More importantly though, it’s a great global community.

I should also say that my new job, as Web GIS Specialist/Lead Consultant at Astun Technology, could only have happened through the experience I’ve gained over the last five years, so thanks to everyone that’s helped with that. I’m sad to be leaving Oxford Archaeology, but massively looking forward to my new role, which I hope will allow me to keep up my involvement with OSGeo, and in particular the UK chapter.

Will I keep blogging? Yes, and hopefully more often as I learn lots of new and exciting stuff (new to me- I’m a recent very enthusiastic convert to Python-years after everyone else). Will I still be doing open source stuff? Hell, yes! Will I change the name of the blog? Pehaps…

Never apologise for delayed posts… this is a Sunday Tip Day post, not a Thursday! Anyhow…

I just found a super little cross-platform utility that takes shapefiles and dumps them to a variety of text-based formats. Download it here, and simply unzip it to use it.

There isn’t much documentation, but basically your options are to download to gpx or spreadsheet. The following gives you a simple delimited text file with the coordinates and values from your attribute table:

./shp2text -–spreadsheet /path/to/your/shapefile.shp > /path/to/output.txt

Bob is indeed your mother’s brother.

Apologies for the delay- it’s been a roller-coaster week at Archaeogeek Towers due to family health issues. Hopefully it’s getting sorted now though.

In the process of preparing Portable GIS, I needed to change a string in multiple files of multiple formats within multiple folders. I looked at various windows- based options, most of which had a charge associated with them, but one of my colleagues suggested a linux approach.

“Sed” is the command that you need, found in most linux systems. In windows you can install it as part of the unxutils toolkit. The following command in linux works recursively through the folders in a root directory changing string1 for string2 (be careful with those single quotes if copy and pasting as wordpress likes to restyle them):

find ./ -type f -exec sed -i ’s/string1/string2/’ {} \;

With a large number of files this might take some time to work (a usb stick took an hour to work through on my ubuntu server vm but maybe that’s not a fair benchmark).

How to mount a USB stick in Ubuntu server using the command prompt only:

Do a sudo tail -f /var/log/syslog and plug the usb drive in.
Look for lines like these:
Feb 18 12:58:32 shuchi kernel: [17192272.616000] sda: assuming drive cache: write through
Feb 18 12:58:32 shuchi kernel: [17192272.616000] sda: sda1

Make a directory in /media named usbdrive.
Mount it like this: sudo mount /dev/sda1 /media/usbdrive

To unmount before ejecting:
sudo umount /media/usbdrive

If you get a message saying that the device is busy you can find out what process is using it using with the fuser command, with the -u parameter to see which user it is:

fuser -u /media/usbdrive

This will give you a number which refers to a process, and a user, then run ps -A to list all the processes and see which one it is. If you need to kill that process (USE WITH CAUTION) then run:

sudo kill -9 yourprocessnumber

You should then be able to safely unmount your USB drive

February’s quick links:

• Via my new colleague (Hi Ben), some tutorials for using QGIS and GRASS as opposed to (or in conjunction with) ArcGIS on an archaeological project
• Andlinux- a different way of accessing linux-based programmes without going through cygwin.  I’ve managed to get GRASS installed, though I haven’t tried it out in anger yet.
• A n00bs guide to installing linux programmes from source. Don’t be scared, it’s not that bad, though I’d add in how to get the command line to send it’s output to a log file for those times when it all goes wrong and you didn’t happen to catch the error as it scrolled past really fast…

Man is not the only option for help about a command in linux.
whatis — Display a summary of a command (rather than the entire manual)
apropos — Display a list of commands that pertain to (are apropos to) a keyword
whereis — Display information about the location of a command: the executable, the source code (if any), and the man pages.
which — Display which version of a command will execute (for when there are two, or more, commands with the same name installed on the system).

For all linux/Unix n00bs out there, here’s a really good explanation of the layout of the Unix filesytem.

What’s in a file name? The files on a UNIX machine are organized in a hierarchy. The very top of the hierarchy is / —commonly referred to as “slash” or “the root directory.” If you change your working directory to / and run ls, you’ll see several subdirectories with cryptic names like etc, bin, var, home, and tmp. Although UNIX now supports long file names, most of the monikers of these top-level directories hark back some 30 years to the origin of UNIX. Similarly, by the same long-standing conventions, each directory contained in / serves a special purpose:

• /bin is but one of many directories that contain applications and utilities. However, /bin typically contains utilities that are essential to system operation. Hence, the shells, file-manipulation commands such as cp and chmod, compression and decompression, and diagnostics reside in /bin.
• /sbin also contains utilities crucial to system operation and maintenance. However, the programs found in /sbin can be executed only by superusers—hence, “superuser-bin” or /sbin. /dev incorporates all the hardware installed on your system, including terminals and USB devices (and other peripherals that are physically connected to the computer), pseudo-terminals (used to interact with X terminal windows), and hard disk drives, among others.
• /etc (often pronounced “etsee”) is dedicated to system configuration. The /etc directory contains configuration files for the system daemons, startup scripts, system parameters, and more.
• /home contains a user’s home directories. For instance, if your login name is joe, the directory /home/joe acts as your personal file repository.
• /lib is the coffer for essential system libraries. In modern UNIX, system libraries are typically shared, meaning that the libraries are not linked and included in each binary (which would waste space, at the least), but are loaded on demand when needed and shared by many applications at once. Hence, core applications and utilities installed with UNIX require the libraries in /lib to run, and you need at least a small handful of the libraries to create new executables from source code. All files here are vital, and the corruption or removal (whether intentional or accidental) of even one file can render a system useless.
• /mnt, short for “mount,” is the standard location to mount hard disk drive partitions and other devices. If you want to see which devices are currently mounted and accessible, simply run the mount command.
• * /tmp, or “temporary,” is the system-wide scratch pad. Your Web server might stash session data files here, and other utilities use the space in /tmp for caching intermediate results. Files in /tmp are considered disposable. Indeed, your systems administrator probably deletes all files older than a certain expiry every evening.
• /usr is the umbrella for a great number of files. End-user applications—from editors, games, and interfaces, to system features—are here, as is the library of man pages along with much more. Chances are that if the file is useful but not mandatory for system operation, you’ll find it in /usr.
• /var —short for “variable”—is the repository for files that typically grow in size over time. Mailboxes, log files, printer queues, and databases can be found in /var. It’s commonplace also for Web sites to be kept in /var because a Web site tends to amass data preternaturally over time.

For all you linux n00bs out there just getting into compiling and installing programmes, this is really handy when your compilation fails and you’ve just watched reams of text scroll past with no chance to read what it says.  Basically it records the I/O of a command line session into a log file for later perusal.

Before entering your command in the terminal, type:

script yourscript.log

At the end of the session, press:

ctrl-d

To dump the output to the above-named file. Then you can read it with nano or the text editor of your choice.