I moved back from the California Bay Area to Belfast a while back and for various reasons it looks like I’m going to be here a while, so it made sense to have my belongings shipped over here. They haven’t quite arrived yet, and I’ll do another post about that process once they have, but I’ve been doing various tweets prefixed with “[shipping]” during the process. Various people I’ve spoken to (some who should know me better) thought this was happening manually. It wasn’t. If you care about how it was done, read on.
I’d been given details of the ship carrying my container, and searching for that turned up the excellent MarineTraffic which let me see the current location of the ship. Turns out ships broadcast their location using AIS and anyone with a receiver can see the info. Very cool, and I spent some time having a look at various bits of shipping around the UK out of interest. I also found the ship’s itinerary which give me some idea of where it would be calling and when. Step one was to start recording this data; it was time sensitive and I wanted to be able to see historical data. I took the easy route and set up a cron job to poll the location and itinerary on an hourly basis, and store the results. That meant I had the data over time, if my parsing turned out to miss something I could easily correct it, and that I wasn’t hammering Marine Traffic while writing the parsing code.
Next I wanted to parse the results, store them in a more manageable format than the HTML, and alert me when the ship docked somewhere or set off again. I’ve been trying to learn more Python rather than doing my default action of turning to Perl for these things, and this seemed like a simple enough project to try out. Beautiful Soup seemed to turn up top for HTML parsing in Python, so that formed the basis. Throwing the info into a database so I could do queries felt like the right move so I used SQLite - if this had been more than a one off I’d have considered looking at PostgreSQL and its GIS support. Finally Tweepy made it very easy to tweet from Python in about 4 lines of code. The whole thing weighed in at only 175 lines of code, mostly around pulling the info out of the HTML and then a little to deal with checking for state changes against the current status and the last piece of info in the database.
The pieces of information I chose to store were the time of the update (i.e. when the ship sent it, not when my script ran), reported area, reported state, the position + course, reported origin, reported destination and eta. The fact this is all in a database makes it very easy to do a few queries on the data.
How fast did the ship go?
sqlite> SELECT MAX(speed) FROM status; MAX(speed) 21.9
What areas did it report?
sqlite> SELECT area FROM status GROUP BY area; area - Atlantic North California Caribbean Sea Celtic Sea English Channel Hudson River Pacific North Panama Canal
What statuses did we see?
sqlite> SELECT status FROM status GROUP BY status; status At Anchor Moored Stopped Underway Underway using Engine
Finally having hourly data lets me draw a map of where the ship went. The data isn’t complete, because the free AIS info depends on the ship being close enough to a receiving station. That means there were a few days in the North Atlantic without updates, for example. However there’s enough to give a good idea of just how well traveled my belongings are, and it gave me an excuse to play with OpenLayers.
(Apologies if the zoom buttons aren’t working for you here; I think I need to kick the CSS in some manner I haven’t quite figured out yet.)