robert's diary

Recent diary entries

Ordnance Survey have recently released their November 2014 version of OS Locator, the comprehensive gazetteer for GB. According to my calculations there are 12,201 new or changed entries and 10,203 removed entries since the last release in May.

I've updated my comparison tool Musical Chairs with the new data. New entries tend to show up prominently in the "recent relevant updates" view mode for a week or so after an update, so this is a good way of taking a look at what's changed in your area.

I'd suggest GB mappers take a look at their area, even if not for the purpose of mapping - new releases of Locator often reveal some interesting things about new building projects and developments.

Additionally, I've recently added RSS feeds to musical chairs to make it even easier to monitor your area for possibly problematic changes.

Next weekend is OpenStreetMap's 10th anniversary and people around the globe are celebrating.

If you've been around OSM for a while I'll bet you have a pile or two of these old flyers which have now been superseded by these ones, generously produced by Andy Allan. These old ones are therefore kind of useless. However, not being one to want to throw things away I've been searching for a use for my old ones for a while.

You will need:

Scissors

Tape

Two old OSM flyers

Use tape to join the ends of two flyers, making sure you get the orientation the way you want it.

Join the remaining ends of the flyer strip, producing a continuous ring of flyer.

Re-fold flyer ring back on top of itself and cut off top corner at approximately 45 degrees.

Ordnance Survey have just released their May 2014 version of OS Locator, the comprehensive gazetteer for GB. According to my calculations there are 17,925 new entries and 13,847 removed entries since the last release in November.

I've updated my comparison tool Musical Chairs with the new data. New entries tend to show up prominently in the "recent relevant updates" view mode for a few days after an update, so this is a good way of taking a look at what's changed in your area.

I'd suggest GB mappers take a look at their area, even if not for the purpose of mapping - new releases of Locator often reveal some interesting things about new building projects and developments.

Ordnance Survey have just released their November 2013 version of OS Locator, the comprehensive gazetteer for GB. According to my calculations there are 10,151 new entries and 7,730 removed entries since the last release in November.

I've updated my comparison tool Musical Chairs with the new data. New entries tend to show up prominently in the "recent relevant updates" view mode for a few days after an update, so this is a good way of taking a look at what's changed in your area.

I'd suggest GB mappers take a look at their area, even if not for the purpose of mapping - new releases of Locator often reveal some interesting things about new building projects and developments.

So I was rubbing shoulders with the traditi-gis crowd on Tuesday & Wednesday last week for the FOSS4G GeoHack event in Nottingham University. The team that I gravitated towards was being run by DEFRA folks eagerly clutching copies of their Open Government Licensed air quality data and providing the task of creating an air-quality-aware router for cyclists, walkers or other health conscious individuals.

The data is fairly interesting as it's a mix of measured and modelled data, and the most high-resolution data provided was that which was modelled specifically along a patchy network of major (OS-Meridian-derived from what I could tell) roads - so not necessarily the easiest thing to work with.

The plan from the table involved using postgis & pgRouting to route along the provided linestrings & build a web interface on openlayers for users. Now, OS Meridian isn't known for being particularly routable at the best of times, and the patchy nature of the provided linestrings was making this look like a daunting problem. On top of that I personally was worried about the ambition of building from scratch a from-to routing interface on top of openlayers in the remaining time (I've spent two day stints just working on single fiddly javascript bugs on web maps before). The stumbling hurdle seemed to be that everyone was running a windows on their laptops and so needed to negotiate use of a "server" with the requisite software installed for their purposes.

Now, see, I'm an openstreetmap dude. And I was a bit puzzled by why we weren't just using all the tools we already have. We're quite spoiled in the openstreetmap community, having a largely complete & high quality, inherently routable road network in many countries and a decentarray of webfrontedroutingtools, even counting just the open source ones. And just because we were given the data based on OS Meridian, doesn't mean we're stuck with OS Meridian for the routing as long as we're able to find a reasonable way to apply the measured data to the new road network. Yet people were still intent on re-doing all the legwork in patching together a melange of tools.

I've become quite familiar recently with Dennis Luxen's OSRM (Open Source Routing Machine) through my work on GPS trace analysis in That Shouldn't Be Possible™. The great thing about OSRM of course is that it comes with a pretty impressive web frontend that's stupidly quick to set up and truly shows off the speed & interactiveness you can get with OSRM. OSRM's also very customizable - the "profile" determining the weights given to each road is defined by a lua script. This gives a low barrier of entry to users wanting to create even fairly complex rules.

It just so happened that the hack I wanted to perform wasn't quite something that the lua scripting would allow. I wanted to add an extra weight to some streets by edge, and the OSRM lua calls are really built around assigning weights to whole ways. But this is Free software ("open source" if you like). We don't tend to let things like that stop us. So I hacked a call to a postgres database (using the wonderfully straightforward libpqxx) into the inner loop of OSRM's extractor pre-processing tool, modulating the weight of each edge by the result of this call. Really, it was such an easy hack that the hardest bit was remembering how painful string manipulation is in c++. And almost as much effort went into getting the build system to cope with the new libpqxx dependency, but hey, that's c++ for you.

As it was, I was left with a few options over how to apply the air quality data to the roads. In the end, I decided to use postgis to locate the nearest linestring in the air quality dataset for each edge and assign that air quality to it. If there's no air quality linestring within ~100m of the target edge, we assume a reasonably high air quality. There are many other ways we could have done it involving various interpolation schemes, but this we decided was good enough for a proof of concept.

I only had my little cheapo netbook with me and I spent 80% of the first day worrying about why my wifi card wasn't working, but fortunately I was able to scrounge some network access from others for any pieces of the necessary toolchain I was missing and remembered I had the greater_london osm extract from geofabrik lying around (my little netbook is comfortably able to cope with London given enough patience), so by the Wednesday morning I had a built & working demo london dataset. So there you go. Sixteen lines of c++. This is the power of Free software and open source that you don't get anywhere else.

We presented our work along with a nice looking "my local air quality" map put together by others (including a latecoming strk who also provided the air quality overlay used in our OSRM hack) using cartodb, and came in third. Which is not bad considering the amount of work some of the teams had clearly put into their projects over the two days.

The problem naturally with this tool and its limited dataset is that it will always just try and route around the areas it has data for. Authorities will necessarily measure and model those areas that are likely to have poor air quality and the resultant routes will more or less just try their best to stick to backstreets. This is a data problem that I will leave people like Tom Chance to rant about. If we wanted to just route across streets for which data is present (as was probably the organizers intention), it would have been a simple one line change to simply weight edges beyond n metres from an air quality linestring out of existence.

I left the GeoHack with a little bit of worry in the back of my mind as to why this solution wasn't obvious to others. I think it's that traditi-gis folks still see openstreetmap as a pretty tile ("base layer" as they would say) provider, and not the massive dataset and tool resource it is. This is possibly an attitude hangover from Google Maps, which for years people chose as their default "base layer", but would give you nothing but tiles. No data. No fun. And many people see openstreetmap purely as a Google Maps replacement. It's understandable. But we clearly still have a lot of awareness work to do amongst even our neighbour communities.

Ordnance Survey have just released their May 2013 version of OS Locator, the comprehensive gazetteer for GB. According to my calculations there are 5247 new entries and 3694 removed entries since the last release in November.

I've updated my comparison tool Musical Chairs with the new data. New entries tend to show up prominently in the "recent relevant updates" view mode for a few days after an update, so this is a good way of taking a look at what's changed in your area.

I'd suggest GB mappers take a look at their area, even if not for the purpose of mapping - new releases of Locator often reveal some interesting things about new building projects and developments.

Ordnance Survey have just released their November 2012 version of OS Locator, the comprehensive gazetteer for GB. According to my calculations there are 7609 new entries and 5077 removed entries since the last release in May.

I've updated my comparison tool Musical Chairs with the new data. New entries tend to show up prominently in the "recent relevant updates" view mode for a few days after an update, so this is a good way of taking a look at what's changed in your area.

I'd suggest GB mappers take a look at their area, even if not for the purpose of mapping - new releases of Locator often reveal some interesting things about new building projects and developments.