Ward maps on OpenlyLocal (& how I did it)

Yesterday we added a feature to OpenlyLocal that I’ve been wanting to do since the beginning: ward maps. Why is this important? Simple, because hardly anyone knows what council ward they’re in, and nobody knows where the boundary lies, and the ward is the most basic unit of democratic accountability.

True, some councils have outlines of the wards (but without placing them on a map) and it is visible on the ONS’s Neighbourhood Statistics site, though it’s a pretty challenging user experience:

But to me, this should just be something that is there on the OpenlyLocal ward page, where the links to relevant councillors, stats and other data sit… and now it is. Here’s the page for Wembley Central ward in north London, for example, complete with zoomable maps with all the usual Google extras (satellite view etc):

Since tweeting about this, I’ve had a few people ask me how I did it, and so here are the details.

Over that afternoon (on the suggestion of @danslee and with input from everyone else at the event) we did a ward comparison proof-of-concept using some boundaries Stuart had already imported into a database. The key thing, I think for both of us, was getting a few hours of focus just on the ward mapping problem, checking up on the various bits of code and interface with Google maps, and working out how to draw the outlines. Stuart used a Windows command line program and some PHP to get the boundaries; when I did it for the wards I used Ruby, the language OpenlyLocal is programmed in.

How I did it

The data comes from the newly opened-up OS BoundaryLine dataset (easiest to download from MySociety here), specifically the May 2010 data. The first problem is that this comes in the form of ESRI Shapefiles, which are standard for geo geeks, but not for data mashups, or mixing with online maps. The first stage was to import these into the database, and for this I used the GeoRuby library. Specifically I did this:

This just reads the shapefile (and associated data files), goes through the shapes, extracts the geometries and associated attributes (e.g. IDs, area size etc), and lets you do something with them.

If you’re using the Ruby Spatial Adapter library you should then be able to store them in the database easily. Except…

Except, the geometries are polygons made up not of latitude/longitude points but of OS northings/eastings. More than that, they are using a different model for the shape of the Earth (OSGB36) rather than the more normal (although arguably US-centric WGS84). Now doing these two conversions is not trivial, and while there are quite a few libraries for other languages I didn’t find one for Ruby, and so I wrote one, basically converting a Javascript version line by line into Ruby (gist here).

Like all direct code conversions, it’s not pretty, it isn’t fast, but I have tested it and it seems to work well. After that it was a simple matter to add this to the loop, put it in a ruby command line script called a rake task and let it run (took about an hour or so). Specifically this is what I put in in place of the ‘do something with the data’ line:

wsg84_polygon = Polygon.from_coordinates(wgs84_lat_long_groupings)
# if you're using the Spatial Adapter you should be able to save this polygon in a 'polygon'
# type field in the database.

So far I’ve done all the district wards; I’ll do the Unitary and County Councils next, but showing them as polygons on a map you quickly start to add a lot of bulk to the page, which for people using screenreaders or mobile devices is not good. So I’m investigating other options, including Google Fusion tables (though I’d like to steer clear of Google-only solutions), and running my own tile server. Suggestions, comments welcome.

In the meantime, hope this helps, and if people will find it helpful, I’ll make the boundaries available via the API as a KML file.

Another way of converting the eastings/northings to WGS84 is through ogr2ogr, something like this:

ogr2ogr -s_srs EPSG:27700 -t_srs EPSG:4326 destination.shp source.shp

Used this method for my election map experiment recently, seemed to work ok.

You could possibly generalise the polygons which will help somewhat – there might be a Ruby implementation of the Douglas-Peucker algorithm somewhere which you could use to cut down on the number of points in each poly.

I’m interested in the KmlLayer class as part of GMaps API v3, seems like it could be a good way of avoiding performance issues, though that would make your maps more tied to Google.

Any database worth its salt nowadays can deal with co-ordinate transforms, which I strongly encourage people to use wherever possible. Mapit just imports Boundary-Line OpenData shapefiles into it directly with the database thankfully doing all the hard work of remembering the co-ordinate system used. It’s a far cry from what we used to do back in 2003-4 and very welcome too.

When we rebuilt the openly local map we knew that there were hundreds of hyperlocal sites, Facebook pages and Twitter accounts out there that weren’t on the map. And that the map database contained hundreds of sites that had ceased publishing as natural wastage since 2009 in a fast moving sector where people are testing … Continue reading Archiving sites […]

The old OpenlyLocal map only had about 5 sites for Devon, mainly the defunct ‘Local People’ franchise. This couldn’t be right for England’s second largest county. Devon has a strong tradition of local pride in place and its rural nature meant there are obvious advantages to having websites for local civic life. As well as a … Continue reading Devon and Somer […]

Living and working in rural Aberdeenshire, where I run my own hyperlocal site, I am the official Scottish correspondent for Talk About Local. Following on from Will’s post about filling in the white space around Wakefield on the Local Web List map I thought I’d share my experiences adding in sites from Scotland. Way back … Continue reading Finding Scottish h […]

We’ve updated the search on the site so it includes meta data. WordPress doesn’t search this data, tags & categories, out of the box so we’ve added it in. So if you want to search for sites built on a certain platform type the platform in to the search box and hit enter, the same … Continue reading Updated Search

Carnegie UK Trust is supporting Talk About Local in improving the map of UK hyperlocal sites. Despite having hundreds of entries, the map when we rehosted it had some curious white spaces – it can’t be right that Devon only had five sites listed nor that towns such as Wakefield had none, nor Scotland about … Continue reading How to approach white spaces in […]

Email Subscription

Enter your email address to subscribe to this blog and receive notifications of new posts by email.