Monday, September 22, 2014

Parsing an Open Street Map pbf file with Python

Chris Hill at http://pbf.raggedred.net/ has written a parser in Python for open street map pbf files. His parser is free software, so I used this liberty to adapt it for my needs. In particular, his script either collects osm node, way and relation data either in memory (which is a problem for big pbf files) or it emits xml files with the osm data. I have changed his script so that I can pass three callback functions that are invoked as soon as the parser finished with one of the three fundamental osm type node, way or relation.

Here's a template that can be used to write a script that uses my adaption of the parser:

Each of the callback functions has exactly one parameter that corresponds to the classes
OSMNode, OSMWay and OSMRelation (see the
source at github).

Installing google's protocol buffers

To parse pbf files, google's protocol buffers are needed. The can be optained from
code.google.com/p/protobuf/downloads/list.
For Windows, I downloaded protoc-2.5.0-win32.zip which
contains one file: protoc.exe. After extracting this file, the environment variable PATH should be changed so that
it points to the directory with protoc.exe.

For the python installation, the full source protobuf-2.5.0.tar.bz2
is also needed. After extracting them, cd into the python directory and execute: