For better or worse, Mapnik has long had a reputation for being difficult to install. Ditto OSM planet dumps. I know and you know this is not true, but just because Dane has done so much work to drastically simplify the process doesn't mean you want to do it yourself, like how people stopped soldering their own MP3 players when the iPod came out.

Meanwhile, Amazon has a thing called EC2, the Elastic Compute Cloud, and it's amazing, really one of the biggest changes to my work as a technology/research/CTO guy. One of the things you can do with an elastic compute cloud is to create templates for new machines, and let other people run them. So you put these things together, and you get the Tile Drawer.

Tile Drawer is a new thing I've been working on that makes it fast and easy to run an OpenStreetMap tile server on Amazon's infrastructure for ten cents per hour, with just about no configuration at all. When you instantiate the Tile Drawer AMI (ami-e1ea0a88) you provide two bits of information:

Your new instance will boot itself up, and immediately start pulling down OpenStreetMap data to render map tiles with. This is pretty much the whole story.

I have a couple example stylesheets up for your use, and I'm very interested in adding more. I'm excited by the idea of a community-maintained collection of cartographic styles, all of which can be immediately applied to any geographic location covered by OpenStreetMap. I'm also excited by the idea that designers can create and rapidly use new kinds of map renderings - we've been plumbingthe depths of customized Mapnik renderings for a few years, and it's time to see it break out into wider use. We're already seeing beautiful new map designs like Craig Mod's Art Space Tokyo book, Wilson Miner's EveryBlock maps, and Matt Jones's Lynchian_Mid Cloudmade style for Dopplr. There should be more.

There are a few things that Tile Drawer does not do yet, because it's new. For example, it does not keep up to date with OSM updates. Since it's so cheap and easy to fire up additional instances, I'm imagining that this need can be addressed for the moment by periodically creating replacement Tile Drawers and killing old ones. It doesn't serve anything other than OpenStreetMap data. It doesn't do WMS. It doesn't render multiple styles at once. None of these things seem important yet, which doesn't mean they won't happen at some point in the future.

Comments
(22)

This is really, really cool. I've fired up a couple of instances but they are taking hours to download the initial data set - I wonder if the planet.openstreetmap.org server is being clobbered by a bunch of people trying this out at once. It would be nice if there was a mirror of the file on S3 or EC2 somewhere to save on the cost of bandwidth to an outside-Amazon IP.

Indeed - super nice.
This will definitely clobber someone's mirror or server.
You could simply make an EBS snapshot and make that available for anyone to attach to an instance. The problem with making it an official AWS public data set is that it would be a snapshot for a particular day - so they would want to keep moving the pointer up after diffs are applied.
This is particularly useful given the recent (planned) server outage of OSM. So encouraging people to go ahead and fire-up their own tiles.

Thanks guys!
Simon, it's possible that the planet server is getting a bunch of requests. You could try using one of the Cloudmade extracts, which are smaller and on S3. It would also be a good idea for me to start mirroring the main planet to S3 myself - I'll try and get that going today.
Anecdotally, I did a test run with the complete planet file and an extract of the NYC metro area, and it took about an hour for the planet file to download, and another *8* hours for the chosen bounds to be extracted - those files are big! It was more like 30 minutes total for the Cloudmade New York State extract.
Andrew, is it possible for multiple people to have the same EBS snapshot attached to individual servers? Does it become a read-only filesystem? I'll investigate this, too. Maybe that's what the public data sets are. I've talked to the InfoChimps guys about getting OSM up as a public data set, but it has that disadvantage of requiring a refresh every week.
Also if you're curious about progress, I think the output of the extract script should be visible on the EC2 console, since it's run as a startup script out of /etc/init.d.

Really great,
any chance for an AMI in the EU-West region (for us non-Americans). Or access to the code so that we can build it ourselves and/or run it on a local virtual machine?
Just something similar to the presenter notes for the MapsFromScratch AMI
This would be really helpful.

I'm so excited I could plotz. This is absolutely spectacular! Good show!
Here's another +1 for creating EBS snapshots of popular datasets. And yes, this fits *squarely* into the realm of public data sets, which Amazon hosts as EBS snapshots for free.
Check out the site here: http://aws.amazon.com/publicdatasets/

Posted by Aron Pilhofer on Thursday, August 27 2009 1:28pm UTC

I've been working on a set of scripts that can seed and expire tiles at: http://github.com/aub/tile_flip/tree/master and also another set for our app that will pull osm patches, apply the changes, and expire the appropriate tiles. If that's interesting to you, I'd be happy to have them as part of this project, which sounds great.

Marc, that sounds suspiciously like a bunch of work but not out of the question. ;)
Gerd, can you explain? I hadn't realized that AMI's were in some way region-bound - can't you just instantiate them in any datacenter?
John, do you think calling out additional zipped shapefiles would be enough? This is already effectively happening with the processed coastlines. If there were clear rules for the intake and postgis-ification of shapefiles, I think it would be possible to add new data like you're describing. Maybe in a near future revision!
Aron, I'm excited that you're excited. =)
Aubrey I'll have a look at that repository, it looks pretty awesome.

Hey, just one more note about the whole OSM mirrors deal - I'm very interested in the idea of a public data set of Planet.osm and/or a pg_data representation of it (since importing Planet.osm is time-consuming and PostGIS is the preferred backend for rendering) - but in the meantime, the king-nerd mirror ( http://planet.king-nerd.com/?C=M;O=A ) is absurdly fast from an EC2 instance - it might be actually an EC2 instance, I just haven't checked. Currently I'm pulling from it at 4-10MB/s, getting a Planet.osm file in about 25 minutes.

Wow, thanks for the tip Tom. I tried to push planet.osm to S3 but got a "file too big" error, maybe this other mirror is the right thing.
One other way to deal with the issue is to do very coarse sub-planets. I imagine just eastern/western hemisphere divisions might be enough to fit within S3's size limit.

I'm loving this whole notion. Quick question: can the renderer be instructed to render everything to every zoom level, producing a complete tileset that could then be slurped off the cloud and used in prosaic, static ways (publicity, slippymaps on extant real servers, etc)? Or is it driven by requests?

Posted by Andy Gates on Friday, September 25 2009 12:25pm UTC

Andy, the renderer is TileCache, which has a tilecache_seed command that can be told to pre-render a full set of tiles. It's driven by requests, but everything gets bagged off into a directory of images that you can use. You could definitely slurp it up into a prosaic, static context!