Discipline – nearly every day there’s some degree of “meh I do NOT want to do this, or at least there was for me. Once I reached the 3/4 mark though it switched to “meh I do NOT want to do this, but I’m so dang close to succeeding!”

Basic blog software. Like I said at the beginning of the month, my time was better spent writing content that taught than hacking on some blog I could then use. You’ll bikeshed the crap out of a blog you build yourself, don’t fall in this trap.

My advice

Take it small – I would up about 500 words (according to WordPress) a day by accident. Wasn’t a goal, but that’s about where things felt “done”

Work a little every day. I thought taking weekends off would give me a chance to pre-fill the buffer with entries and leave weeknights free – nope. I was too busy enjoying NOT writing and doing other things.

Pre-plan: I didn’t get to everything I put on my list, and things came up that I didn’t even think about. But when crunch time came to prep an entry for the next day, the list gave me a clear direction of a sequence that made sense and if all else failed gave me an idea I could run with on a short deadline. Ask people what they want to know about your topic. In my case I asked several experienced Django devs I know what they needed to know about GeoDjango, mixed in the stuff I thought they should know too and wrote them.

Don’t be afraid to cut into pieces, or walk through a “project”. There seem to be a lot of people showing up to read entries, but a lot of the time they’re coming a couple days after publication. And now the entries will be out “forever” so those that find it can continue at their convenience.

DO IT. You’ll learn a lot, you’ll practice even more, and you’ll help people who don’t know things just like others helped you when you didn’t know things.

Don’t shoot for more than a month.

Give yourself time off – weekends, Monday/Wednesday/Friday, anything.

It’s been fun, and this blog isn’t going anywhere – the topic sheet still has a bunch there. But I’ve got some life, and a mess of housework, to catch up on.

and users can drop points and polygons on your model, which will be saved into the database. Do you want the Google basemap instead? Extend BaseGMapWidget instead of BaseOsmWidget, remove the srid attrib, and you’re set. It’s amazing how easy those widgets make it – you don’t even have to know you’re dealing with a geometry in your save method.

The Django admin saves us mountains of time. Back when Django still wasn’t quite Django and it was an internal project it was common for the team to leave a meeting, scribble a schema on a napkin and have the admin ready for data entry in the matter of a couple hours.

Models are still my first stop for any project – build the basic frame, and start gathering data with the admin. But over the years we’ve all come to expect more from applications and Django has gotten much better tools for it too. Need to reorder fields? Hide the stuff that rarely gets entered? Use a custom form? It’s in the docs. With the GeoDjango admin though, things aren’t as well documented. So let’s look at some of the simplest things that can make an application better.

Changing the default map

Two modes are available, admin.GeoModelAdmin and admin.OSMGeoAdmin. GeoModelAdmin uses very simple tiles from OpenLayers that have state lines, rivers / water sources and a few other unmarked areas. I can’t imagine how it would be useful because there’s nothing to reference from, but if you have your own set of WMS tiles you can set wms_url and it will use them.

By default when you load the admin you’ll find yourself looking in the Atlantic off the coast of Africa. If you’re US-centric, try

Which will center you in a spot where you can see the entire US. default_zoom of 4 is a closer view if you don’t need to see New England. For Kansas, default_zoom of 6 gets you the whole state. Experiment with these three values and you can set up your web app to be “optimized” for entry anywhere. As even a neogeographer can tell, they’re in decimal degrees like you get from your GPS.

This works in OSMGeoAdmin as well (the admin I use the most). Just as it says on the tin, OSMGeoAdmin uses OpenStreetMap so there’s a LOT of data available on the basemap you edit on top of. BUT this piece of the docs means one thing – “that uses a spherical mercator projection” – we can’t just put decimal degrees in and it work. But we have the tools to make it work.

Other options in the admin class are available but some of the ones I could see using don’t seem to work – like max_zoom / min_zoom. “modifiable” set to false will force geometries not to be editable, but that’s not particularly useful in the admin.

Adding More Layers

We can add additional WMS layers to our maps, too – in this example we’ll add the current NEXRAD radar composite and a Topographical map. (additional layer info can be found at http://trac.osgeo.org/openlayers/wiki/AvailableWMSServices but many of them did not work for me)

IF you want this to be the default behavior for ALL of your admins, just name the file openlayers.html and openlayers.js instead of openlayers_extralayers.html/js. NOTE: If you do this you must copy/paste the entirety of openlayers.html and openlayers.js into YOUR openlayers.html and openlayers.js and put your code inside of the appropriate blocks. If you use my example, the template will extend itself, cause an infinite recursion and the dev server or your server process will promptly quit with no warning. Ask me how I know!! 🙂

Note this won’t work with OSMGeoAdmin immediately unless you use the openlayers.html / openlayers.js naming and follow the above concerns. If you don’t want to do that (and I would try to avoid it because it could get you stuck on an old version of the javascript in the future) create your own osm.html:

Occasionally you just need a quick map, and it’s not going to be used for interaction or even necessarily display on the internet (say a Keynote/PowerPoint presentation, for example). We could build it as a Google map, get a gigantic computer monitor and fullscreen the window and take a screenshot, and have it look like every other web map out there – or we can reach into the toolbox of those who are NOT neogeographers and borrow a few things.

Of course, it bears repeating that the results from doing this are not survey quality. Don’t use my recommendations / guidance to determine where not to dig to avoid buried lines, where to build a road, or anything else. But zoomed back a ways where the dot on the map is easily the size of a 40,000 population town, it’s close enough.

This is returning results with too much emphasis on distance and not enough on how long ago the activity at that distance was. So I tried a few things:

Restrict to activity only in the past 4 hours instead of a week (not bad! helps quite a bit.)

Restrict to activity within 1000 miles (maybe restricts too much? This left me with no results for Missouri where I write this, but there are no quakes recently near here.

Between them I got a pretty usable app. If it can’t find anything, “Nope.” is returned. For now, if anything was found it returns a list. If we wanted we could hook up GeoIP as a fallback, but since I’m playgrounding this on ep.io I don’t have c extensions to use the module and have to call it good with just HTML5.

With that we’ve built a small app with a data source, views, templates, and a “real” purpose all the way through. I hope it’s been helpful for several of you and apologize that some things wound up basically just being repeated from earlier topics, but hopefully after the in-depth back then it was easier to walk along as something was built from it.

In the spirit of the holiday (in the US we’re celebrating Thanksgiving) I thought I’d offer my appreciation / thanks to the projects I use and lean on to make my applications work. This isn’t a canonical list, I’m sure I’ve forgotten libraries here and there. And regardless of whether I use your code right now or not, thanks for releasing it where I can someday if it does something I need. I contribute everything I can when I can because everybody else has contributed so much that makes me more productive I want to follow suit.

If you blog, I’d ask you to consider something alone these lines of your own. As software authors there are lots of instances where our code is doing things we have no idea about until we’re told.

We’ve got our quakes app installed (update it if you’re following along at home, I committed a bunch of code changes for this entry) and our initial data imported.

However, we need this data to be updated somewhat regularly for the service to be useful. Sean shipped django-quakes with Celery support (which I commented out of the requirements file because I’m running this app on the neat ep.io auto-scaling webapp service‘s free tier and don’t have Celery access. If you run Celery just re-enable it, if you don’t use Celery or even know what it is (check it out!) you can use cron to schedule it just as well.

To avoid polling too often and bugging the USGS I’d set the interval at hourly or more under normal circumstances. We only care about quakes when our users do. I’d run it at least daily though so you keep history.

Since we care when our users do and earthquakes can’t be convinced to ONLY hit at certain minutes after the hour, what do we do? How about we check the latest quake we have and see if it’s beyond a certain age to force a refresh?

This is more of that “please don’t do this” code – yes, it works, but it’s bad. If you’re like me and want the site to auto-retrieve at a faster schedule when users are using it, you might be tempted to do this:

This is bad. Why? Our webapp isn’t a single-file line. When there’s an earthquake, people start going nuts. The resulting traffic will be somewhat of a flood, so imagine a bunch of these views all running at once. USGS will not be happy with us, and we’re doing all kinds of extra processing that just gets thrown out. (NOTE: If you’re trying this in the default dev server, it won’t work – it’s single-threaded so both will never be going at once.)

So what’s our next thought? Lock files (virtually, in cache instead of filesystem)!

We’re still not quite there. After all, at the time of this writing there hasn’t been an earthquake in 45 minutes – so every access kicks off a poll. Let’s add caching to store the last time a poll was done.

But what about people who hit the site while a poll is in progress? How about a time.sleep(0.5) loop?

Did you just throw your mouse? I dodged it. Yeah, that’s a bad idea. After all, it does tie up our Python processes at the moment when we’ve got hordes of eager, shaken (pun only slightly intended) users ringing the doorbell.

So what’s a programmer to do? Let’s set a context flag in the template and use it to trigger something in the template about “we just heard, we’re checking into it!”with what we already have.

We’ve discussed all sorts of things this month, now it’s time to start putting them together more completely than snippet-for-this, snippet-for-that. I’m not going to inline command + output on everything but I am going to spell out all of the steps – if you have trouble, let me know in the comments.

To accomplish this we’re going to build a simple web service I’ve snagged “isthatanearthquake.com” for.

First order of business is creating the project.

Because virtualenv and virtualenvwrapper rocks (`pip install virtualenv virtualenvwrapper`), I started with `mkvirtualenv isthatanearthquake`. My global virtualenv `postactivate` script includes the line `cd $VIRTUAL_ENV` which drops me in the env whenever I activate it. Next I created a `src` directory, created and cd’ed into `isthatanearthquake-git`, created `templates` and created requirements.txt with these contents:

Create the Django project with `python $VIRTUAL_ENV/lib/python2.?/site-packages/django/bin/django-admin.py startproject isthatanearthquake` (note you will need to specify your python version in this command)

`add2virtualenv isthatanearthquake` will make the new project show up on your PYTHONPATH.

Open isthatanearthquake/settings.py and `import os` at the top. Somewhere in the file (I tend to put it nearer the bottom define

PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))

Set up a database: DATABASES[‘default’][‘ENGINE’] should be ‘django.contrib.gis.db.backends.postgis’ and [‘NAME’] should be whatever you want to call the new database.

Set your template dirs to use the templates folder created just outside the project from earlier:

TEMPLATE_DIRS = (
os.path.join(PROJECT_ROOT, "../templates"),
)

Add ‘django.contrib.gis’, and ‘quakes’, to INSTALLED_APPS. Uncomment ‘django.contrib.admin’ (and ‘django.contrib.admindocs’, though that one isn’t strictly required it’s just my habit.) We’re done with settings.py for now.

Uncomment the line in urls.py including admin.site.urls. If you uncommented django.contrib.admindocs in INSTALLED_APPS, uncomment its include statement as well.

Create the database you configured for with `createdb -T template_postgis <dbname>`

`python manage.py syncdb` will create the necessary tables, create a superuser of your choice, and `python manage.py load_quakes` will grab the last seven days of earthquakes from USGS.

`python manage.py runserver` will pop up the development server, and now with a web browser we can hit http://localhost:8000/admin/, login and choose “Quakes” to see a list of what came in.

One of the hardest parts of doing geo projects is getting the data you need to do it in the first place. In the US at least there are mountains of data at the federal level, some at the state level and who knows what at the local level. There isn’t a single place I can go to for anything outside of census-type stuff. Which school for a given grade level would a child at this point attend? Maybe it’s published, maybe it’s not.

Most cities or counties will have a GIS department. Some of them will be great, and helpful towards your goal as a developer. Others won’t. Our world of the tools and technologies are leapfrogging traditional methods.

So here’s some of my favorite sources of different kinds of data, worth looking at for your next project.

TIGER/LINE: Gigabytes of shapefiles of things the US Federal government collects. I tend to look here first if I need basic location data.

SimpleGeo Places DB: SimpleGeo put a dataset of 21 million places (12ish million in the US) into the public domain last summer, and links to the file here. I have played with the data some and it’s pretty clean but often categorized inconsistently (but hey, it’s free for all to use) There are discussions online about various ways of getting it imported, shapefile conversion didn’t work for me and neither did wrapping it in a feature collection in GeoJSON – it’s just too big – nearly 8GB of JSON. The method I got to actually work was taking it line by line, deserializing and then processing. I split into a TON of files containing 20,000 places each and ran several processes to get it imported. It’s a BIG database, and pretty slow in PostGIS so be warned. I have no plans to put anything in production from it so speed isn’t that big of an issue. As a side note, people importing this dataset are probably their best sales tactic toward paying for their SAAS version.

Flickr Shapefiles: Also public domain, potentially useful if you need to bring photos into the mix.

Timezones: Full shapefile of timezones of the world. Useful for auto-detecting your users’ time by their location and auto-shifting times to them. Public domain.

Free IP/Location database: Similar to GeoIP but community built. Likely not as comprehensive, it had no idea where I was for instance.

data.gov: It’s hard to find what you want and sometimes hard to figure out how to use the format it’s in, but data.gov is as close to a one-stop-shop as you may come.

data.nasa.gov: Datasets published by NASA. This one’s still a work in progress, but has a better search than data.gov and includes most NASA data listed on data.gov.

I stopped short of displaying an equivalent overlay in Google Maps v3 yesterday, but wanted to circle back and show that it’s doable as well. It’s far more verbose than the magic-y “pass it a polygon and it does the rest”, but you have more control too.

View, pretty much just like the one we used for points only with a model that has a polygon:

But we want our apps to be more interactive – so let’s quickly add popup windows when something is clicked on. The blog isn’t going to handle a diff very well, and that’s the best way to show it, so check out this gist. It’s not very difficult at all. Now we get popups when somebody clicks a marker, without writing a single line of view code. I tend to build methods on my models for map_display_html() or something similar. If I’m really returning HTML I’ll put it in a template and use the template language to make it work, if it’s just a name and hyperlink or something similar I just leave it in the model.