Eighteen Months of Django: Part 2

On Tuesday, I shared some best practices I picked up while using Django. This is a follow up post to share the packages that I found useful as well as various hiccups I encountered when using them.

django-registration and django-social-auth: Combined, these packages let you handle the basic user registration and activation. Most likely, you will end up having to customize them a bit to do what you want. For example, allowing a user to register using an email address instead of a username or requiring an email address for a user who signs up using Twitter. A small issue that annoyed me is that the signals generated by these two packages occur at different points: django-registration generates signals that includes the request while django-social-auth generates signals that contain the response from the OAuth provider. Depending on your use-case, it may be worth it to use the simple backend for django-registration, it automatically activates and logs-in the newly registered users, making your app a bit easy to get into.

django-storages and boto: If you plan on using S3 to host static content, definitely take a look at these. They provide backends to make it easy to save and access your static content to S3 without having to deal with the AWS API. I ran into some issues using this along with Cloudfront and django-compressor but I was able to fix them by looking at Stackoverflow.

django-compressor: This is a neat library that will compress and minify your JS and CSS, check if anything’s been updated, generate an upload the result to static files location, and update the HTML to point to the new location. This makes sure that users never end up with older, cached versions of your static files. One thing to note is that you need to make sure that your Javascript are properly formatted and all end in a semi-colon; otherwise you run the risk of the compression failing. I know that there are other Django compressors out there but I’ve been happy with django-compressor.

sorl-thumbnail and PIL: If you allow users to upload images this is a must have. It provides a standard way of resizing the images and caching the result. The library comes built in with support for cropping and a variety of other processing options so you don’t have to worry about it. One thing to note is that if a user is loading a page where none of the images have been generated yet, it will delay the page load until all of the images are generated. As long as you know the required sizes of all images, you can run a task on the backend to generate each of the images. You may have trouble installing PIL in a virtualenv but doing some Googling it should be easy to figure out.

django-extensions: Just a neat library that comes with additional management commands to make developing Django easier.

django-debug-toolbar: This intercepts every Django request and provides some debug information to help you optimize your code. The most useful piece to me is being able to see the SQL queries that are being executed and helps me figure out what needs tweaking/caching.

django-crispy-forms: If you’re using Twitter Bootstrap, this is a library that lets you generate Bootstrap forms in Django.

django-celery and celery: This is a way to run tasks in the background. With Pressi, we initially started with some management commands behind some cron jobs but we ended up switching to Celery when we wanted to distribute it across multiple machines and have built in support for threading and error handling. One thing to note is that we used RabbitMQ as the backend but it takes a bit of time to setup and I’m still struggling to understand the ways to manage it. A lot of people have been using Redis as the backend successfully and I think I’ll give that a go in future projects.

mongoengine and pymongo: If you’re using Mongo, take a look at mongoengine, which serves as an ORM for Mongo, and is built on top of PyMongo, a Mongo API. Mongoengine makes it very easy to change your models from a relational database to an documented-based one by keeping the field types and model definitions similar. Be aware that document-based databases are significantly different from relational ones and that although cosmetically your models look similar, the interaction with the backend is very different. You shouldn’t switch to MongoDB just because you can - make sure you’re switching for the right reasons. For Pressi, we use a hybrid approach where we use MongoDB to store a user’s social media content with everything else stored in MySQL. Something to be cautious of is that both of these libraries have been evolving pretty quickly and we ran into an issue where we weren't able to consistently connect to a MongoDB instance until we stumbled unto the right versions of the libraries (in our case, 0.6.20 for mongoengine and 2.4.1 for pymongo).

django-haystack: When you’re ready to graduate from implementing a search using QuerySet filters to an indexing backend, take a look at Haystack. It provides a pretty simple search interface that integrates pretty well with Django and supports a few different backends. We ended up settling on the Xapian backend because it was supposedly simpler but ran into some trouble installing it inside a virtualenv until I found this post. Note that although Haystack supports multiple backends, not all features are supported by every backend so make sure the backend you choose supports everything you need. I believe Solr has the most functionality out of the box but we wanted to keep it simple for Makers Alley.

django-postman: We just implemented this for Makers Alley but it’s a very simple way of doing user to user messaging. It comes with the standard messaging features (inbox, reply, archive, delete) but one thing I wish it had was a way to include attachments.

Fabric: I mentioned this in the previous post but wanted to reiterate it since it makes building and deploying your code easy. It also forces you to think about your environment and you end up with a better structured project as a result.

South: Another package I mentioned earlier that makes it significantly easier to deal with database migrations in Django. The only time we've run into issues using South is when two of us were making changes to the same model in parallel branches. Even then it's easy to replace the two flawed migrations with a functional one.

Unidecode: This isn’t a Django specific library but we found it useful when cleaning up unicode data. If you ever get random unicode exceptions in your code, Unidecode should be able to help.

BeautifulSoup and PyQuery: If you need to do some HTML scraping in Python, take a look at BeautifulSoup. It turns HTML code into an object that’s easy to navigate and search. After getting more and more familiar with jQuery, I found a python alternative in PyQuery but am still getting comfortable with it. If you come from the jQuery world I’d try using PyQuery first; otherwise I’d try BeautifulSoup.

requests: Just a nice and simple replacement of urllib and urllib2 that makes it much simpler to make HTTP requests. Your code becomes cleaner, more readable, and more expressive.

I tried to highlight the libraries that have made developing in Django easier but I’m sure there are tons more. I’d love to hear about them so do share.