Migrating assets to Django Filer

August 13, 2015

django-filer is a great tool for reusing uploaded content across Django sites. It's an easy choice for new projects, but what about existing projects? Painless steps for migrating existing image and file uploads to django-filer.

When you want to add image support to your Django app, like allowing
content editors to upload images, how do you do that? Probably by using
Django’s built in ImageField model field, which will store the file on
the file system (or other storage backend). But what about when you want
to be able to reuse images? Using a plain ImageField means you have to
upload images for each new use or perhaps create your own Image model
as a related entity… neither option tends to work out well for the
people actually using the site.

For Django projects, the best solution we’ve found so far to this
problem is
django-filer. Filer is an application that combines
convenient model fields for developers with a familiar file
tree like structure for users. Combined with standard image fields like
captions and alt text, as well as extra thumbnailing support, filer
stands out as a superior way to manage image content.

This is all well and good when you’re starting a new project, but it’s
never too late to make the transition to django-filer. Here we’ll walk
through the process of rescuing your image content and surfacing it for
your users.

Install Filer

Installation is a perfunctory first step. You’ll need to download
django-filer (preferably by adding it to your project’s pip
requirements file) along with easy-thumbnails, which filer uses to show,
well, thumbnails of your images.

Pretty easy! You’ll notice two important things right off the bat. The
first being that we have to use a new name for this field since two
fields can’t share the same name, and two that the new field is
nullable.

Regardless of what you want your final data model to look like, the
column has to be nullable in our first step in order to simply add the
database column. After we’ve added all the content in you can go ahead
and remove this ‘feature’ if you want.

The great data migration

The big step then is the data migration. We need to move all of the data
from the old image fields to the new image fields. The nice thing is
that we don’t need to move the files themselves! That’s a misconception
I’ve heard voiced before, but in reality all we need to do is ensure we
capture the references to these files and then delete the old references

which is really all a file field is, image fields included.

For current versions of Django that looks like this:

python manage.py makemigrations --empty breeds

Using South with an older version of Django, the command looks like
this:

python manage.py datamigration breeds migrate_to_filer_images

That will create a data migration file named migrate_to_filer_images
for the app breeds.

In our data migration we’re going to cycle through all of the existing
Breed instances and either find or create new FilerImage instances
for each image path.

The first thing to notice is the no good, very bad, terrible thing here,
directlyly importing the models into the migration file. This is
exactly what the default migration template tells you not to do!
There are good reasons for not doing this, generally, however here
following the guidelines doesn’t work. Filer uses multitable inheritence
to subclass File in the Image model, so South’s internal schema (and
likewise the subsequence Django machinery)
doesn’t see a relationship between our table and the Image table. So
instead we import the models with the implicit understanding that we’ll
squash these migrations later (its terrible anyhow to find long since removed
apps in your migrations).

The next thing to notice is that we’re using the get_or_create method
here to avoid creating duplicates. We shouldn’t find any, but this is
an excellent way to avoid problems with edge cases. We can populate some
of the initial data from our model directly and change it later as
desired.

The ImageField on our model is really a foreign key so we need to
create our Image instance and then assign it to the individual breed.

{% include image.html src=“/images/django-filer-vizsla.png” alt=“filer” caption=“Both the old ImageField and the FilerImageField shown” %}

Update all of your {{ templates }}

We have filer images now so we’re ready to start using them.

A simple URL reference like this:

<imgsrc="{{ MEDIA_URL }}{{ breed.image.url }}"/>

Now references the image attribute using the ImageField as a foreign
key:

<imgsrc="{{ MEDIA_URL }}{{ breed.img.image.url }}"/>

If you happen to be using easy-thumbnails you’re simply change the field
name provided to the template tag, from this:

<imgsrc="{% thumbnailbreed.image400x200 %}"/>

To this:

<imgsrc="{% thumbnailbreed.img400x200 %}"/>

If for some reason it turns out that changing your templates is too much
of a hassle, keep reading for a few alternatives.

Update references in forms and views

Similarly with templates you’ll need to update any forms and views. This
is usually pretty straightforward, with the exception of any custom
validation or data handling.

As with templates there’s an alternative way of getting around this at
least for simple cases. Any code in your forms or views that references
the image field as an image field will need to be updated to ensure
comptability with the foreign key presented by the ImageField.

Swapping out the old field

The last step is swapping out the old field. The primary way of doing
this is to make the old field nullable and ensure it’s no longer
required. You can take care of this in your forms, and if you’re using
the Django admin’s default ModelForm you’ll need to ensure this field is
allowed to be blank.

Removing the old field

The follow up here would be to remove the old field altogether. This,
however, is a post-deployment step. You should only do this once you’re
ready to squash or remove your migrations, since the way we’ve
implemented the data migration here depends on the presence of specific
fields on the model. Simplest way to do this? Just remove the content
from the data migration so that it does nothing and imports none of your
models.

This kind of data migration is a one-shot migration to deal with
legacy content. Once you’ve executed it you don’t need it anymore. You
won’t be running the migration again in your production environment,
only in fresh environments like test or development machines, in which
case there is no legacy content. So if you do decide to get rid of the
old field and/or rename the new field, clean up that data migration
first.

I referenced a couple of work arounds with regard to changing the field
name in the rest of your code, i.e. templates, forms, views, etc. Both
options require that you’ve gone ahead and removed the original field.
The first is to add a model property with the name of the old field.
This should return a file instance just like the models.ImageField
would.

If, say, what you’re primarily worried about is templates and you
happen to be using easy-thumbnails then there’s an alternate solution:
rename the new field to that of the old field. You’ll need to specify
the database column name to avoid having to do yet another migration, a
rather pointless one by this time.