django i18n support: fuzzy!

It took me some time until I discovered this, so I think it would be nice to post about it.

In a recent project, I needed to add i18n support to a django app. This was the first project that needed such support since the first time i used django some time ago, so I took a look at the official docs, I read some posts in some blogs over the internet and I followed three easy steps:

1- Inside my app dir, I used the manage.py tool to generate the needed .po files for my language (.es):

../manage.py makemessages -l es

2- Then I opened up emacs and I used the po mode to edit the .po file (locale/es/LC_MESSAGES/django.po inside my app dir). It is pretty easy to edit a .po file, you could use any text editor to add the translated strings in place, but the po mode has some interesting shortcuts to get it done quickly.

django verbose_name limitation

This is one of those things you never know about until it happens to you.

Some days ago, while working on a django project, I made some changes to an app models. Since I was at the beginning stages of the development process, I just droped the database, created it again and I used syncdb to recreate the whole thing (yes, I used syncdb because I didn't use nor django-evolution neither south or any other migrations tool).

It seems like something (probably in models.py) was wrong. I revised my models file, nothing about a field limited to 50 characters... strange.

As usual, I asked in #django (on freenode) and, as usual, Magus- pointed me in the right direction, asking me if I had any model with a name longer than 50 characters. Of course I do not have any model with such a long name, but searching a little bit deeper, I found that one of the models had a verbose_name had a 80-character long string associated.

Replacing that with a more appropiate string (below 50 characters) fixed the problem.

django forms and where to add dynamic choices

In django, you can use the so-called ModelForms to generate a form directly from a model definition. This is pretty useful, as you can have a models.py file like, for example:

classProject(models.Model):name=models.CharField('Name of the project',max_length=100)code=models.CharField('Code number of the project',max_length=10)creator=models.ForeignKey(User,related_name='created_projects')maintainer=models.ManyToManyField(User,related_name='maintained_projects')

In this case we are creating a Project model, that is, a model to define information we will handle for those projects (this is an example, of course it would have more fields). The first two fields are CharFields (text fields where we will save the name and a numeric code for the project). The next two fields are ForeignKey and ManyToManyField fields (links to another model, this time User, which is django.contrib.auth.models.User, that is, the default django user model).

I've added two different relations to the User model, one will be the creator of the project, the other one the maintainer (or maintainers) of the project.

Now, back to ModelForms, I can create an HTML form just adding something like that to the app forms.py:

and you can use the form as described in the forms documentation (I'll not step there in detail).

But there is one problem with that approach. Imagine I've created two different groups (using django.contrib.auth.models.Group), let's call them project creators and project maintainers. Only users in the project creators group should be available in the creator field and users in the project maintainers group should be available to select for the the maintainer field.

Ok, what we've got here is that I've overwrote or replaced the default creator and maintainer fields automatically generated by django, setting two new fields. This allow me to set certain default options for the fields, like the list of choices available (which I've generated searching for users in the needed groups).

Now a user will be able to select only users in the project creators group in the creator field and users in the project maintainers group in the maintainer field.

BUT, there is a problem with that. Doing it that way, the list of choices for each field will be generated on startup time (when you launch the django development server or apache or lighttpd or nginx or whatever you are using). That means that if you add a new user to the maintainers group, it will not appear in the maintainer field until you restart your server!

To avoid that, we will need to add the current available choices before using the form, overwriting the default list of choices:

Create your own pony with magical powers

Someone sent yesterday a link to mylittledjango.com to the europython mailing and I couldn't resist myself to take a look... This is the result, after some tests:

I have to say that I like the idea of the django pony since I heard about it for the first time, bud I didn't like the default colors chosen for the pony (pink?? what about the usual django green?). I really like the drawing, so having a web to generate a copy of the pony with your own custom/favourite colors on it... awesome!!

Now, go and get you one!

UPDATE:

I've see a lot of then in the pony gallery, but this is the funniest one of them all!

But then, this was not an IntegerField, but a CharField, which means that
using order_by would return something like:

0
1
10
11
12
13
...
2
20
21
...
3
...

And so on. Not exactly what was needed.

The following is the write-up covering the little adventure that was the work to
fix it.

My first thought was - hey, this is a string, it will be properly sorted if we
just fill the numbers properly -. Using python's zfill it would be really easy
to calculate the larger value, and then pad zeros to the smaller values and
sort them.

Sounded good, but you can't tell django's ORM to sort by a model method or
property. So that approach would mean writing some code to sort the results in
python code, after retrieving them from the database. Something you could do,
but that is not exactly a good choice (and then you would have to parse codes
a bit, as they are not numeric only).

Doing it at database level sounded like the way to go. The database should
be able to handle sorting efficiently, no need to write some more code at
application level.

Thinking a bit more about it I did recall of the CAST function from
PostgreSQL, that can be used to change dinamically the type of a field,
for example when making a query. This means that instead of a query like
this, that will sort the results incorrectly:

select*fromcustomerorderbycode;

We could do something like:

select*fromcustomerorderbyCAST(codeasint);

Which would first transform codes into integers, then order the results by
that. Nice! - I thought - That should do it. But it did not.

Remember that code has values that cannot be directly casted to an
integer (3-groupB for example). That means that if you run that query,
you will get an error like this one:

ERROR: invalid input syntax for integer: "3-groupB"

Here is where the substring function from PostgreSQL becomes handy. Using
such function, you can extract parts of a string/char value, matching
patterns, even using some regexps, as it is needed in this case.

Something like that should do it:

select*fromcustomerorderbyCAST(substring(codefrom'^[0-9]+')asint);

There, substring extracts characters from code that match numbers from
0 to 9, CAST changes that resulting string into an integer and the query
orders the results by that.

"Got it" - I thought. "Now, let's tell django's ORM to use that".

That was the easy part. Django makes it really easy with the extra method
of django QuerySets. Something like this should do it:

customers=Customer.objects.extra(select={'sorted_code':"CAST(substring(code from '^[0-9]+') as int)"}).order_by('sorted_code')

"Great, that was easy, let's give it a try" - and then I opened a django
shell and gave it a try, but all I got was a huge traceback ending with:

OperationalError: near "from": syntax error

"W T F" - I did exclaim. I did check the syntax like a thousand times, and
it would have worked flawlessly on any PostgreSQL server. I was sure of that.
And that was precisely the problem, for this project we have been using
SQLite instead. FACEPALM.

Anyway, SQLite is good enough and should be able to do this too, maybe just
using a slightly different syntax, right?

This is a bit more "complete" than the previous code we used in PostgreSQL,
as it has a fallback in case GLOB do not find any match to be casted,
returning the value of code as it comes from the db [1].

Et Voilà. That code could be used anywhere in your app. In my case, I had
to put it inside the get_queryset of a ModelAdmin subclass. This way the
django admin shows customers sorted correctly by default: