Blogroll

Forums

python

Translating your Python/PyGTK application

In this tutorial we will go over the (very) basic steps that you can use in order to translate (localize or internationalize) your PyGTK application. For this tutorial we are going to use the PyWine application that we have been working with in two previous tutorials:

The full source and necessary files for this tutorial can be downloaded here.

Creating our Portable Object Files

In order to translate our text we are going to use the gettext module. The gettext module is basically a wrapper for the GNU gettext utility whose documentation can be found here.

The first step is to edit our python file (pywine.py)and “mark” all of the strings that we want to translate in our python code. To do this we will use the _(“xxx”) standard that most people use to mark strings that need to be translated. Otherwise in situations like the following:

#Get the treeView from the widget Tree
self.wineView = self.wTree.get_widget("wineView")

“wineView” might be thought of as a string to be translated when it clearly should not be. So we need to go through the .py file and wrap each of my English strings in _(). Here are all of the strings that need to be marked in their marked form:

The next step is to generate the Portable Object Template file using the command line gettext utility. If you are using Linux or OS X this should be no problem and gettext will probably already be installed. If you are using Windows the tools are available as .exe files from the GNU gettext FTP or this gettext for Win32 sourceforge project.

To create the PO Template you need to pass the gettext utility some information. The first thing is that the file being scanned is a python source file:

--language=Python

The second is that we only want to translate our marked strings:

--keyword=_

The third is that we want to create a .pot (PO template) file:

--output=pywine.pot

And the final argument is the file that we want to get the strings from:

pywine.py

So to put it all together it will look something like this:

xgettext --language=Python --keyword=_ --output=pywine.pot pywine.py

That would be all that we would have to use, but since we used Glade for our GUI we also have to extract and translate the strings from our Glade project. To do so the following line is used (taken from the great PyGTK FAQ entry: How do I internationalize a PyGTK and libglade program?):

intltool-extract --type=gettext/glade pywine.glade

That will create the file pywine.glade.h which contains the following:

You’ll notice that the strings are marked with the N_() mark. So when we create our PO template file we will want to take into account the pywine.py file, the pywine.glad.h file, the _() mark, and the N_() mark. To do so we will use the following command line:

As you can see it contains both the translatable strings from the pywine.py file and the pywine.glade.h file. The next thing that we need to do is create a .PO (Portable Object) file. Each .PO file represents a specific translation to use. To create our PO file we will use the PO template file that we just created. We will use the msginit command line utility.

When running msginit we need to tell it what the input file is:

--input=pywine.pot

And what our current locale is. For me since I am an English speaking Canadian mine would be en_CA, which is language_COUNTRY. If I spoke Quebecois fluently (French Canadian) I might use fr_CA. You specify the locale as follows:

--locale=en_CA

If you want to see what locale you are, enter locale -a on the command line on a Unix-like operating system.

The command line to enter is as follows to create an English Canadian file:

msginit --input=pywine.pot --locale=en_CA

Let’s create an English US file as well

msginit --input=pywine.pot --locale=en_US

You may be asked for your email address so that someone can email you about your translation. If you entered the above you should see: en_CA.po and a en_US.po created in your pywine directory. Let’s edit this two files so that they are slightly different:

The next thing that we need to do is create the .mo file, which is the file that the gettext module will use for translation. The .mo files are simply binary versions of the ACSII .po files.

We first have to create the correct directory for the .mo files. Which is explained in bindtextdomain function for the Python gettext module:

bindtextdomain( domain[, localedir])
Bind the domain to the locale directory localedir. More concretely, gettext will look for binary .mo files for the given domain using the path (on Unix): localedir/language/LC_MESSAGES/domain.mo, where languages is searched for in the environment variables LANGUAGE, LC_ALL, LC_MESSAGES, and LANG respectively.

So for the above example we need to create the following directories:

en_CA/LC_MESSAGES
en_US/LC_MESSAGES

Which can be done using the following command:

mkdir -p en_CA/LC_MESSAGES
mkdir -p en_US/LC_MESSAGES

Now we use the msgfmt command line program to create our .mo files in the correct directory:

The Python Code

Now that we have our translation files ready we need to actually implement some python code in order to use them. Fortunately there really isn’t that much code for us to write, but I found a real lack of examples or information on the correct way to do this, and the code to be somewhat confusing.

The first step is to import the gettext and locale modules that we will use.

import locale
import gettext

The next step is to create a simple define for the domain of our translations, which is generally just the application name. The gettext module will use the domain name to look for .mo files. So if you domain is set to “12345” gettext will look for 12345.mo files. You’ll notice that we created our .mo files are pywine.mo above so we are going to use “pywine” as our domain:

APP_NAME = "pywine"

Now, in general on a Unix like operating system you would install your .mo files into the correct locations in the /usr/lshare/locale directory. But since we are not installing this application and we want it to be easily portable to other operating systems, we will be storing the translations in the local directory.

Here is all of the code that we need to add to the __init__() function of the pywine class. It is quite complicated but hopefully the comments and my explanation after will make it easier to understand:

#Translation stuff
#Get the local directory since we are not installing anything
self.local_path = os.path.realpath(os.path.dirname(sys.argv[0]))
# Init the list of languages to support
langs = []
#Check the default locale
lc, encoding = locale.getdefaultlocale()
if (lc):
#If we have a default, it's the first in the list
langs = [lc]
# Now lets get all of the supported languages on the system
language = os.environ.get('LANGUAGE', None)
if (language):
"""langage comes back something like en_CA:en_US:en_GB:en
on linuxy systems, on Win32 it's nothing, so we need to
split it up into a list"""
langs += language.split(":")
"""Now add on to the back of the list the translations that we
know that we have, our defaults"""
langs += ["en_CA", "en_US"]
"""Now langs is a list of all of the languages that we are going
to try to use. First we check the default, then what the system
told us, and finally the 'known' list"""
gettext.bindtextdomain(APP_NAME, self.local_path)
gettext.textdomain(APP_NAME)
# Get the language to use
self.lang = gettext.translation(APP_NAME, self.local_path
, languages=langs, fallback = True)
"""Install the language, map _() (which we marked our
strings to translate with) to self.lang.gettext() which will
translate them."""
_ = self.lang.gettext

Now I will explain the code in smaller pieces so that it is easier to understand what is happening. The first step is to get the path to the python file that is running, it’s pretty straight forward so I won’t explain it her.

The first real step is get get a list of languages, or locales that we want to support. This list will be used to determine both the order in which we search for .mo files and what .mo files we look for.

The first locale we get is default locale, we use the locale module to get this:

# Init the list of languages to support
langs = []
#Check the default locale
lc, encoding = locale.getdefaultlocale()
if (lc):
#If we have a default, it's the first in the list
langs = [lc]

The next step is to get the list of languages supported from the operating system environment. The string that is returned by this needs to be parsed up into a list so we will also handle that. On Win32 and OS X this value is not set, so we need to make sure we check if it is None:

# Now lets get all of the supported languages on the system
language = os.environ.get('LANGUAGE', None)
if (language):
"""langage comes back something like en_CA:en_US:en_GB:en
on linuxy systems, on Win32 it's nothing, so we need to
split it up into a list"""
langs += language.split(":")

The final step in building our locale (or language) list is to add on the locales that we know we support, i.e. the .mo files that we have. We can add these to the list in whatever order we want, or if we are sure that they are all installed we could just add what we consider to be the default locale since we are sure that it will be found.

"""Now add on to the back of the list the translations that we
know that we have, our defaults"""
langs += ["en_CA", "en_US"]

Now the reason that we add these last, is so that we can let the user’s language settings control what .mo file is actually used to for the translations. We add our own just in case nothing is found.

The next step is to bind our domain (pywine) to a specific folder (the local folder) and then to set the current domain, this will be used when we perform the search for .mo files:

bindtextdomain( domain[, localedir])
Bind the domain to the locale directory localedir. More concretely, gettext will look for binary .mo files for the given domain using the path (on Unix): localedir/language/LC_MESSAGES/domain.mo, where languages is searched for in the environment variables LANGUAGE, LC_ALL, LC_MESSAGES, and LANG respectively.
If localedir is omitted or None, then the current binding for domain is returned.

textdomain( [domain])
Change or query the current global domain. If domain is None, then the current global domain is returned, otherwise the global domain is set to domain, which is returned.

The next step is to get the actual translation that we want to use and connect the getttext with the .mo file:

translation( domain[, localedir[, languages[, class_[, fallback[, codeset]]]]])
Return a Translations instance based on the domain, localedir, and languages, which are first passed to find() to get a list of the associated .mo file paths. Instances with identical .mo file names are cached. The actual class instantiated is either class_ if provided, otherwise GNUTranslations. The class’s constructor must take a single file object argument. If provided, codeset will change the charset used to encode translated strings.
If multiple files are found, later files are used as fallbacks for earlier ones. To allow setting the fallback, copy.copy is used to clone each translation object from the cache; the actual instance data is still shared with the cache.
If no .mo file is found, this function raises IOError if fallback is false (which is the default), and returns a NullTranslations instance if fallback is true.

So the function will search for .mo files to each of the languages in langs in order and then returns a GNUTranslations object on success or a NullTranslations object on fail (since we set fallback to True.)

We then bind the object returned by translation’s gettext function to the _. So when you call _(“xxx”) you are really calling self.lang.gettext(“xxx”). Calling _(“XXX”) is just a convenient shorthand and somewhat of a standard way to “mark” strings for translation, as we saw above.

So that’s really all you have to do. Now if you execute py wine using the following:

PyWine$ LANG=en_CA python pywine.py

You will get the “Wine Canadian Style” header, then if you execute it as follows (on a Unix like OS):

PyWine$ LANG=en_CA python pywine.py

You will get the “Wine US Style” header. Then if you execute it with French from France (on a Unix like OS):

PyWine$ LANG=fr_FR python pywine.py

The results depends on how your system is configured, and you will either get the Canadian text or the USA text.

The full source and necessary files for this tutorial can be downloaded here.

Conclusion

So that it for translation, using this method should work on all available operating systems with no problem. I wanted to include a language file for totally different language (i.e. a French, or Spanish, or whatever .mo file) but my skills in multiple spoken languages are not that strong.

So if anyone wants to perform a simple translation on the pywine.pot file and sent it to me that would be very much appreciated. Then we could start getting people to test this on systems with different language settings.

While researching this tutorial I came across some really good information from these sites:

Pythy – Thanks for the information. I tried simply using gettext on my debian machine however it never seem to grab the correct language. Once I switched to using my method everything seemed to work. I will look at it a bit more.

Also thanks for mentioning pygettext and msgfmt.py, I was going to use those but then I read in some places that there were some problems with those and Glade, so I just when the GNU route. I will look into that as well.

It looks like you’ve got some really neat things happening on your website as well, too bad I dont’ speak any Russian!

If the program will be used on Windows, I believe it is a good idea to not just rely on gettext’s behavior of searching only the environment strings for the list of languages. (From what I can tell from reading the Python 2.4.4 implementation, gettext.find only searches the env strings; it does not call the locale module to obtain either the current or default locale.) These env vars are typically not set on Windows.

If you are developing library modules rather than a stand-alone Python program, it seems like a good idea to first try the language returned by locale.getlocale before trying locale.getdefaultlocale, in case the program that imported your module has changed the locale.

Thanks for the information, that’s part of the problem that I had with this translation work. There is a lot of information out there but it’s really difficult to figure out exactly what all of it means.

I’ve read the PyGTK FAQ Entry and I’ve made some changes: I’ve add the following line:

_ = gettext.gettext
FILE_EXT = "pwi"
APP_NAME = "pywine"

But if I do that almost all the translation stuff code is not used, because we will not call gettext.translations, so we don’t use langs or language. So if I comment all the translation stuff lines except:

thanks for this nice tut. I shamelessly cut’n’pasted the needed python code into my project BlueProximity at http://blueproximity.sourceforge.net which I hope is OK to you. You will get credits in the Changelog as being helpful with the translation infrastructure if that is alright with you. Please mail me for details.

First, thank you for the excellent write-up. Very few examples of using internationalization and localization exist for Python.

Second, many operating systems, including Ubuntu, fail to explicitly set the LANGUAGE variable. This leads to many non-working pieces of code. Your solution is one of the few that addresses this correctly.

Third, the most simple usage of “from gettext import gettext.gettext as _” is often used as a temporary solution between coding your Python program and localizing it. Alternately, some put in the placeholder “def _(str): str” to allow coding in internationalization before beginning it.

Finally, could you explain the odd issues of making a translation for a language. That is, will a locale of “de_DE” grab the “de” translation if there is no “de_DE” directory?

I have a problem of locale,i have set my os(ubantu) locale to en_us,its like that your example pywine.py i am able to translate only in the locale i have set my operating system to,i want translations to happen in say en_CA, i have done all the required things like creating pot files,po file ,editing po ffiles,creating folders,creating mo files but then also i am getting an error.
\\Gtk-WARNING **: Locale not supported by C library.
Using the fallback ‘C’ locale.\\

i working on an application that is build in python, pygtk and glade but i say i add any module to the application ,i need to again generate all the pot files and po files,again after creating the po files all translations vanish up and we are left with to again add the translations for the whole application,is there anything can be done regarding it or at least that if we again generate the po files, the already translated files are not affected….

Thank you very much for writing this tutorial. I have also included my thanks in the ChangeLog for my project, Rapid Photo Downloader.

One comment: while testing out translations people had done for my project, I spent some hours trying to figure out why the strings generated by Python code would be translated without difficulty, while the glade generated GUI would not be translated at all. On Ubuntu, the solution is to replace this (for example):

LANG=fr_FR ./code-to-run

with this:

LANG=fr_FR.UTF-8 ./code-to-run

The second method works just fine.

Another comment: I was able to translate combobox values by subclassing gtk.ComboBox, storing the “real” value in a hidden 1st column, and displaying the translated value in a 2nd column, which is what the user sees. Perhaps there is a better way but this works fine for me at the moment.