Posts tagged with 'translations'

Launchpad translations will be unavailable for around one hour, starting 10.00 UTC, on Tuesday 2011-11-29, to allow us to open the translations for the next Ubuntu release, Precise Pangolin (to be 12.04 LTS).

We tried this last week but hit some problems. Rather than prolong the disruption, we decided to bring translations back online and delay the opening of Precise’s translations until after we’d fixed the issue.

While we’re opening Precise’s translations, Launchpad will not be importing translation files and the web interface for making and reviewing translations will be unavailable. This includes imports for translation uploads, but also imports from Bazaar branches.

Once this is done, imports will resume normally and any backlog should be processed quickly after that.

Launchpad translations will be unavailable for around one hour, starting 10.00 UTC, on Tuesday 2011-11-22.

During this time Launchpad will not be importing translation files and the web interface for making and reviewing translations will be unavailable. This includes imports for translation uploads, but also imports from Bazaar branches.

We are suspending the service temporarily to allow us to set up translations for the next Ubuntu release, Precise Pangolin (to be 12.04 LTS). Once this is done, imports will resume normally and any backlog should be processed quickly after that.

If you use Launchpad’s Translations feature, then as of today, you may notice a new kind of error for uploads in your translations import queue. Look for the “” icon next to your upload’s status, and click to unfold. The error message says “Can’t auto-approve upload: it’s not clear what template it belongs to.”

You may find this annoying, and as the person responsible, let me apologize. I hope when I’m done explaining, it won’t seem so bad.

To the right is an example of the error. Click it to see more.

Where does the error come from? Actually it’s a problem that’s always been there, but instead of telling you about it, we Launchpad developers tried to hide the complexity from you and take care of it all ourselves. And we weren’t keeping up. Some of your uploaded translations would just sit in the queue forever, with status Needs Review, for no clear reason. All that’s changing now is that Launchpad will tell you when this happens, so that you can deal with it without waiting for us.

Why do some translations sit in the queue forever?

Every translation you make in Launchpad has to go into a specific template and language. Usually it’s obvious where you want your translation to go: when you translate in Launchpad’s web UI, you’re already on the page for a specific template and language. If you have upload privileges for the project and language, you can follow the upload link from that same translation page and again it’s obvious which template you’re translating and to what language. If you always upload the same file with exactly the same name and path, new uploads of that file go to the same place as before. But what if you upload a new file from the release-series page, or the translation comes in from a Bazaar branch or an automated build?

Then it’s up to a script we call the Translations Import Queue Gardener. The Gardener periodically scans all waiting uploads—the ones marked Needs Review—and tries more advanced ways of matching them up with known templates and languages. When it finds a match, it approves the upload’s import to the template and language it has found.

One of the Gardener’s most important tools is a template’s ”translation domain.” This is a simple name; no slashes or weird characters allowed. Launchpad figures out a template’s domain when you first import the template, based on its directory path. In principle a template’s domain should identify the template uniquely on the end-user’s system, but Launchpad isn’t as strict about it. It just assumes that the domain name is tied to the template file’s path within a project’s source tree. You probably shouldn’t, but you can give your project two templates with the same domain if you want.

When you upload a translation, the Gardener tries to figure out its translation domain based on the file’s path, just like what happened when the template was created. The Gardener looks for a template with the right domain in the same release series. If it finds one, presto: it’s got the template that the upload is meant for. If not, then the Gardener tries a few other things and if nothing works, simply keeps the upload on the queue.

So the entry doesn’t get imported, but usually the Gardener can’t give any single reason: all it knows is that it tried many ways of matching the file to a template and none of them worked. Maybe it’s an error; maybe it’s just a matter of waiting for the right template to be created.

So what’s changed?

The new error message is about “approval conflicts.” You’ll see it when there is ”more than one” matching template for your upload. This can happen because your project’s directory structure is unusual and Launchpad can’t extract a meaningful domain from it. Or a template’s domain may have been changed, or an old deactivated template is reactivated when a new one has already taken its place.

Whatever the cause, the new error message tells you that this has happened, and what the matching templates are. It’s up to you to decide what needs to be done about it:

update one of the templates’ domains, or

deactivate an obsolete template, or

move a file, or

re-upload your file to a specific template, or

re-upload your file with a different name, or

upload translations as a tarball so that Launchpad sees their full directory paths.

How much you can do, of course, depends on the permisisons you have. Only the project’s owners (and Launchpad’s administrators) can deactivate templates or change their domains. But you can always delete your own uploads if you want to upload your file differently: on the import-queue page, click on Needs Review and select Deleted instead.

Let us know

There’s still plenty more we’d like to do to make imports easier and more efficient, if we can find the time. But I hope this small change will make your life a little easier.

Is this error message working for you? Is it helpful? Are you seeing a lot of these errors, or none at all where you were expecting them? Is the explanation unclear? Do you see something happen for lots of people that we could fix in the same way? Please get in touch:

Tips

If you want to become more familiar with the translations import queue, check out the global queue to see all uploads in Launchpad. The version you see there is just a copy on a test server, so don’t be afraid to play with it.

By the way, did you know about our test servers? We test our changes on staging and qastaging servers before they go live. You can try out most of Launchpad’s features there. Look for the grey “demo” text in the background. They get restored to a fresh copy of the real Launchpad once a week.

Tired of creating translations tarballs and uploading them to your project? Automate it all with Bazaar integration.

You want Bazaar integration but your code is hosted outside Launchpad and/or in a different revision control system? You can tell Launchpad to mirror a branch from elsewhere. Then you can import translations from the mirrored copy.

We like transparency! If you’re interested in the engineering details of this change, it’s all online.

Until today, when you imported a template or translation file into Launchpad for the first time, you’d have to wait for a member of the Canonical Launchpad team to review and then approve that file before your project’s translation community could make use of it.

Now, if you’re a project maintainer, you can manage your project’s translations import queue yourself. All you need do is follow the “import queue” link on your project’s translations overview page and you’ll see something like this:

Once you’ve approved a file, and it has been imported, subsequent changes will go through Launchpad’s automatic approval process.

It’s incredible to think that more than 57,000 people have used Launchpad to translate software from English into their own language.

Many of them have worked directly on upstream projects, such as the OpenShot video editor. Others have helped translate Ubuntu packages of software. And then there’s a whole group of people who translate upstream software outside of Launchpad.

Today we’ve taken another step in bringing those efforts closer together by making it far easier to get upstream translations directly into Ubuntu.

We want the strings produced by translators working directly on software projects, whether in Launchpad or elsewhere, to be easily available to the Ubuntu translators and we believe it should be just as easy for software projects to take advantage of the work of Ubuntu translators.

How it works

Translation sharing between different releases of a project, or Ubuntu, has been available in Launchpad for some time now. Also, sharing translations between an upstream project translated in Launchpad and its Ubuntu package has been helping to bring those two communities of translators closer together.

What’s changed today is that strings from upstream projects who make their translations outside Launchpad are now just as easily imported and ready for use by Ubuntu.

Now, so long as the upstream project is set up in Launchpad to do this, a change made in an upstream project’s source code — whether hosted directly in Launchpad or elsewhere in Bazaar, Git, Subversion of CVS — will be available to Ubuntu translators just a few hours later.

Previously, Ubuntu took translation templates and files from the source packages as they were uploaded. There was no automated route for those new upstream translations to get into Ubuntu after that initial import. In effect, this allowed Ubuntu translations to diverge from upstream during the six month Ubuntu cycle.

This change has a nice side benefit of making it easier for upstream projects to make use of translation work done for Ubuntu, because the English strings will diverge far less and it will be easier to spot where the Ubuntu community has done new translation work, rather than their being a divergence due to the two efforts drifting apart.

To start with, this is available for projects that use intltools, which includes all of GNOME. To get your project’s translations automatically imported into Launchpad, see our help guide.

Even at the time, we knew that the story wasn’t complete: you still had to somehow generate your translation templates (in the form of GNU gettext’s .pot files) and get them into your Bazaar branch before people could start translating your project in Launchpad.

However, we also knew that automatically generating translation templates was a big task.

Now, though, I’m pleased to say that Launchpad can automatically generate the templates on your behalf.

How to get it all set up for your project?

Automatic translation template generation relies on something called intltool. You’ll need to be familiar with intltool before you can get started with automatic template generation.

You first need to enable your branches for intltool and then set up a translation template import fromn the Bazaar branch that is linked to your project’s release series.

This means that, provided your branch has proper structure, you don’t even have to keep the POT file committed anymore (as a matter of fact, it’s better if you don’t). If your branch is not recognized as intltool branch, everything will keep working as before.

At this time, limits to what branches we consider intltool based are pretty strict: it has to have a POTFILES.in file in each of the template subdirectories, and be able to derive the domain name from MakevarsDOMAIN variable or Makefile.in.in, configure.ac or configure.ingettext_PACKAGE variable (with very limited substitution supported). This will be further improved in the future, but plan is to support much more different layouts than just the intltool one.

We’ll be writing more about how to make the most of this in the coming weeks.

The last few months we’ve been doing a lot of work to enable direct import of translations from different upstream VCS systems. For now, we’ve focused on getting one very important case right first (GNOME), and then we’ll extend it to supporting other upstreams as well.

How are we going to do it? First off, we’ve split it all into two separate stages:

get upstream translations into Launchpad

push upstream translations from Launchpad into Ubuntu

For some upstreams, getting them into Launchpad is trivial: they might already be hosted in Launchpad. For majority of them, however, it means pulling from different VCS systems. Thanks to Launchpad Code and Bazaar teams, getting the code in the form of bazaar branch is not that big a deal. However, when pulling translations from a VCS instead of getting them from tarballs means one slight complication. Translation templates (POT files) won’t be there, and we’ll have to regenerate them.

Regenerating templates differs from project to project. And doing it should be considered an unsafe operation. So, in the first step we are only going to support intltool-based modules, and generation of templates will happen inside a sandboxed environment. This will enable us to import upstream translations directly into read-only Launchpad projects: this is marked with green-coloured arrows on the diagram.

After that is done, we’ll start pushing all these translations directly into Ubuntu (blue-coloured arrows), minimizing the time it takes for translations to get from upstream translators to Ubuntu users.