Yahoo! Pipes Wizardry for Drupal translation teams

The Hungarian Drupal interface translation team used to use a private Subversion repository to store translations. Our reason for that was that we initially had many people contributing, and it seemed to be difficult to apply for CVS accounts for each of them. It also happened that we had some of our own tools developed and used. Now there is not that many contributors and many of our tools are already migrated to Drupal.org (and the others can be migrated too), so we are moving to Drupal.org. This will most importantly be better for our users, so they can find Hungarian translations in the tarballs downloaded from Drupal.org, without browsing through our own translation repository.

The 'problem' with moving to Drupal.org is that it is quite hard to have an overview of what is happening with Hungarian translations. Although there is a Hungarian translation project, that only hosts the Drupal core files. Module and theme translations are under those module and theme projects. Although there is a move to have these module translations as their own projects (which would result in an explosion of projects there), until that is done, we would still need an overview of what is happening around Hungarian translations.

Yahoo! Pipes to the rescue! We need to watch CVS commit messages, but these commit messages are from all kinds of projects, so we need to watch the general CVS commit page. The patterns we need to watch for are translations/$language, /$language.po and .$language.po. Note, that the last two are delimited at the start so that substring matches are not possible.

I needed to formulate these into Yahoo! Pipes objects and publish the pipe. The language code needed to be a user specified value, so that any translation team can use this pipe. The search strings needed to be built dynamically as a result of this, and the CVS RSS feed was filtered with these patterns. The result is the Drupal translation commits for a given language code pipe.

Finally the only problem with this pipe is that this works 'live' on the given feed, and historical information is not kept. To have an overview of what happened in the past, I have added the resulting RSS feed to the aggregator at drupal.hu, which stores historical data of our commits. A possible problem here is that the refresh interval can only be set as short as 15 minutes, which could be too long, given the frequency of commits at drupal.org. If commits run out of the ten commits long window showed in the RSS feed, we don't see them in Yahoo! Pipes and as a result, we don't aggregate them at drupal.hu. Thankfully aggregator.module can be form_alter()-ed, so we can set a shorter interval if need be.

For now however, we monitor how our new pipe works, and encourage other translation teams to get an overview of their work this way (unless they know something better in which we would be interested too).