Drupal planet

Drupal 8 is in the beta releases stage getting closer to supporting an upgrade path. It is still in development and there are still some interface changes expected. As per the beta changes policy interface string are unfrozen and user experience improvements are prioritized. On the other hand, I don't know of major initiatives to change the interface strings and I believe most of the interface should be considered stable at this point.

As we are getting close to releasing beta versions with support for the upgrade path, that means more sites will start to use pre-release versions of Drupal 8. This helps find bugs and we would love the multilingual system to be (even more) thoroughly tested as well. We need translations for that to happen. We also need translators to look at source strings, so they submit issues found in them.

So while the user interface is not yet frozen, there are various good reasons to start translating Drupal 8 now. How fast should you plan to complete translations? Strings will likely be frozen with the first release candidate (which is cut when there are no more critical issues against Drupal 8). However that release candidate may become a release if no more critical issues found as early as a couple weeks after, so starting translation at RC1 would be too late to be ready for the release. You should consider starting sooner than later.

To make that process easier, we introduced a new Drupal 8 translation status page at https://localize.drupal.org/translate/drupal8 which lists the languages by completion status based on the last exported Drupal 8 translations available. Congratulations to the Spanish, Danish and Finnish team for being the top three! We also made it easy to access the multilingual demo from that page and included some advice on translation readyness as well as a step by step guide to new contributors.

All of the new Drupal 8 APIs should be supported now on the site including default shipped configuration. If you find something on the Drupal 8 user interface that is not translatable, submit a Translation template extractor issue and tag with "Drupal 8 compatibility".

One final tip to get started faster. Localize.drupal.org does not (yet) support matching of strings that slightly changed from Drupal 7, however it brings in exact translations of modules that were added to core, such as Views. To match slightly changed strings, you can do the following:

There is a huge amount of exciting things happening around Drupal multilingual at DrupalCon Amsterdam. This time we'll have a meetup of all the people who love localize.drupal.org. The site seriously needs people who care about it enough to devote time to maintaining and fixing bugs. I set up this BoF to gather people interesting in the well-being of the site titled We love localize.drupal.org. We need to upgrade to Drupal 7, support the whole range of new Drupal 8 APIs, drastically improve performance and then get new features going. See you there!

As we kept following the changes made to Drupal 8 and the dozens of new ways of adding translatable strings to code in the new version, we worked on support for TWIG translation constructs and all kinds of YAML file sources (routing titles, menu items, action links, local tasks, configuration schemas, etc) recently as well as some misc new APIs like TranslationWrapper objects. Huge thanks to the amazing work of @ksenzee, @herom and @hron84 building string extraction support for these in our not very well loved issue queue.

Support for these APIs is now rolled out live on localize.drupal.org and I sent Drupal 8 alpha12 to be parsed again with the new code. The result is the number of strings made available for translation jumped from 6820 to 7769, making available almost a thousand previously hidden strings. (Also almost 800 new files are considered now for source strings, jumping from 3972 to 4722). For comparison, Drupal 7's latest release only contains 4645 strings to translate. Our advice from last June that this may not be the time to jump on translating it all yet still stands though.

The quest is not over. API changes that affect translatability are still made. The latest one is the logger API that replaces watchdog(). We still need to figure out how to support that in string extraction. Help needed there! I'm not sure how the string extraction based method can sustain itself for Drupal 9, we'll need to take a hard look at this definitely. We are doing out best now in Drupal 8 to cover what is possible.

The largest outstanding item keeps being support for shipped configuration translatables. All the default user roles, filters, views, content types, menus and so on that Drupal 8 itself supports to be translated with sources from localize.drupal.org, so only the server side part on our side is missing still. There are probably hundreds of translatable strings hidden there still.

Drupal 8.0 alpha2 is the first tagged release of Drupal 8 that comes with a package downloadable from drupal.org. What does this mean for Drupal software translators? Well, it means it is made directly available on localize.drupal.org for translation. We make tagged releases available so that translators don't need to chase a moving target.

Looking at Drupal core releases on the translation site, Drupal 7 has 440+ files parsed for translatable strings and about 4600+ strings found there. In comparison, Drupal 8 has 5200+ files parsed (almost 12 times as many files) with 6700+ strings found (almost 1.5 times Drupal 7). So translating Drupal 8 will definitely be a bigger task compared to Drupal 7.

But! You should not jump into translating Drupal 8 yet if you don't want your changes to go unused in the final version. First of all many things are still in flux, texts may change dramatically, user interface cleanups will happen, and so on. Watch out for an announcement much later on text to be ready for translations.

What is also pretty important, all these translatables are just from the places where Drupal 8 uses the same API as Drupal 7. Drupal 8 for example removed st() and $t() in favor of using t() at more places, which does not change the situation for localize.drupal.org. But there are also API changes which made translatable strings move around places. Module info files are now info.yml, some hooks got converted to annotations. Default configuration that used to be saved into the database with a t() at install time are now shipped in .yml files in English (and not translated to the site install language in the original file by design). These are all great changes, but the translation server needs to adapt to them.

So far @ksenzee is the only person who signed up to work on making the backend of localize.drupal.org capable of handling the new APIs properly. There is a whole set of issues to make these happen at https://drupal.org/project/issues/search/potx?issue_tags=Drupal%208%20co... and we need more hands to help, test the patches and make this happen. I believe it would be a grand mistake to attempt to release Drupal 8 without these issues resolved, because only if these are resolved can translators ensure that Drupal 8.0 is fully translated in their language for release (even if there are already 1.5x as many strings, there will be even more). And given our enormous push for multilingual improvements in Drupal 8, we should support this with full translations as possible for release.

As part of working on the site upgrade, our messaging and notifications system is discussed. Unfortunately we don't have a reliable number on how many people rely on email notifications from localize.drupal.org (due to automated signups) and the complexity of the upgrade process for notifications (given that we need to migrate to new modules) might set back our upgrade process even longer.

While the Drupal Association is busy with updating drupal.org itself to Drupal 7, the DA is not directly involved with the localize.drupal.org site maintenance and upgrade (similar to other subsites), so we are not tied to the main site upgrade in any way (neither do we get financing for this work).

One possible shortcut is to sidestep email notifications for the immediate upgrade and get back to the feature later. Are you heavily relying on notifications from localize.drupal.org? Please speak up at http://drupal.org/node/1392694! Thanks!

Localize.drupal.org is becoming one of the least taken care of properties on drupal.org. Although Sebastien Corbin does a heroic job preparing the Drupal 7 site upgrade, that is still off in the future. There are existing bugs (in translation packaging, project parsing, performance, etc). Noticed that the frontpage stats have been disabled for performance reasons for months? Yeah. There are infrastructure tasks and Sebastien also needs people to join porting the site to Drupal 7.

With Drupal 8 planned to be relying even more on data from localize.drupal.org directly, the site will gain even more prominence. We can safely say though that pretty much nothing is going to change/be fixed on the site if people don't step up. Time for you to make a contribution!

To help people get started I'm holding a BoF at DrupalCon Munich titled "We love localize.drupal.org". If you do too, and want to help, come and discuss.

We are planning several changes for Drupal 8 to improve the user experience around translations of the Drupal software (including its modules, themes and distributions). We plan to build in the Localization update module into Drupal 8 core, and we are on track doing some of those steps. We also want to improve the general user experience of some software translation tools we have in Drupal, and your feedback on both counts would be very appreciated.

Improvements to the built-in software translation interface

We have fixed a long standing bug of idenification of singular/plural string pairs such as "1 minute"/"@count minutes". If you did not load in a .po file in Drupal 7 and before, the identification of these source strings never related them to each other, and translating them properly was impossible on the user interface. We fixed this bug by changing the storage for singular/plural pairs, and introduced a temporary complication to the built-in translation UI. We kept the search/overview screen as-is in Drupal 7 and before. See the third and second images at http://drupal.org/node/532512#comment-5641302.

We think that people go to the translation screen to actually translate things, and also that it is more likely that people want to translate multiple things into one language (which the current UI does not support at all) versus translating one thing to many languages (which the current UI is optimized for). So we have an ongoing discussion at http://drupal.org/node/1452188 with various UI idea to move forward. The most up to date list of options we considered is at http://drupal.org/node/1452188#comment-5777686

Because there are so many directions we can go, and we don't necessarily agree on the exact goals of the UI either, feedback from translators would be extremely valuable. If we cannot agree, the worst thing that can happen is that we'll release with the UI as-is now in Drupal 8, which I think is not ideal. Your feedback welcome!

Custom string translation tracking

We already centralized .po file lookup into one directory, so Drupal 8 will not look at various places under modules, etc. for .po files. This is very useful to support distribution and staging/deployment of translations without interference to version control of the source repositories. Adding in automated download and import of .po files from the community is still to be done.

However, because most community translations are not complete and we want to keep the possibility for people to customize translations downloaded from the community, we are building custom string tracking and customized string protection into the APIs and the UI. This is being discussed in http://drupal.org/node/1445004 with the main UI simplification / new UI additions explained in http://drupal.org/node/1445004#comment-5673836 and the following comments. I hope/believe that even though we are adding some new functionality here, we can also use this opportunity to not complicate the UI much at the same time and build in the automated translation feeding in a way that people can still customize and divert from as needed. Because the proposal does add some new concepts and UI components to the .po import/export screen to understand, it would be great if you could help provide feedback on whether the additions make sense and the simplification of the extended functionality helps make it easier to understand overall.

Future issues

If the changes proposed look exciting to you and you are interested in being involved, the Drupal 8 Multilingual Initiative is meeting every second Wednesday on IRC in the #drupal-i18n channel. Meeting announcements with exact times are posted on http://groups.drupal.org/drupal-initiatives. People from all experience levels and all angles are welcome!

Drupal 7 was released 13 months ago, and while Drupalcon sites (like Denver and Munich) are built on Drupal 7, none of the permanent Drupal.org sites are on Drupal 7 yet (or started to being ported to Drupal 7). This is usually due to (a) huge custom built modules or setups used that are hard to migrate (b) ongoing improvements to introduce better user functionality. With localize.drupal.org the situation is a bit more interesting. As I wrote in my last news post in October, Dashamir Hoxha mostly updated the huge base module for the site to Drupal 7 and Organic groups has a Drupal 7 version too, so the two biggest base modules are available. There are some interesting challenges however. The drupal.org theme (bluecheese) is not yet ported to Drupal 7 (help at http://drupal.org/node/1438716), the Organic Groups User Roles module functionality is integrated in Groups for Drupal 7 but there is no upgrade path and we'd like to switch to simpler solutions from the Messaging and Notifications monolith.

The update to Drupal 7 would be great to lead by example for other sites on drupal.org as well as keep improvements to the Localization Server module suite to one active branch. With the Drupal 7 port there and the Drupal 6 version used actively on the site, feature requests and bugfixes come in for Drupal 6 and there is a lot of extra work to port them to both versions at once.

Finally, I don't personally have time to head the effort but I can help provide guidance. Sebastien Corbin signed up to help do test upgrades and would very welcome any of your help. Want to work on updating the Drupal.org theme to Drupal 7? Figure ou an upgrade path for Organic Groups User roles to Drupal 7 Groups configuration? Help migrate Messaging and Notifications signups to a simpler framework? There are lots of opportunities to help out there. If you are interested see Sebastien's issue at http://drupal.org/node/1424984 and signal your interest there!

It has been a while since I've posted an update on localize.drupal.org services for various reasons. I greatly hoped that our Google Summer of Code project for activity tracking announced with big fanfare in May goes through well, but unfortunately my student dropped out from the program lacking time due to family reasons and did not make useful progress to roll out improvements.

Also, Dries selected me to lead an effort to dramatically improve language support in Drupal core for Drupal 8 and that has a scope way beyond just software translation. We did a pretty good job of providing tools for software translations so far with localize.drupal.org, l10n_update and l10n_client modules, but for Drupal 8, the focus is to bring complete core features and more usability to these and content and configuration translation alike. It is very exciting and going to bring Drupal to a whole new level where language support is needed! To follow that effort, see the video and mindmap at http://drupal.org/node/1260534. It is a huge undertaking and it required me to dramatically dial down my focus on localize.drupal.org improvements.

However, in the meantime, our misterious problems with some packages being partially parsed did not solve themselves, and we could not figure out the real reasons, so Zoltán Balogh got tired enough of the problem that he coded a workaround for every team manager to be able to request a release to reset itself and parsed again. That was just rolled out, so now every translation community manager should be able to see a Start over link on project release lists, with which you can request for a broken parsed release to be parsed again. Because our release parsing problems are transient, this should get you back a fully parsed release pretty soon afterwards.

There is also continued interest to set up new teams on the site, I've just added Oriya, Lao, Azerbaijani and Tamil (Sri Lanka) to the list of languages, which is now just one shy of a hundred (not counting our test language playground). That is an amazing number! Get ready to pop your champagnes! Who is going to be number 100?

Finally, Dashamir Hoxha did a stellar job to come in fresh to the Drupal community and update such a huge module as the Localization server module to Drupal 7! Yes! It passes all tests (although the test suite does not cover all of the functionality, it does cover all the basics and a sizable part of what localize.drupal.org needs). It would need more work, but we are a lot closer to be able to update localize.drupal.org to Drupal 7 in the future.

Now with all this while there are very nice prospects for localize.drupal.org, I will likely not be able to dedicate a good amount of time to the site, so I decided to post a public call for help instead, while I still have resources to help mentor people who would like to contribute and possibly eventually take over maintenance and improvement of the site. For a long time, localize.drupal.org was the posterchild of great improvements on drupal.org, now that all the awesome improvements are going on on drupal.org we should not leave localize.drupal.org behind either. Are you interested in working on gradual improvements? Porting the site to Drupal 7 (og, messaging and notifications and all that)? Comment and let's see how I can help you help Drupal!

Due to technical and scalability reasons the site uses custom code to provide translation of 7000 modules and about 20000 releases of these modules. Each release is parsed and ends up in the database of the site which results in almost 270000 strings to translate. That's a lot of stuff! Now add up that we have almost 100 languages and each string can have any number of suggestions for any language. Finally, each of these suggestions have their own "commit" history with data on who submitted them, when, via which method (multiple sources supported with a remote API in the mix), when was it approved, declined and approved again by whom and when, and so on. So you can consider this a 3 dimensional version control system in a database (version control per strings * languages * suggestions).

Now since this is all stored in this form, the activity happening is pretty "invisible". Moderators and fellow translators see little of this activity. Some teams use a wiki page to list things they work on, informing moderators that new stuff is available to review. Without that, reviewers can only look actively for stuff to review, which is tedious. Its a definite characteristic of communities online and offline that seeing the momentum, the activity and work being done energizes others and moves them to contribute more. So exposing all this information in natural and automated ways is crucial.

I'm happy to report that the immense task to expose all this activity with activity streams and graphs was taken on by a Hungarian student named Ádám Lippai, who is going to work on the project this summer. It should be an interesting mix of working on the drupal.org testing/staging environments, Hudson, our localization data storage and exposing the data changes in a meaningful way on the UI.

I'd like to thank both Ádám for signing up and the Google Summer of Code program for financing the project. This should really help take localize.drupal.org to another level.

Ps. Check out the other accepted projects at http://drupal.org/node/1139168 - note that we also have a project to work on activity exposure on drupal.org projects, so there'll be even more of this type of work on the site. Also, I'm still looking for people to work with on Gettext API for contrib and core (although no financing attached), as it should be part of efforts to improve multilingual support even further in Drupal 8.