Maybe I scan titles too fast, but it took me a few moments to wrap my head around the fact that this issue is exclusively related to Drupal.org website instances/infra, and has nothing to do with any D8 core codebase issues. The ".org" part of the title just didn't register for some reason. Given that the title appears as a link anchor in lots of places (I got here via a related issue dup link) I figured it could not hurt to make this slightly more unmistakable.

Adding l.d.o Drupal 8 support issues as suggested by @catch, although one could say these should not only block release but even RCs as soon as we want to go out and say translators should go and translate Drupal 8. So these should be done even sooner.

I don't think its a blocker for any of the other blockers, we still keep building D8 supporting code on D6 now. I'm happy though its considered a release blocker, that drupal.org does not want to run an unsupported version of our own system. That's great.

Should give us a nicer way to view the issues, and see what queues they are in.
(we can also add "related" issues here without that needing to indicate they are blocking release. just the ones with the tag will block.)

David_RothsteinCreditAttribution: David_Rothstein as a volunteer commented 11 July 2015 at 16:06

8.0.0 release candidate blockers
.....
#1867192: Testbots need to run on php 5.3.10, 5.4, 5.5, 5.6 and 7
#697220: Run tests on all supported database servers/engines (unless we drop core support for both postgres and sqlite)

Although these are really great features, I'm not sure why they'd be release blockers... Their absence has never blocked releases before.

In any case, they look to me to have been mostly implemented already for Drupal 8, so can they be removed from the list either way? :)

Although these are really great features, I'm not sure why they'd be release blockers... Their absence has never blocked releases before.

They never blocked release before, but we've never had a release with working sqlite or postgresql support before either. So the idea was to release either with those working, or without them altogether, but not to keep pretending we support postgres and sqlite when we don't. This is all working now though which is fantastic, and sqlite/postgres should be 100% pass.

David_RothsteinCreditAttribution: David_Rothstein as a volunteer commented 22 July 2015 at 20:13

Supposedly both PostgreSQL and SQLite work OK on Drupal 7 (no critical issues filed, at any rate) - not sure if that was the case when Drupal 7 was released. But yes, it's really nice to be able to have the testbot run on those.

The impression I had was that it was only an issue for 8.0.x-dev snapshots or git checkouts, which feels like it should be solved by either git_deploy or l.d.o - I don't think the right translation fallback for dev snapshots is release blocking (and also don't think an extra release step is worth it as a workaround either).

If we have an issue for tagged releases then that obviously is release blocking but could use a reminder of the exact bug and what the core workaround would be.

@catch: It is true that beta releases started coming out with a version constant fitting the release with beta10 and later. This makes install_get_localization_release() kick in adding 8.0.0-beta12, 8.0.0-alpha1 and 7.0 as a candidates for the upcoming beta13 for example. So if there was a beta12 translation, it will be downloaded fine. This may be unpredictable if two releases come out in quick succession, as we have seen with some betas before and may see with minor releases later on. Then the fallback is to a much earlier version because we don't have info on what was the latest earlier version (the server side fallback would know). So if beta14 comes out within say a day or beta13 for whatever reason, l.d.o is not yet caught up with all the .po files, so translation downloads will then fall back to alpha1 until beta13 and beta14 translations become available. (Beta12 is not checked in this case because the list of fallbacks was designed to get to a somewhat useful file at least sooner than later to not slow down the installer with too many HTTP requests).

The same effect is going to be even stronger right when Drupal 8.0.0 is released, because until translation files become available, it will attempt to first do an HTTP request for 8.0.0's translation, not find it then the next fallback 8.0.0-rc1 will be looked for (even if RC1 was months earlier and if there were possibly multiple string changes due to critical bugs).

I am fine if we know about these problems and we don't consider them release blocking. I understood that both the multiple HTTP requests required in the installer (slowing it down) and that old translations will be used due to unknown prior versions were the reason #2113957: Build server side version fallback system for translations was made a blocker. We can remove it from the blockers if these two are not release blocking problems.

@Gabor so from #50 I'm still not sure what the workaround in core is - that we'd manually add 8.0.0-rc3 as a fallback for 8.0.0? Where does that go if so? Or are you saying the current state of core fallback is as good as it gets now, and it's a question of whether the l.d.o work should block release or not?

So if beta14 comes out within say a day or beta13 for whatever reason, l.d.o is not yet caught up with all the .po files

What's the reason for the lag? Is it literally a 24 hour cron job or something else?

l.d.o first needs to get to know about the release, that is parsed in from a dump from d.o

then l.d.o needs to grab the tar.gz and extract/parse it for all the source strings

then l.d.o needs to export the .po files for the strings found

All three need to happen in succession. The cronjob for the first two is the same and runs every 5 minutes. *However* the export for the first one is only refreshed daily or so on drupal.org. (I've only found the release history XML job in jenkins, which may be tied to the tsv we use for (1), but indirectly based on when l.d.o's jobs find new projects, it looks like daily). Then (3) also runs every 5 minutes. It gives precedence to projects based on usage (well, at least an outdated copy of usage from a year or so ago), so it will generate core stuff first and foremost if all goes well. So unless there are issues on l.d.o, the bottleneck looks more like d.o giving out new project info only daily.

As for knowing the last release on a lower level, that either needs to be hardcoded when release levels are bumped (alpha to beta, beta to rc, rc to release, 8.0.x to 8.1.x, 8.max.x to 9.0, etc) or a server side fallback needs to be provided. The good thing about the fallback at least is it only needs to be provided manually when a release level is jumped over, if we choose to handle it manually and not automate it.

We had a pow-wow today with most of the core committers (@xjm, @alexbronstein, @alexpott, @catch) as well as most of the DA tech team (@drumm, @basic, @isntall, @joshuami, @Mixologic, @hestenet, @japerry... hope I didn't forget anyone!) to go through this list.

Seeing #2113957: Build server side version fallback system for translations done looks good, however it does not help until #2113955: Rely on proper server side version fallback for translations is also done (or Drupal core is manually updated with last version numbers). While it does not block RC1, it would be sad to get it done as late as 8.0.0 if we want to get translators test with the latest translations using Drupal dev versions and not require them to use concrete tagged betas/RCs. If we are fine requiring translators to use tagged releases at all times to test their translations and/or manually keep updating last versions in core PHP files, then this does not block any release whatsoever.

webchickCreditAttribution: webchick at Acquia commented 28 August 2015 at 17:14

My layman's, not-a-translator opinion is that testing against latest dev will become important as Drupal 8.0.0 gets closer and closer to release, but that for the first little while during RC there'll be more than enough new/changed strings in D8 to keep translators busy, so tagged releases only is fine for the first few.

There was also some concern from @catch that without one/both? of those fixes, translators would get beta1 for the first 24 hours of rc1's release? Any way to mitigate that, such as manually returning "8.0.0-rc1" from api.drupal.org/api/function/install_get_localization_release/8 in the tagged release?

Basically, I think the bottom line: what of this must block RC1 (which I believe means: prevents translators from starting their D8 translations) vs. what of this can wait until after RC1 but must be done before 8.0.0 (which I believe means: prevents translators from finishing their D8 translations)?

webchickCreditAttribution: webchick at Acquia commented 4 September 2015 at 17:49

I was still confused from that comment, so asked Gábor to tell me in very small words what is still needed here. :)

Basically:

- All of this revolves around the "little function that could," function install_get_localization_release(). This is the function that returns which Drupal version's translations to pull in when downloading translations during installation.

- For tagged releases (beta15, rc1, etc.) this function does what it's intended to do. There is a problem where .po files on localize.drupal.org lag about ~48 hours behind the creation of the release on Drupal.org. However, this is an existing problem, doesn't block RC1. (It could use some investigation by DA staff, however; summary: d.o generates a dump of their new releases every day. l.d.o tries to in fact parse it every fine minutes (LOL) but its only updated once daily on d.o.)