As the Fedora 18 schedules slipped by another one week, current week before beta freeze is Oct 2. This means this should be discusses on on Oct 3 meeting. Also I'd like to extend this ticket not only to the upgrade functionality but to installer's beta release criteria. This way we could avoid long freeze in case Anaconda is not ready by Beta freeze. I'll talk to QA/Anaconda to provide composes with latest builds to review if the criteria are met.

"From wwoods: So the code is currently here: https://github.com/wgwoods/fedup. Plan for F-18 Beta is to have an F-17 based upgrade tool that fetches packages or sets up an upgrade using a local DVD/USB image.

It might not have any real GUI for beta, but it will for final.

It will involve a special upgrade image, but it's just a dracut-built initramfs. This could be built by lorax, or it could just get built by running 'dracut' inside a mock chroot. This might require rel-eng involvement.

I might need some assistance with: i18n stuff, GUI polish, a special plymouht theme for the upgrade, and network monitoring (e.g. running sshd in dracut, like we do for s390 cmdline installs)."

We discussed this topic today at the Fedora QA meeting: we may revisit it next week also.

We agreed to some revisions to the Fedora 18 release criteria which are of obvious interest on this topic. The criteria relating to partitioning that will be enforced for Beta are these:

"The installer must be able to complete an installation using automatic partitioning to any sufficiently large target disk, whether unformatted, empty, or containing any kind of existing data"

"The installer must allow the user to select which of the disks connected to the system will be affected by the installation process. Disks not selected as installation targets must not be affected by the installation process in any way"

"The installer must be able to complete an installation using automatic partitioning to a validly-formatted disk with sufficient empty space, using the empty space and installing a bootloader but leaving the pre-existing partitions and data untouched"

"The installer's custom partitioning mode must be capable of the following:
1. Creating, destroying and assigning mount points to partitions of any specified size using most commonly-used filesystem types
2. Creating encrypted partitions
3. Rejecting obviously invalid operations without crashing"

We also agreed on a revised Beta criterion for upgrading:

"It must be possible to successfully complete an upgrade from a fully updated installation of the previous stable Fedora release with the 'minimal' package set or the package set for a release-blocking desktop, using any officially recommended upgrade mechanism. The upgraded system must meet all release criteria."

Note the use of the word 'or' there is ambiguous and we will improve it: the intent is that it must be possible to upgrade to Fedora 18 Beta from all three of those Fedora 17 package sets, not just from any one of them. In practice this criterion is not really a change from the status quo prior to F18, it still means 'upgrade has to work at Beta' - it's just better wording for a world in which the upgrade process isn't necessarily part of anaconda.

So with those criteria - plus the other Beta criteria, which so far we believe do not require modification - established as the 'ground rules', our assessment is that Fedora 18 as it stands right now is not freeze-able for Beta. Our understanding is that the agreement is that we will not freeze until the basic code relating to all Beta release requirements is in place. The custom partitioning code in 18.10, the latest available anaconda build, does not have code to meet all the requirements above - it does not include 'autopart-into-empty-space' - and the new upgrade tool has not yet seen a testable release (or any release).

However, the anaconda team affirmed at the meeting that they believe the required code will be in place by the freeze date, 10-09. If the upgrade tool is in a testable state and the partitioning code covers the requirements stated above by 10-09, we would consider that to be good enough for the freeze to occur.

"From wwoods: So the code is currently here: https://github.com/wgwoods/fedup. Plan for F-18 Beta is to have an F-17 based upgrade tool that fetches packages or sets up an upgrade using a local DVD/USB image.

"The installer's custom partitioning mode must be capable of the following:
1. Creating, destroying and assigning mount points to partitions of any specified size using most commonly-used filesystem types

To clarify, does this require that it must be possible to assign a mount point to a partition on RAID/LVM, or to an encrypted partition or not? If it does require RAID/LVM, does it require arbitrary configurations of the storage stack or a limited subset?

Adjust summary per jwb request. We would like this ticket to be a general review of whether the Fedora 18 codebase is ready to be frozen for Beta, i.e., is code to cover all major F18 Beta requirements in place at the freeze date.

As well as the new upgrade tool, the other currently known area of concern is partitioning. anaconda 18.11 (latest available version at time of comment) does not contain code to cover the Beta partitioning requirements. Other areas of concern may also be discovered.

agreed short special meeting on Monday (2012-10-08) at 17utc. Those that cannot attend will vote in the ticket.

The current status: fedup (in testable state) will be ready by the end of the week partitioning
* raid code is in the current build, in testable state
* luks code under patch review (posted on Thursday)
* [http://qa.fedoraproject.org/blockerbugs/current Current blocker/proposed blocker bugs list]

So, it's a week later, and the upgrade tool still is not available for testing. Additionally, RH has asked its staff on the anaconda team to prioritize work on a pending RHEL release over work on Fedora 18, and that is happening, which may further delay work on the upgrade tool and the current blocker list. Note the blocker tracking app - http://qa.fedoraproject.org/blockerbugs/current - may soon go down for an upgrade; if it's down, you can check Tim's personal dev instance at http://supermegawaffle.com/blockerbugs-devel/ to get the blocker list.

As discussed at this morning's QA meeting, QA still cannot recommend we freeze for Beta with the upgrade tool incomplete and no firm ETA. It's worth noting that, as the upgrade tool would be packaged for Fedora 17 for upgrade to Fedora 18, it does not necessarily technically affect the F18 Beta spin process. However its smooth functioning may require changes to F18 components (e.g. dracut). We would also be very hesitant to freeze for Beta - and, perhaps, eventually ship it - with the upgrade story 'we're writing a brand new upgrade tool which will be the sole recommended upgrade method for F18 Final, but we haven't finished it yet and we don't know when we will. We're hoping it'll show up some time between now and Final, and work. We'll keep you posted. In the meantime, use yum, which we always tell you not to use'.

That would suck.

Aside from the upgrade tool, I'm not aware of any ways in which Beta is significantly code-incomplete at this point, the partitioning code is mostly there, although there are still some big bugs. There are five unfixed confirmed blockers on the list (we have to have all confirmed blockers fixed before we start spinning RCs, by policy) and nine unfixed proposed blockers which we need more data to evaluate and confirm or reject as blockers.

Current fedup status (based on wwoods input): it is still not yet in testable state.

What works: Fetching packages from network working
* GUI is in-progress Media detection works (DVD/USB/etc) Messing with bootloader works Hooks for third-party upgrade "plugins" present
* Migration stuff can be handled by packagers Running the upgrade transaction works Plymouth progress working

Current fedup status: it should be ready for QA to look on it known issues:
* systemd issues are worked on, Lennart knows about the needed patch + discussion about better upgrade support ongoing
* plymouth issue, not a blocker for beta
* logging to journal does not work, not a blocker
* lorax patch needed in case of no surprises, Will is ready to start packaging/test documentation

Well, fedup should be in testable state (not internal only this week), with luck also packaged and ready for Beta...

I have a middle road idea. I know we do slips in 1 week increments, but in this case how about we instead do:

As soon as the upgrade tool is packaged and available for testing, we freeze. So, if that happens tomorrow, we freeze tomorrow. If it happens friday, we freeze friday. Then we just jump on our regular freeze schedule after that (go/no-gos on thursdays, release on tuesdays, etc). This may give us a bit more non freeze time than simply slipping a week.

QA discussed this at the meeting this morning. We still can't really recommend freezing with the tool not yet available for testing (jreznik says Will says it's testable, but we haven't been given anything). There are two days before the freeze date, but as things stand, QA still doesn't recommend freezing. However, we note that we considered the issue strictly on the QA merits, and FESCo may take a wider view and decide to freeze. If FESCo does vote to freeze, we would like to see a definite plan for how we can ship Beta even without the upgrade tool being fully ready.

If we receive the upgrade tool for testing by Wednesday and it passes smoke testing, I will update the ticket again.

The schedule should be taken into consideration, we are getting close to the Christmas - see [https://fedorahosted.org/fesco/ticket/960 ticket 960] - with Red Hat being closed... Kevin's proposal can give us a some time, to avoid one week long slips (as we can freeze once the feature is in testable state). But we still need a backup plan for Beta as we can't wait forever, we can consider alternative upgrade methods to fedup - yum (even we discourage users from using it), preupgrade (if still possible) or relax Beta release criteria in some way.

I have a middle road idea. I know we do slips in 1 week increments, but in this case how about we instead do:

As soon as the upgrade tool is packaged and available for testing, we freeze. So, if that happens tomorrow, we freeze tomorrow. If it happens friday, we freeze friday. Then we just jump on our regular freeze schedule after that (go/no-gos on thursdays, release on tuesdays, etc). This may give us a bit more non freeze time than simply slipping a week.

Thoughts?

I also don't want to preemptively freeze, and I think this is a good proposal.

At the moment, I have no thoughts on a backup plan in regards to the schedule.

One of the problems we had in the past was confusion about when we were frozen, when it wasn't tied to a particular date. It also leaves the FESCo meeting where we review feature completion (as an example) up in the air.

So while we could do it as a one-off for this feature, I'm not really sure it's the best idea.

"But we still need a backup plan for Beta as we can't wait forever, we can consider alternative upgrade methods to fedup - yum (even we discourage users from using it), preupgrade (if still possible) or relax Beta release criteria in some way."

preupgrade is not still possible. preupgrade relied on the upgrade code in anaconda - all preupgrade really did was download all the packages, put them in a directory, download the anaconda kernel/initramfs, write a kickstart which said 'do an upgrade install using the packages in this directory', and write a bootloader entry which ran the installer kernel/initramfs with that kickstart. (simple, eh?) since The New Anaconda has no upgrade code, there is no way preupgrade can possibly work any more, unless someone puts upgrade code back into the installer.

really, I think yum is the only available 'fallback plan'. I'm not personally terribly happy with that idea as it would really screw up our messaging around upgrades - "so wait, Fedora has a new upgrade tool called yum? Oh, that's not it? But I have to use yum now? But I shouldn't ever use yum again? What should I use instead? preupgrade?", oh the hilarity - but it is at least viable. And in purely practical terms, F17 to F18 happens to be a bump which yum handles pretty well, we don't have any crazy bootloader changes or /usr move stuff in F18.

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

I agree with jreznik. If the fedup is in anaconda before release of F-18, then it's fine with me.

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

I agree with jreznik. If the fedup is in anaconda before release of F-18, then it's fine with me.

Did you mean "in F17" ? Fedup isn't going to be in anaconda, ever. That's the entire point of this whole exercise of removing upgrade functionality from anaconda.

Well, we need some kind of decision - do FESCo wants to vote on Kevin's proposal? As it's the only I found here or just vote on freeze/not freeze + if freeze, how to proceed with upgrades (see my comments with a few ideas, forget preupgrade ;-).

-1 to Kevin's proposal. beta is supposed to be 100% feature complete, and looking at comment 25, that's apparently not the case.

I think we should decide whether we cut the feature scope (= what kinds of upgrades are supported), or slip; not go ahead with a known incomplete beta and risk having to cut the scope for final.

I didn't suggest that. I simply said we don't freeze today (since we are not 100% feature complete), but if we are tomorrow, we do so then instead of waiting until next tuesday to decide to freeze. I realize that could confuse folks, but I think it buys us a partial week slip hopefully so it's worth it.

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

Well that really depends on when is "shortly after beta" ... the whole point of requiring this by beta time is to have sufficient time for testing it. Given the recent track record "shortly after beta" could be "a few days before GA". I still think dismissing #944 as "insane" was a mistake but well it is to late for that.

-1 to Kevin's proposal. beta is supposed to be 100% feature complete, and looking at comment 25, that's apparently not the case.

I think we should decide whether we cut the feature scope (= what kinds of upgrades are supported), or slip; not go ahead with a known incomplete beta and risk having to cut the scope for final.

I didn't suggest that. I simply said we don't freeze today (since we are not 100% feature complete), but if we are tomorrow, we do so then instead of waiting until next tuesday to decide to freeze. I realize that could confuse folks, but I think it buys us a partial week slip hopefully so it's worth it.

(Unless I am mistaken, the proposal has +5 already, though.)

I think so, it's hard to tell. ;)

I agreed with nirik's "As soon as the upgrade tool is packaged and available for testing, we freeze." I didn't say let's freeze no matter what. We should really stated proposals better.

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

Well that really depends on when is "shortly after beta" ... the whole point of requiring this by beta time is to have sufficient time for testing it. Given the recent track record "shortly after beta" could be "a few days before GA". I still think dismissing #944 as "insane" was a mistake but well it is to late for that.

It wouldn't help to use old anaconda. I asked before the decision about #944. Old anaconda would also have bugs, because it wouldn't work with F-18 without lot of fixes and we would end up with two broken versions.

For the future FESCo must review contingency plan better and follow on important features more closely. Especially, if there is not any contingency plan.

Well, as I don't see freeze notification, I expect the result of this ticket was to go with Kevin's proposal (and mitr pointed it was already +5). Just checking, it would be great to communicate this - I can take care.

Yeah. So, we are waiting for the word from wwoods. As soon as fedup is packaged and ready for testing, we should freeze. If that doesn't happen before next monday, we should re-evaluate again I suppose, but I'm really hoping for somtime in the next few days. ;)