Phased updates of software packages

Giving an entirely new version of any widely used software program to our entire user base all at once is unnecessarily fraught with peril.

Instead, let's employ a phased update strategy wherein we provide the updated software to an ever-expanding set of random users. This pool of users will be grown as our confidence of that software update grows, fed by realtime information from the crash database and other potential sources.

We should discuss the tradeoffs of doing this just post-release or during development, when we're also trying to land large changes to the operating system.

We should build this at a level that still affords power users the ability to forcibly install all updates. Update-manager is one such option.

It might be worth also discussing the possibility of using feature flags.

Related branches

Related bugs

Sprints

Whiteboard

For bug reports, we would obviously want the reporter to be updated to the newest versions of everything in case the bug has already been fixed. It may be worthwhile to have reliable bug reporters be updated in the first batch, both to get better bug reports and to avoid reports against older versions.

jdstrand> not sure how this will be implemented, but security updates in stable releases probably should not be phased by default. It might be worthwhile to have a way to flag a particular security update as phased however.

Work Items

Work items:
[cjwatson] Work with mpt on UI design: TODO
[cjwatson] Launchpad implementation of phased updates (bug 1100748): DONE
[ev] add alarm detection to the crash detection to see bad updates: TODO
[brian-murray] create and update a column family for release:package with per day counts of crashes to use in rate of crashes regression check: DONE
[brian-murray] create and update a BucketSystems column family to keep track of unique systems in a bucket: DONE
[brian-murray] create periodic check for rate of crashes increasing: DONE
[brian-murray] create API call to see if there is a regression about a package version: DONE
[brian-murray] Writing up what we're doing and why we're doing it: DONE