Search

A story about updates and people

A bit ofdiscussionabout update policy in Fedora has been brewing lately and I’ve been reading and thinking (and stewing and moaning and wringing my hands) about the discussion a lot. I noticed a few archetypes in the positions taken throughout the discussions:

Caroline Casual-User

Why should we care about what Caroline has to say? Well first of all, she isn’t involved directly in any of these discussions, so she has the least voice in all of this and to some point might not even be able to follow the discussion (“What’s a koji? What’s a FESCo? Who you callin’ YUM?”)

We should also care about Caroline if we want our userbase to continue to grow, and if we want to better fulfill our mission which is in part “to spread free/libre code and content.” I don’t think you can really spread anything without reaching beyond where it already is. Caroline has just as much right as the rest of us to enjoy software freedom, and she may even help spread it to more new users than we could reach alone if we can only capture her interest and inspire her.

Pamela Packager

Why should we care about what Pamela has to say? She’s pitching in here. She’s devoted a great chunk of her time (in many cases, personal & volunteer time) to help out, and she cares very deeply about the mission and soul of the project. The technology excites her, a lot more than it does Caroline, and she can help push more innovation and expand our capabilities if given the chance. If we lose all of our Pamelas, we lose our capacity to get keep things humming along.

Connie Community

Why should we care about what Connie has to say? She’s a community gardener. She’s making sure we’re open and transparent and enable new contributors to enter the project and she notifies us of issues as they happen, helping us course-correct. She helps recruit new contributors and make sure the environment for contributors is a fun and welcoming one.

Nancy Ninja

Why should we care about what Nancy has to say? She may have been around Fedora for a very long time, from it’s pre-beginnings through all its growing pains up until now. She feels very close to it, and has for a long time, and is afraid of change and that the project is shifting to cater only to a user base that doesn’t include her and feel unwelcome. She doesn’t want to lose the OS she’s relied on all these years.

Where you stand depends on where you sit

This is something my colleague Adam Young told me this morning. It kind of resonates for me here – “Where you stand depends on where you sit.” I think at some level in the various discussions about Fedora updates, Caroline, Pamela, Connie, and Nancy are making assumptions and taking away completely interpretations of the same idea, and communication is breaking down, because they come from such different positions. Computing I think is really the world of the abstract, and because so many things like, ‘packages,’ ‘updates,’ ‘repositories’, ‘environments’, don’t really have real-world objects bound to them, folks end up using the same words to describe completely different things a lot.

Pamela wants updates to be constant throughout a release, no holds barred – she wants the latest Gimp and she wants it yesterday. Caroline just wants her computer to work – “please don’t change a thing – it worked yesterday – if it breaks before my presentation I’m screwed!” Can both their needs be met? I think so! But it’s easy to completely miss where interests and needs can both be met when the language is so easily interpreted to mean the problem is untenable. Let me give you an example of how I think both Pamela’s and Caroline’s needs could be met here:

Here, if Caroline runs stable (as she should be with a default install of Fedora), she might notice an update to the core system once a month or so, maybe occasional additional security or critical bugfix updates now and then. Not two hours’ worth of updates on a fresh F13 that took 15 minutes to install a month after release…. ahem. Pamela wants her newest Gimp and she wants it now – well, she’s got options. She can hook her system up to testing, and if the Gimp she so craves is not there, she can enable rawhide real quick to grab it, or look for a koper that’s got the latest and greatest Gimp – maybe a pre-release development version.

“Well, wait a minute!” you cry. “Once-a-month updates???” No, not exactly. Let’s zoom in a bit on that stable package graph, okay?

Why should update policies for the kernel, dbus, firefox, inkscape, xorg-x11-server, and cowsay be the same? Does that make sense? If an update breaks my graphics, I can’t use anything. If an update breaks cowsay, well… my clever MOTD is a little less clever but it shouldn’t break other apps. So why not bundle critical stuff that’ll really hurt our users in a huge way – the basics like networking, graphical display, hardware support, i18n input methods, sound – and put much more stringent guidelines on them than apps like figlet, xbiff, or xbill? If an application is relatively self-contained and can really only break itself – is it so necessary to be as strict about updates to it within a stable release?

I thought this game was the bee’s knees when I was a junior in high school. Take that, The Man!

Then we have a base platform that’s a bit more stable, letting Caroline live without fear, all without stifling gotta-have-the-latest-apps hunger of folks like Pamela. Hmmm. Stable base platform. That might make life easier for 3rd party developers such as Boxee, Amazon, and Adobe to provide support for Fedora, no?

Bonus

If we had a concept of a base platform treated separately from applications…. for Caroline I think PackageKit would go from this:

To this:

About this blog post

I’m a designer, I’m not an engineer. I may be missing something insanely basic and be sitting in the midst of y’all, Dr. Evil-like with a big puff of Cappucino foam on my nose here.

That’s okay!

Maybe I should have posted this to a mailing list. Well, I hate mailing lists, and I wanted this to be visual. If anything I hope you walk away from this post thinking that maybe sketching out some diagrams or working through some (basic, not fancy-pants) personas might help solve the problem in a more productive way than the flamewars that inevitably break out whenever you try to do something productive on mailing lists. (I wish we had something like this today, so bad.)

Actually even more importantly than that, I want you to walk away with the idea that this is a solvable problem and doesn’t have to disintegrate into bad feelings and flames. Even better, start sketching out your own ideas and possible solutions, feeling free to mark up my little sketches if desired. The Inkscape sources for my diagrams is available. If you’ve got an idea you can describe to me that is indeed awesome and might help solve some of this mess, I’d be happy to help you put together some graphics to help communicate your idea more effectively.

So what do you think? I’m going to try to keep my bar for the comments on this one really high, so I’m going to be really strict in moderating. You’ve been warned :)

Rate this:

Share this:

Like this:

LikeLoading...

Related

About Máirín Duffy

Máirín is a principal interaction designer at Red Hat. She is passionate about software freedom and free & open source tools, particularly in the creative domain: her favorite application is Inkscape. You can read more from Máirín on her blog at blog.linuxgrrl.com.

Discussion

109 thoughts on “A story about updates and people”

People will not want too many updates, at the same time they WILL want the updates to their favourite applications.

The problem here is one that there is little distinction between the OS and the applications as there is in the Windows world.

People will want their OS to be stable but their most used apps to be current.

I envisage a scenario where Fedora 14 will be using Firefox 3.6 even though Firefox 4 may be released at sort of the same time. Many even casual users will not think it acceptable to wait for 6 months to get the newly released Firefox, but they will not have much choice if they want their base OS to remain stable.

Okay, then. Completely agreed if you’re reiterating my points on why different types of packages should be treated differently. I don’t understand the simplicity point you made, first sentence though. I’m not sure which part you’re saying isn’t that simple?

That seems reasonable on paper, and I for one would love something among those lines.

You could expect that what constitutes your “base platform” would be either well disciplined (i.e providing stable releases containing only bug and security fixes rather than bundling them with tons of new features), or that their package maintainers would have enough skills to backport the appropriate patches and thus keep the “base platform” with a sane update flow.

The same is already less true for the “core desktop” packages. Take the kdelibs for example, if I understood correctly what Kevin Kofler keeps saying on the mailing-list, KDE upstream releases a new stable version and abandons the previous one, and their release schedule is not synced with Fedora. Asking the packagers to backport only bugs and security fixes would put a *huge* burden on their shoulders (think about the number of patches to merge and maintain, the likelyhood of introducing downstream bugs,…).

Do we ask KDE upstream to sync their schedule with us? (and thus with at least Gnome and Ubuntu with which we are already synced in practice)

Do we say “okay, KDE is not part of the core desktop and they are ‘applications’ and they should do whatever they want”?

(of course, feel free to replace KDE with any other piece of software, I’m not picking particularly on KDE and saying they behave poorly or anything, this is just the first example that came to my mind that could be a problem with what is suggested here)

Even in the applications we could have a problem. What if the new release of Foobar requires a new GTK? GTK is in the “core desktop”, so we won’t update it. Do we block the new Foobar? Do we patch it to work with the older GTK?

If Foobar is in fact Firefox, we might not be able to do the latter since that might (note the uncertainty implied by the words I used) be an infringement of their trademark usage requirements, so we would have to block it. How do we explain users that we update all applications in Fedora, except for Firefox?

Sure, Pamela could provide it in a Koper, as you explained. But then she would also have to provide the newer GTK release, and with it rebuilds of all of the Fedora packages than depend on it. That would imply:
– a *lot* of work for Pamela
– the need for a *lot* of storage in Pamela’s Koper
– lots of bug reports in the Fedora BTS against Pamela’s packages, and thus lots of noise for the regular packagers of the software that Pamela rebuilt
– potentialsecurity issues for the users of Pamela’s Koper as she will probably not be able to provide updates in a timely manner (see the first point above)

So in practice, I’m not sure your Koper proposal would be as efficient as you think. :(

However, the issues I mention need to be weighed against the issues we have right now, and I think I would be happy to wait in a few cases with a good reason rather than saying “yes, I just installed Fedora on your computer Caroline, but you can’t use it yet, you still have to download and install 2 hours worth of updates as soon as you get back home” at each release party.

“The problem here is one that there is little distinction between the OS and the applications as there is in the Windows world.”

It is not the problem. People do not know that the Linux kernel is the operating system and all other are just non-OS software. Most people have wrong conclusions that Linux is a kernel like it would be a microkernel. They do not know that the monolithic kernel is the whole operating system.

That brings lots of problems for normal users. It might be confusing first time. But in long run, when normal user does not know what is what, they can not solve problems or even get help as they have harder time to explain things to those who know the technology.

The situation right now with Linux kernel is that it is taunted that human body is a heart. Most people will not know that heart is just one big muscle in your chest bumping blood. Normal people does not see the heart so they believe the easy explanation of anything.
But what then when there comes a problem with the real heart and you need to explain it? What then when you need to explain other things related to heart, like how oxygen or even poison from animals bite is transferred trough your body?

Same thing is here. Even the average user needs to find easily all the technical information from updates. They need to have possibility to choose what they install and have a change to learn in time. It is not wise enough that updates are merged to one or they are shown as one. That just cause the problems as what marketing has done for the OS (People does not know the Linux kernel is the OS, but believe that OS is Linux kernel + something) that makes smaller group of the real expertets to work and bigger group of people who just spread believes and not facts. Disinformation and misbelieves are the problem, not the correct technical information.

If we would like to make things very easy to normal users. Lets just show them a ballot “Install updates, estimated download time 15 minutes (download size 59Mb)”. Then a checkbox what would allow to see all the packages and so on.

If we try to protect normal people from technological information, we make them dumb and that is not good for them at all. Same thing is with rising kids, if we overprotect them from possible accidents and so on, they will never learn. They are the future and if they can not develop basic knowledge of world and their own capabilities, they will not ever evolve better people.

We do not go to school just to learn that 1+1=2 or that dolphin is not a fish. We go there to learn basics what we can later use to understand more complex stuff. At some point almost every kid thinks that some talents and informations are useless for them in their life. But later they notice that they made mistakes by not learning the hard thing in the first place and later it is almost impossible.

And there is always as well the government problem what comes if we limit the amount of information to people or we do not tell the truth. When normal people talks with the mouth of the marketing or what they have heard from other people who does not know the stuff, they became easily controlled by those who has the real information. As the information is the power. Truth and information are those what keeps all us free from others controls.

It might sound small thing to have a update list with one package instead multiple. But it is the first stone what is thrown in the glass house and if not then it is not warned to do again, it will happen again until the glass house is gone.

Even that how nice toughs and ideas it is to make things simpler for normal users, there are just many things what must be learn at some point. And faster and easier the learning comes, faster the whole use experience comes easy.

Fedora has stable, testing, and rawhide (the equiv of unstable I think) right now. I think perhaps maybe they aren’t run the same way or maybe as disciplined as Debian’s though. Are you a Debian developer? Can you provide some comparative insight on how the two distros treat these repos differently in terms of update policy?

I wouldn’t say that Fedora has anything comparable to Debian’s stable; a tree which only receives security updates. In my experience, the main Fedora tree is closest to testing, maybe a little bit more unstable.

With Debian’s testing, its stability varies over the interim release period. After a release, it’s a flurry of activity as new packages are added and major changes are made for the upcoming release. Over time, it stabilises and eventually freezes, as at present. In its frozen state, it’s reasonable to upgrade to it from the previous stable release on non-production systems, in expectation of the upcoming new release.

To me, the biggest problem with Fedora is a lack of an equivalent stable repository, probably because I’ve used Debian for so long. If I installed a release version, I don’t expect things to break with updates. This is close to the ideals of the casual user stereotype, which I guess is probably far more common than the other three. The term ‘casual’ is a bit of a misnomer; we’re basically talking about anyone who wants to use a computer other than to work on Fedora, including everything from someone surfing the web to developers.

Of course, a stable repository is not a panacea. As already highlighted, one of the problems is that everything is then stable from the kernel through to Firefox. I think segmenting stable into various types of packages and handling them differently, as described in the blog, might be the start of a solution to this problem.

My only negative comment would be on the ninja stereotype. It seems overly negative and to denote a cracker (‘I’m reading your email right now’) rather than a savvy user.

The “Bonus” part seems to be a sorely needed feature for distro’s across the board. I wish everyone implemented it. I tried running Fedora on my Desktop at home, for my sister, and she was confused by the huge list of updates. I think users like her ( the Caroline category ) will be far more comfortable with updates split into “Platform” and “Applications” sections

I agree, mostly, although the “Bonus” part scared me a little (it’s OK to make things simpler by default, but please… I’d like to know what’s being updated anyway, the “lots of important things” it’s not enough).

But overall it’s a good idea. The point is, how do you implement this?

Reminds me on what the Chakra project is trying to do on Archlinux. Switching from completely rolling release to a stable system with a rolling release desktop/application on top.

I really like the “Platform Update #1″, but I would love to have more details what changed, at least optional. I mean 99% of the people updating their system don’t really care what is updated, as they install ALL updates anyway. So why not a rollup package?

I know Caroline and I grew tired of her, she usually ask only about playing mp3 files and Flash player crashes. I honestly don’t think Fedora is the best distro for her and would consider making her a Fedora user only if she was part of my family or an employee at a company where I am the sysadmin. Remember when we were targeting the people willing to give something back? She is not one of those so I don’t really care about her.

Pamela and me have so many things in common that I want us to hang around. I hope she’s coming to FUDCon so we can have a beer together. This is the traditional core of our audience, I care primarily about her as an user, as she is part of the ones that will make Fedora better, so somehow improving my life too.

I know Connie too, she is very idealistic (too idealistic?) that she may become counter-productive at times. But generally she is useful to have around. Still she needs to grow a pair and understand you have to make some decision and can’t make absolutely everyone happy.

I know some Nancy too but I suspect she has the servers split, some on CentOS and some on old, out-of-support Fedora releases (I won’t be surprised to see her with a FC6 server somewhere). And I won’t be surprised to see her making some contributions. I think she will be up for some beers at our next LUG meeting in a couple of days.

When Pamela will install the latest GIMP from Rawhide she may encounter a Python or gtk+ dependency[*], virtually ending with her *entire* desktop moved to Rawhide. Is a much realistic scenario for her to build her own package, then maybe let in on Koji for others to take benefit. Or just forget about stables and hook directly to Rawhide.

[*] I believe Inkscape 0.48 from F14 needs Python 2.7.x so I didn’t dare… will move to Alpha soon at my own pace.

As a systems architect, I see the whole “shared object
dependency” thing as a terrible failure in design modularity.

With that “virtually ending with her *entire* desktop moved to Rawhide”, how much else was broken? Terrible architecture.

And I have to support userland environmental modeling apps
across a bunch of servers running a wide variety of distributions — and I don’t have “root” on any of them.
When the ideology is that “static libraries are forbidden”
at the distribution-level, my job is made far harder: I
must plan to “compile the entire universe” from source.

Thanks for this and your post blog-posts! I think you are right. Actually even if I am a GNOME Developer it is super annoying for me to fix stuff on my Desktop. I expect it to just work besides the stuff am actually hacking on.

I would love if Fedora had a policy like this:
* install security updates in background
* only install updates to the base system if *really* necessary
* Prompt the user that there are end-user application updates

I admit that I left Ubuntu because they didn’t even offer to install the stable GNOME updates and as I think those are really rock stable there is no reason to skip them. But Fedora far to often breaks something in the base system (read: libraries, kernel, system configuration).

BTW, I like the service-pack mockup though it would be nice to have some kind of “Details…” button can still query what is installed exactly.

I really don’t like the ‘hey, you can just enable Rawhide temporarily to grab the latest $FOO’ idea. Rawhide’s a platform, just like each stable release is a platform. We’d never recommend someone install a package from Fedora 13 on Fedora 12 just because its contents happen to be newer, because there’s no guarantee it will work well on F12 (in fact it’s pretty likely not to). The same is very much true of running a package from Rawhide on a stable release. That package was built against the rest of Rawhide, not against the stable release. It’s not at all likely to work perfectly in a different environment.

“If we had a concept of a base platform treated separately from applications….”

That’s certainly a possibility, though drawing the dividing line gets tricky, and it’s not uncommon to not be able to update an application without upgrading a bit of the base platform (‘well, OBVIOUSLY ShinyFoo 2.0 needs udev 1.5, duh!’) The interesting thing, though, is it’s more or less the exact opposite of what Fedora did in the past: Core and Extras was fairly close to being a ‘core platform / applications’ split. It may have been a split in other ways too, but that’s certainly the most obvious attribute of the split. Presumably there were some good arguments in favour of integrating them at the time, or it wouldn’t have happened. So I’d say any argument in favour of separating them again should consider whatever the arguments were for *combining* them in the first place – go back to the archives and look at that discussion.

The purpose of the Great Merge was to allow community access to the Fedora build process & infrastructure. The idea of a split between ‘Core’ and ‘Extras’ was never inherently bad – in fact it has some genuine, tangible advantages – but keeping ‘Core’ built behind the magical Red Hat firewall where nobody but Red Hat employees could touch it definitely *was* bad.

But there are ongoing efforts to restore some of those useful differences – the ‘critpath’ changes, for example.

I _really_ _really_ want everyone to adopt this.
It’s not really about the packages being updated, it’s about explaining to the end user (of which I think the first one represents a grand majority) what is updated in a short, concise and non-scary way.

Maybe we COULD cater to most needs. Not all (mixing server & desktop is a no-no IMO), but for example I can see selective updates working. Take for example as it is in Gentoo. You can have stable system and then turn on unstable updates for specific applications. We talked about this with a colleague, and AFAIK this is not possible in Fedora as of now. There could be “unstable” branch of current stable where updates could go faster, but people would have to opt-in for packages they would like to keep REALLY up-to-date, not just bugfixes.

packages are not all the same, there are *system* packages and extras. OpenOffice is NOT a system package, even though it comes bundled with the distro; people should be able to upgrade their OOo without having to upgrade the whole system.

I’m not talking about a “rolling release” system like people normally intend a rolling release, where still all the packages are first-class citizens (which means *nobody* is really first-class), I mean something like OH! TEH HORROR! Microsoft Update where “optional” updates are served along with mandatory, but they *are not* the same at all.

Of course incremental upgrades (Service Packs anybody?) could improve the stability of the system, without affecting backward compatibility. Sure this would require some collaboration from upstream: how many times couldn’t you upgrade your The GIMP becase GTK was too old? (and you couldn’t update your GTK libs because it could break GNOME aka the whole user interface — pass me this oversimplification)

And then, once the “base system” is defined, you wouldn’t have to release every 6 month.

And being stable and all would enable people to target a platform which lasts longer, without having to care about targeting the correct libs…

Completely agree with this post. It’s natural for free software developers to make software look like it does because they are scratching their own itches. We can do better with Fedora and make it nicer to use for everyone.

I like the ideear, but I also like very mutch to see what is uppdating on my system, even that the list can be long, so if we started to get collectiv updats for Fedora, I really would like to expand the packet, just to see what I’m gettin with this 100MB update packet… It is a part of feeling free, to see what goes on, not just eat huge update packets. And run out of space on your harddrive. So I see why somebody would like this, but also I like scrolling down the list of updates, not to forget the option to uncheck packets ther make dep.troubel or conflicts with some stuff I downloaded.

A nice post.
Presenting updates differently would make sense in my eyes.
Maybe updates can be summarized by the pkg-groups they belong to. This way more fine grained update policies could be established. Based on package groups and based on the update type (enhancement, bugfix, …).
If so, we could assign different update-cycle-times to different pkg-groups or upd-types.
E.g. bugfixes: daily, base-group: monthly.

This times could also be summarized in profiles (like ninja-always-daily or caroline-monthly with daily-critical-fixes) asked for at install time.

Other people make a point of catering to Caroline, and that’s what she’s using. Nothing Fedora does affects her until those other people pick up stable results from Fedora. As far as I understand it that’s the whole point of Fedora — but maybe I’m confusing it with Rawhide? Probably the RH universe needs a Raspberry Beret distro to cater to Caroline and keep from driving her to Ubuntu.

This doesn’t have much to do with distro competition. It has a whole lot more to do with stable Fedora not being particularly stable because of out–of-control updates, and policies to make it more stable. Fedora stable is not meant to be rawhide-like.

I’ve noticed a lot of folks have also misinterpreted what I meant by Caroline (which is fair enough, folks fill in the blanks on character sketches) so I will probably do a follow-up blog post that gives a bit more detail about her case (which is not, by the way, person-behind-me-in-line-at-the-supermarket or my-grandpa).

I would say I’m an advanced user. I would say I even enjoy fixing computers when they break (software wise). However, when I’m busy working on a project for school/work. I don’t want to have to spend time finding out why X stopped working. I want my stuff to work so I can churn it out. Then when I’m all about tinkering/fixing things I will drop to rawhide on a separate partition/virtualbox and mess with broken things there.

So in reality, there is a “Caroline” in all of us.

I don’t get this notion that all tech savvy people are okay with an unexpectedly broken system. Moreover the notion that all fedora people like it because it’s bleeding edge. I see Fedora as being a “Cutting edge” distribution, not “bleeding edge”. Leave bleeding for rawhide, thanks. To be honest, I don’t think that’s “asking” too much. Because I have seen fedora be blissfully stable (in fact most of the time) while still being ahead of the curve. The problem arises when a drunk driver comes speeding at you, driving in the wrong lane and ruining your peaceful Sunday cruse.

In my perception, this whole blog post was about providing better quality + less complex updates and in doing so, reduce bugs and enhance the quality of the distribution and the end-user experience.

Saying we can’t just accommodate Caroline seems to me, to imply that it’s ONLY the “Caroline” type that would benefit from clearer updates that are less likely to break the system. I feel that even “highly advanced” users that like tinkering with the system would benefit from changes long these lines.

I agree, and have thought about that particular dialog before. I agree that it is probably a good idea for certain apps to have different levels of QA. A bit of gut-instinct might be needed: a 32kb command-line-only program that finds bar-codes in images with no reverse dependencies might not seem important, but if a command flag were to change mid-release by accident, well let’s just say it would be important to me and the many other users who depend on our in-house document imaging server. I would particularly like the option to lag behind up to a month on my local-only security updates that don’t seem to have “pandemic” potential, as determined by a real-live human.

On the user-interface standpoint I have a very specific solution which I believe is not only doable but I believe very cleanly implementable: every package which is a dependency of only one currently installed package should be lumped up under that package via an expander widget unless it fails some kind of “I am my own end-user app.” New executive-summary release notes should be issued for all post-update packages with an option for users to click a more button to view the full release notes. Generally these should consist of things like “Fixes a security flaw which allowed users already logged into your computer to take full control over it.” “Fixes a security flaw which was widely exploited to allow anonymous individuals to remotely take control of this web-server when deployed in the default configuration. It is critical that you read the full release notes for more information: a large portion of individuals running this software are infected, being monitored for sensitive information, and generating malicious traffic” etc.

A policy would need to be generated to define an audience for these notes. I would advise “Barth,” my fictional interim IT Director at a 50-employee tire shop who’s sysadmin was just hit by a bus, who’s real title is Customer Service Manager, and has never before been in IT professonaly. He doesn’t fall for phishing scams, and knows his no-name flash MP3 player isn’t an ipod even though he accidentally calls it all that time, but he has no clue about what a service or a Linux is. Barth understands that he has important data on his servers, and the servers are expensive and very important. He checks his screens every day and needs to know:

a. should he apply the update
b. should he call his consultant
c. do both
d. do nothing, click cancel, hopefully the new guy will be here within 3 weeks and can figure it out

I’m very much for an increased separation between system packages and application packages, to better reflect how users actually thing about it.

I’m a developer, and even usually I don’t care about each individual update. Only occasionally do I care enough about some of them that I want detailed information.

In your last screenshot, the details of the update (individual packages) could be made available to the people that care by letting them click on the “platform update”. A suggested way to present them would be as children of the platform update:

That should be very easy to implement, assuming the list uses a GtkTreeView. But the package and update policy is probably the hard thing here anyways.
I’m not familiar with Fedora development practices, but you should consider writing a short email with the main part of the argument and link to this blogpost, to better communicate this idea to the relevant people.

1) If we don’t get bleeding edge software on our desktops, they don’t get enough testing and the quality of the software takes much longer to mature

2) I hate it when I dist upgrade and suddenly sound or accelerated video stops working. Most recently F12-F13, ATI driver did not support the new Xorg stack.

Things like #2 almost never happen in the Windows/OSX world, so after a while, you just have had enough of it all and scream bloody murder.

As a developer, I am frustrated / embarassed when I show up for work, and my laptop has reduced fuctionality. Eg, I have to give a presentation and I can’t get the projector working anymore, despite the fact that it worked beautifully the day/week/month before.

I have been distro-hopping for a while now & have developed a strong liking for fedora, but I wonder if we need a stable / testing model too.

I love the idea of classifying the software stack and managing the releases / stability issues, but its a tough balance.

I have an OSX machine and precisely due to the closed / proprietary nature of the beast, updates are less frequent and it has been far more reliable than any of my linux machines, much to my disgust. I know its not ideal, but maybe we could learn from them and have a more co-ordinated release model. (dons flame retardent suit)

I think key to the future would be more colloborative sharing of roadmaps of projects. If a development change is coming which would break other dependent applications in the stack, all affected developers should be able to anticipate and accomodate these changes so that we don’t have that much ripple effect when Xorg changes something major.

1. The pair of pictures of PackageKit, which show how updates might change in a way that finally makes sense to Caroline, are brilliant. Right now, there is almost no way for any true QA to happen post-release. The moment we cut DVDs/CDs, Fedora’s at its maximum “tested” value (even though we could always use more people’s help there!). Once we start issuing updates, those get tested incrementally but the *integration*, despite packagers working hard, will tend to increase in entropy. There’s no one testing the whole distro as a unit at that point. If we had blocks of updates in some way — superpackages? — it might be possible to build further QA efforts around those blocks.

2. One of the reasons “let’s push new stuff now” people might feel unhappy is that they feel Rawhide is unusable. Something you’ll see over and over in current list discussions is that Rawhide is often mentioned briefly, and usually in the context that “Rawhide often is broken” or “Rawhide is too fragile.” The question I keep asking myself is, is that true, or necessary? What if Rawhide did let you dump brand new stuff in, but only if you didn’t break it untenably for everyone? What if our tools made it simple for developers to do that sanely? Or is Rawhide fated to be overtaken by kopers (koprs, coprs, whatever) acting like private branches, where crazy work will be done, and then delivered into Rawhide/Fedora(N+1) when it’s ready for more mass consumption?

Hmm..
On point 2 I think there CAN be a middle ground between “Rock solid, productivity ready” and “it will bite you every once in a while” without forcing people to decide between “stable” and “it will eat your cat”.

I think the point is TOO much of the testing and Q/A happens in what is thought to be a stable release. If the goal of Fedora is to push boundaries and enable free software contributors I don’t see what providing a less than stellar quality control in between releases does to advance the cause. If I’m trying to improve GIMP, but broken gtk updates are getting in my way, then how does Fedora enable me? Or better yet, a broken Xorg.

On a slightly different note I think it’s worth Fedora’s time to try and push forward improvements not just in the software, but how that software is delivered. The entire process of “free software” development is just as important as improving the technology. Are we really delivering updates in the best way (and I’m not talking about delta-rpms vs. full or what not, but rather the process of getting a package into the repos and the certainty to which we can say, “this is a quality package”)?

Does switching to an update model where we are pushing out updates in “packs” improve the Fedora experience and the quality? If so, then I think it’s worth pursuing.

Just a quick note — more later. I think you’re being unfair to Nancy in a way that colors the discussion poorly. Specifically, she’s not “afraid” of change — she just knows that change comes at a cost, and she wants that cost to be considered in design decisions.

Also: Nancy is more aligned with Caroline than might be apparent at first.

Nancy’s day job involves enabling Caroline to use her computer without technology getting in the way. She might joke about how much easier it’d be if there weren’t any users, but ultimately, she’s in this because she enjoys building and running systems for real users.

That being said, I think there’s one kind of user that is missing: a Pamela that loves her friend Caroline and realises that Linux/FLOSS should focus more on Carolines. Which is what I think I am, except that I’m male :)

But what if Caroline buys a shiny new webcam and just want it to work. Lets say support for said webcam is in a newly released upstream kernel, which pretty much fits into “base OS” due to it not being a “bugfix or security update” we end up with:

Caroline: “My new webcam doesn’t work”
“Go install a kernel from $randomrepo”
Caroline: “What? What is a kernel I want my webcam to work”
“OK, wait for the next release .. which is due in 4 months”
Caroline: “What?!? I bought it to use it today”

It isn’t that simple as “only security and bugfix updates” … don’t have a shiny new feature for some months is fine but not being able to use hardware is serious flaw.

I think that is an important thing to consider when talking about “only (critical) bugfixes and security fixes”

I think it may be reasonable to have a well-documented list of supported hardware, with a dozen or so highly-recommended peripherals in each category. Caroline can then be encouraged to consult that list before purchasing new items. Hardware enablement in a lot of cases is probably okay in a stable release and won’t risk too much breakage. E.g., adding additional ppd files and additional udev rules for more mp3 players and the like….

That does not work… hardware on those list is usually old and hard to find or very expensive. If Caroline is as described, she would have problems understanding the compatibility list and will chose Windows as the “easy” solution where everything woks.

Not always! I have to say in the past 2-3 years I’ve bought a bunch of peripherals that have ‘just worked’ with Fedora. A Canon MX860 wireless printer & scanner…. the Kodak Zi8 video cam…. my Canon XTi SLR (a gift)…. a Wacom Bamboo…. I think probably webcams as a class of devices are perhaps the most picky, but I think a lot of peripherals do just work.

At the time we start telling people “consult a hardware support list” we are moving away from the “I don’t care about technology” kind of users. (i.e Caroline).

“Hardware enablement in a lot of cases is probably okay in a stable release and won’t risk too much breakage. E.g., adding additional ppd files and additional udev rules for more mp3 players and the like….”

Yeah I was talking about the potentially invasive changes like kernel or X drivers.

If a patch is known to bring hardware enablement, the distro should be smart enough to compare what it already supports and what would support if it applied new “hardware patches” that are in remote repositories. Thus, this hardware-enablement packages would be only installed to users that really connect/install some hardware whose support has been fixed upstream. I think this is the win-win solution, but of course the most complicated to implement.

Oh, and with this, the user would connect his shiny new camera, the distro would detect that there’s no support for it, query the packages in the updates repo to find the concrete one and, if found, present a simple message to the user: your system must be updated to be able to use this hardware, do you wish to continue?

This way, we eliminate the risk of enabling all hardware-enablement packages for all users that could break something, if they actually don’t need them.

Well from a maintenance POV this would be a nightmare.
(As Mo said “which is maybe messy”, you can scratch the “maybe” from there ;) )

“This way, we eliminate the risk of enabling all hardware-enablement packages for all users that could break something, if they actually don’t need them.”

I don’t really like that part, if we have support for the hardware we should install it (via updates). Currently in Fedora we have a (unwritten) “hardware just works” policy i.e we install drivers (and firmware) for all hardware by default so that users can just plug it in and use it. Which is a great user experience so we should continue to do so.

Even though I don’t have a magic solution to the problem that make everyone happy (Caroline, kernel maintainers … ) I wanted to note that this problem does exist and should be considered when revising the update system (which is indeed needed as recent discussions have shown).

The Linux kernel is not maintained in such a way to easily allow random new hardware to work with arbitrary kernels. Greg Kroah-Hartman and others have written extensively on this topic and they claim that it leads to better software. Many non-kernel devs claim that it leads to more difficult-to-maintain systems because you need the latest kernel to operate the latest hardware, and getting the latest kernel is not as easy as it could be.

Lately there have been some long-lived “stable” kernels which are receiving driver back-ports. These kernels are good candidates for “stable” distributions. But the reality is that most new drivers land in the latest kernel only and you need THAT kernel to get your new webcam working.

Mo,
I’m confused on your graph. If the zoom-in of the stable graph still has the same timescale then it seems like everyday the user will still see new things which need to be updated. Only some things will be hidden behind a “system update” description/group?

So they are still getting the firehose of updates but we’re just describing some of them differently?

Sort of! I was thinking only some things would be hidden behind the ‘system update’ description/group. Anything above the system-update group in the chart can be updated more frequently than the monthly base platform update packs. Maybe core desktop updates are allowed bi-weekly, depending? Maybe they are not all turned on by default? Maybe the application stream is allowed within a certain time frame / amount of testing karma for a specifically blessed core set of packages (say stuff we install by default – firefox, evolution, gnote, character map, etc.), and there’s the least amount of restriction on packages outside of the core of most-frequently-used apps. (I’m sure we’ve got far less users of OpenOffice Draw than we have Firefox users, for example.)

So, the base platform is not a firehose at all…. core desktop components and apps are occasional annoyances…. applications outside of the default app set are still a firehose.

Certainly more fleshing out is needed. But if the packagekit screenshot I gave listed out updates for user-visible apps – it’s a lot less scary than all the nonsense that was in the actual screenshot I showed.

What do you think? Is there a deal-breaker here or do you think we could make it work?

Ah something I forgot to mention that should have been made clearer—the graph emphasizes time limits. There are other factors though – like the type of fix (security / bug / enhancement), and the amount of testing required to get it through the gate. I think maybe another graphic / follow-up blog post might make those factors a bit clearer. I could see how based on the graph it might seem as if the same firehose is coming out, it’s just in measured steps based on time alone – not the intention!

People who want to use a super stable system should not use Fedora, they should use RHEL/CentOS. Like it or not, Fedora is the playground for all the RedHat engineers. As a developer, I want a distribution that’s as up to date as possible, but I don’t to be a sysadmin and I don’t want it to break. And I find that recent Fedora has managed to keep that balance pretty well. Previously, I used Gentoo as my main system because it has constant updates. But Gentoo has been taken by the “stability” and “QA” diseases and now they’re like 6 months late in Gentoo Gnome stable. Please don’t do the same thing to Fedora. Keep the updates coming!

RHEL/CentOS are not meant for people like Caroline. They are meant for servers. I can’t use RHEL as a desktop without hacking it or building my own Inkscape packages. They are fine as a stable desktop for basic productivity / knowledge workers, but for folks who want to do creative things like work with Inkscape, they are completely unacceptable.

It would be just wonderful if the core of the OS was stable for longer periods, while applications could be faster upgraded. It is just so annoying when version X+1 of my favourite program is released and I have to wait months for it to hit my laptop. A kernel update, on the other hand, I don’t mind waiting for.

Funny, I think the opposite is true for me: I don’t like waiting more than absolutely necessary for kernel updates (it might be important security-related fixes), but if really need version X+1 of an app Right Now, I’ll probably get the srpm of the version X, get the tarball of X+1 and make my own X+1 rpm and install that.

Here’s my idea:
So there is a yum setting that allows you to select the “types” of user you are.

So if you select “Keep it safe” Then you only get security updates and only the most important bugfixes.

If you select “Bleeding Edge” then you get all the latest packages, new kernels and all as they come out, etc (though not quite rawhide bleeding).

Then we can have a section in between which only updates applications but not system components (applications that require new system components are not updated).

How we accomplish this is by adding three option/tags to yum. One to “hold” a package at it’s current version. A “Security updates only tag, and a “bleeding edge” tag which would be most useful when you wanted to keep only a few packages bleeding edge.

I think this method means we keep things mostly the same, we just add some features to yum. The main problem that I see is that now various people are on different “systems”. If half the user-base is on the original kernel, 25% are on the bleeding edge kernel and the remaining 25% are somewhere it between. But isn’t that sort of how we are now??

Sorry, part of the hard part of managing a distribution is making sure that the pieces work well together. If you allow package versions to drift every each way, it will become a nightmare (combinatorial explosion and all that).

Today’s Carolines could become tomorrow’s Connies — especially if we make it easy for folks to understand the social benefit of free software, without requiring that they understand the technical aspects first.

It seems to me that the update dialogue could be made a lot friendlier (more like your second Software Update mockup) just by presenting linked updates as one, rather than a mass of individual packages. There’s still the matter of the Caroline-frendliness of the update details and of the title of the package group, but that can be approached as a separate problem.

For instance, in the first Software Update mockup you have a long list of PK sub-packages, all of which will show the same (technical) update information. In reality, all these sub-packages were pushed out as one update – see https://admin.fedoraproject.org/updates/PackageKit for examples. Each update has a unique id and that id might cover updates of a package & all it’s sub-packages (FEDORA-2010-10667) or a group of packages and their subpackages (FEDORA-2010-5392). Why not present this in the gui, i.e. present one item for each update-id and display the update details when/if requested. For example, https://admin.fedoraproject.org/updates/FEDORA-2010-10667 has (slightly less technical) update details and links to (fairly technical) descriptions of problems solved by the update.

To put it another way, what would a GUI for ‘yum info-updateinfo’ (from the yum-plugin-security package) look like? I’m guessing it would look a lot like your second mockup. Admittedly, it leaves unsolved the problem that PackageKit’s friendly summary (“Package management service”) isn’t friendly enough for Caroline or that the updateinfo description isn’t always Caroline-friendly. But aren’t these separate problems?

My point is that listing updates by update-id rather than by package gets us a long way to the update experience presented in your second mockup, without requiring “a concept of a base platform treated separately from applications” [1]. In fact, it would work just the same for kernel updates, kde updates, openoffice updates, firefox+xulrunner updates as for PackageKit updates.

[1] Please don’t read this as a comment on the “concept of a base platform treated separately from applications” or anything in the body of your post. I’m only addressing the update experience presented in the mockups in the bonus part of your post.

I’m also not addressing Paul Frields’ suggestion of collecting blocks of updates (“superpackages”) to concentrate testing on. The UI for such an update could look a lot like the normal update-id-wise UI.

One point that I don’t think has been made yet: your suggestion would work if Caroline is avoiding installing the updates just because she doesn’t understand what they mean. However, if her logic is “if it ain’t broke, don’t fix it”, she may still avoid installing the unified platform update.

It’s very difficult to convey the concept that even though the software on the computer didn’t appear to be broken yesterday and the software hasn’t changed, the *same* software *is* now broken, because there’s a security vulnerability in the wild.

One solution is education: “strongly recommended”, “fixes security vulnerabilities”. The problem with this is that users will perceive more (or louder) security fixes the same as if there were more *vulnerabilities* in the first place.

So the other option is to silently install security updates by default. The choice between ”install platform update” and “don’t” is still a choice the user doesn’t really understand—they’re just less overwhelmed by jargon.

Asking every user to decide whether they trust the distro’s QA process for security updates, each time there’s a security update, is a bad idea. This decision should be made when installing the distro or enabling the repository. (And there will be an option to disable automatic updates.)

The QA process should already ensure that any security fix doesn’t cause fallout. If one does, another silent update can fix it promptly, without having to bother the user (again).

This approach—silent security updates—is used by Chrome and Firefox on Windows. It’s also used by every web service ever. Desktop distros should use it too.

Great post.
I think your suggestions could be a piece of the solution of this problem. This problem, by the way, is the exact reason why I am not a Fedora user anymore.

I think Fedora should learn from Debian in general and at the sidux derivative in particular, which is basically “transforming” sid (=debian unstable, sometimes broken like rawhide) in some cutting-edge but not bleeding.

Separating out streams of updates for applications vs base platform sounds nice, but runs into a big roadblock. Security and bugfixes. How does one deliver security and bugfix updates to applications to a user like Caroline without also delivering the version upgrades for the same packages that Pamela expects?

The only way I see that working is if every release had multiple branches in source code and package delivery, and when a security issue came along a maintainer would potentially have to issue two updated packages /per Fedora release/ to cover the users. That would be 4 packages for active releases, more for branched releases and rawhide. A significant uptick in the amount of work necessary.

MY brain has been down this path before and found pitfalls, but I can’t recall all of them now. John McCann has asked me to put them in a wiki somewhere and I’ve failed at that so far.

Not bad, not an indictment, just an observation. These are hard problems to solve. If they weren’t, they’d already be solved. :)

I think Jesse is spot on: this kind of mechanism is very likely to increase the workload for already overburdened packagers, and the answer absolutely must take those packagers into account, or they simply won’t do the work.

Mo, is there any chance that you could work with packagers to build this kind of visualization *from the packager’s perspective*? These are great tools to understand the *user’s* problems. But since we are a community of volunteers, it seems to me that we must have an equally good understanding of the *packager’s* problems in order to figure out how to meet halfway.

Where do contributors other than Packagers fit into this? Is Pamela Packager supposed to also represent Artists, Documentors, Translators, Infrastructure, Ambassadors and probably others that I have forgotten.
Sometimes I think us packagers think we are the only people working on this boat and don’t value the others highly enough.

This is interesting discussion, I’m glad it has come up. I’m someone who is not quite a Pamela, although I’m planning on getting involved in packaging software for Fedora soon. A little history, I started using Linux as my main OS with Fedora Core 6, I’ve hopped around a bit, but I’ve ended up back at Fedora. FYI – Fedora 13 rocks — great job. I’m running it on my desktop, laptop and test server (4 kvm vms). I want a “stable” OS e.g. one I can trust to boot and continue to work after an update, with fairly recent application software e.g. software that no more than 6-12months old. In short, I want workstation class OS (e.g. virtualization, development tools etc) with the ability to manage my music collection as well, for example.

Issues with other Distros

1. Ubuntu – Too many issues with non-standard setups e.g. My /home partition is an encrypted Linux MD Raid 1 array
2. Archlinux – Too much work to keep the system upgraded an current
3. Debian Stable – Almost there (for me any way), but the packages the application software are a little too old to support newer devices like my Nexus One
4. Debian Testing – Stable, but prone to things being broken every once and awhile
5. CentOS – Just too old and the packaging situation is just horrendous, it necessitates getting software from multiple somewhat incompatible repos – the debian universe does a much better job at this IMHO.

With that, you can see that someone like me, a technically sophisticated user would really like to see something like what Mairin has proposed.

Food for thought — What if Fedora adopted the following:

This would necessitate shifting to an annual release cycle with ideally 2 years of support from the initial release date.

1. Kernel – Bug Fixes and quarterly releases (maybe monthly)
2. Base OS – Really narrow definition here – python, gnu tool chain and a few other core components (I’m not sure I know enough to suggest what should be included, but the concept still applies) – No updates after release with exception of bug fixes.
3. Desktop environment – 1 update per release with the exception of bug fixes
4. Applications – When new stable / bug fix versions are released

It could be that after the first year the release shifts to maintenance mode and only gets bug fixes to encourage people to upgrade.

Adopting this model offers some advantages:

1. Fedora is still unique – no other distro has a similar support structure that I know of
2. Fedora becomes a real option of servers – 1 yr of feature upgrades and 2 yrs of bug fixes is awesome
3. Fedora becomes the defacto workstation OS – It would be both modern and stable, this is a definite niche that no distro does particularly well

I presently use Fedora on my desktops and Debian Stable on my servers. My test server is Fedora but the vm’s it runs are Debian Stable. I would love to switch to Fedora for my servers and a model like I’m proposing would make that possible.

I’ll say it again, I think Fedora 13 is great and I would love to see it evolve into a workstation type OS, which could be used by casual users who wouldn’t touch most of the power, but was a little more stable than its current instantiation, but less stable in terms of active development / new features than RHEL, CentOS or Debian Stable.

Great discussion, I look forward to being more involved in the community.

I think you mean “security holes” when you say “bug”. Otherwise you will end up updating almost every (is there a release that doesn’t fix a bug or two?)

> This would necessitate shifting to an annual release
> cycle with ideally 2 years of support from the initial release date.

Although I would love your suggestion, I’m 100% sure this won’t happen. They are already complaining about the 13 months being too much!!

> 1. Fedora is still unique – no other distro has a similar support structure that I know of
(snip)
> I’ll say it again, I think Fedora 13 is great and I would love to see it evolve into a
> workstation type OS, which could be used by casual users who wouldn’t touch most of > the power, but was a little more stable than its current instantiation, but less stable in
> terms of active development / new features than RHEL, CentOS or Debian Stable.

Have you tried sidux? The stability of Debian stable from the repositories of Debian Unstable (sid).

I think that’s the model you are looking for. The problem with sidux for an user like Caroline is that some things are not shiny at all (eg the wireless network interface is “ugly”, but it works great).

Very great post! Even if I’m a technical person, I dislike the information displayed in the current PackageKit GUI. However if you take a look at MeeGo, they are already doing it user-friendly.

On another note, it looks like a remark that comes often up lately is the definition of the “stable distro”. IMO, a “stable distro” is only worth within enterprises where you keep, as system administrator and technical support, well-defined versions that do not change except for security updates and the sort.

For the casual workstation at home, I f*g do want a distro that comes with stable and fresh versions. My favorite choice since almost a year now is ArchLinux (told you I’m technical :-p).