FC4 – thoughts about the future.

October 29, 2004

I’ve been thinking a good bit about what needs to happen in fedora for FC4. Not just in a technology sense but also in a community/organizational sense as well as infrastructure needed for the future.

A few of those ideas:

Communication between the users/developers and the steering committee/leadership needs to exist more and be more consistent. Right now communication seems to be fairly sparse and when it does happen fairly one-way. The remark most often made is that it is the same communication style that existed pre-fedora in the rhl timeframe. To be fair it’s improved in the past few months. Gafton is talking more and that’s great. But the rest of the steering and technical committees need to talk and suggest and post and recommend publicly. These people are on the steering committee and technical committees because they are experts in their fields with years of experience. However, they’re also extremely busy. Red Hat needs to recognize this conflict and either appoint different people to the steering and technical committees or free up more time for the people on the committees now.

Leadership: there needs to be more recognition of the people at red hat and outside of red hat who are leading the way on certain projects. Not only because it is heartening to people volunteering but also because it decreases confusion on the part of the new user or new developer who wants to know who to talk to about working on a particular project.

Update posting and notification infrastructure needs a complete overhaul. We need to make it impossible to post an update without an email going out explaining why the update occurred. We’ve had way too many packages get dropped in updates and updates-testing with no explanation for their presence at all. It needs to stop.(Shameless plug: in FC3 and beyond you can run ‘yum list recent’ and get a list of packages added to any repository in the last week)

Announcements and notifications about events or things going on in the fedora world need to exist in places other than just the mailing lists. The mailing lists are great but occasionally things get lost in the sheer noise and volume. We need more of a content mgmt framework available to manage the information that should get greater prominence because of its importance and/or significance.

Fedora Extras needs to actually happen. This is obvious but it can’t be overstated. This has to happen and the only way it will happen is if people outside of redhat.com decide to work on packages. That’s all there is to it. Extras cannot be run solely from redhat.com. There is just no way to do it. Extras needs to get going for multiple reasons. First, there is no good way of packages ever leaving Core. This means a package that is slowly dying and decaying over a period of months as it either grows crufty from lack of development or becomes less and less used for various external reasons will always raise a lot of ire when/if it is removed. Second, we need a way of vetting packages and migrating important/useful/active ones into Core. Third, we need a good place to mentor and train new packagers and developers. Without Extras there is no place for these people to go to learn how to best package for compatibility with Fedora Core.

Mirroring. The mirroring infrastructure needs a lot of love. Right now consistency of mirrors is a dicey proposition. Especially in a rapidly-changing environment like rawhide. We need a few different items to deal with this:

Some way of synchronizing the mirrors of rawhide more efficient than rsync. Rsync is great but for data that is changing over ~5GB a day it just won’t work for a hundred + sites to sync to a single master. We need tiering of the mirrors to be configured and rigidly enforced and we need to consider working on new protocols to synchronize mirrors from each other rather than all from a central server or a central set of tiered servers.

We need a good way of verifying mirror content for listing the mirrors as synchronized and for handing out accurate mirrorlists for yum and up2date. This should include both pull mechanisms that crawl the mirrors and verify that the data is accurate and consistent and push mechanism that let mirror admins submit information about the consistency of their mirror. This is going to take a lot of cooperation from the mirror admins and some coding to make this infrastructure live.

Package Metadata: Right now fedora core 3 ships with 5 different encapsulations of the rpm package metadata:

comps.rpm

hdlist[2]

yum .hdr files

rpmdb-fedora

repodata xml-metadata

I think a lot of concerted work needs to go into reducing this to as few as possible. The yum .hdr files should be able to go away after FC3. So we’re down to four.

Comps.rpm is only used for system-config-packages, therefore in firstboot. We can do better than this. We need to work on merging yum and system-config-packages so the graphical interface of system-config-packages uses the yum modules for finding and resolving dependencies on packages. That could get rid of comps.rpm.

The hdlists are used only by anaconda. If we can get anaconda to be migrated to using the repodata files for installs then the hdlists can go away too. That will be tricky and requires a fair bit of work early on to get anaconda functioning adequately. After all, if the installer doesn’t work, then there’s no point in going on to other things.🙂

Finally, rpmdb-fedora. This is used by ‘rpm –redhatprovides’, ‘rpm –aid’ and ‘rpm –redhatrequires’ for querying what was in the original installation tree. It’s useful for query information from the base, stock distro, but with a distribution as rapidly changing as fedora core it might not be very valuable. It might be more useful to write the code to make it possible for rpm (or some command line tool) to do arbitrary queries from xml-metadata (repodata) repositories out on the network or even ones on the local disk easily.

If we can get there then we’d be at a place with a single metadata type in the installation media and on our way to converging a number of tools. That might save a considerable amount of disk space on the CD images and it will definitely remove some redundant code.

Technical Priority establishment: Right now technical objectives and deliverables are established in what appears to be fairly arbitrary ways. The discussions are closed off from anyone. I’m not recommending that any and every random user’s opinion be taken into account, but I think the discussion of objectives by the technical steering committee should be made available in a read-only medium so external developers and packagers can get a feel for where fedora _should_ be going so they can adjust their packages and development with this in mind. For example: If I knew that by FC 5 the drive is to have an all-singing, all-dancing Desktop install equivalent of OsX but w/o all the bugs then I might keep that in mind as I developed new features for yum.

Specific Features/Items I’d like to see in the distro or at least under development:

An end-user graphical backup tool. Possibly something using rdiff-backup or similar that would let a user backup their data to another directory or another system w/o having them have to learn either a difficult command line option or an arcane and arbitrary config file format.

More and more items using dbus for notification of events. This might include tying in logging tools to dbus for security alerts and events.

A concerted effort to make the acpi sleep and suspend functions work out-of-the-box on more laptops and desktops.

More hardware compatibility reporting. Reports of failed and successful configurations of systems for Fedora Core installs and use.

So, What am I going to do about any of these? The projects related to yum and the package metadata I will be spending much time and focus on in hopes of achieving some of them by FC4. Time permitting I’d like to look into some of the mirroring issues and work on code to resolve and manage some of them. Over the FC3 test cycle I’ve been helping out with mirrorlist maintenance and I can personally say that maintaning those lists by hand sucks.🙂

For the other ideas the only thing I can do right now is to push folks inside red hat to move forward during the post-FC3-final, pre-FC4-test1 period. I’m also going to encourage anyone who thinks these items would be cool to work on them and post your announcements and progress on the fedora mailing lists. In addition, if anyone is working on any of the projects for fedora, or working on new packages for fedora extras and you have a blog that you would like to see added to the Fedora People feed please let Me or Colin Charles know about it.

It’s great that so many people at red hat work on fedora. However, that doesn’t make a strong community, that makes a strong company. Fedora need more external developers and I’ll do whatever I can to help and encourage more external folks to become involved. If anyone has some suggestions, let me know about them.