Which tells us that in 2015, the FTP masters and the decrufter performed on average over 10 removals a day. And by the looks of it, 2016 will surpass that. Of course, the auto-decrufter has a tendency to increase the number of removed items since it is an advocate of remove early, remove often! .
Data is from https://ftp-master.debian.org/removals-full.txt. Scoreboard computed as:

2 September 2015

I m receiving more requests for upload accounts to the Deb-o-Matic servers lately (yay!), but that means the resources need to be monitored and shared between the build daemons to prevent server lockups.
My servers are running systemd, so I decided to give systemd.resource-control a try. My goal was to assign lower CPU shares to the build processes (debomatic itself, sbuild, and all the related tools), in order to avoid blocking other important system services from being spawned when necessary.
I created a new slice, and set a lower CPU share weight:$ cat /etc/systemd/system/debomatic.slice
[Slice]
CPUAccounting=true
CPUShares=512
$
Then, I assigned the slice to the service unit file controlling debomatic daemons by adding the Slice=debomatic.slice option under the Service directive.
That was not enough, though, as some processes were assigned to the user slice instead, which groups all the processes spawned by users:
This is probably because schroot spawns a login shell, and systemd considers it belonging to a different process group. So, I had to launch the command systemctl set-property user.slice CPUShares=512, so all processes belonging to the user.slice will receive the same share of the debomatic ones. I consider this a workaround, I m open to suggestions how to properly solve this issue :)
I ll try to explore more options in the coming days, so I can improve my knowledge of systemd a little bit more :)

21 October 2014

This is an update of my previous attempt at summarizing this discussion. As I proposed one of the amendments, you should not blindly trust me, of course. :-)
First, let s address two FAQ:
What is the impact on jessie?
On the technical level, none. The current state of jessie already matches what is expected by all proposals. It s a different story on the social level.
Why are we voting now, then?
Ian Jackson, who submitted the original proposal, explained his motivation in this mail.
We now have four different proposals: (summaries are mine)

[iwj] Original proposal (Ian Jackson): Packages may not (in general) require one specific init system (Choice 1 on this page)

[lucas] Amendment A (Lucas Nussbaum): support for alternative init systems is desirable but not mandatory (Choice 2 on this page)

[plessy] is the simplest, and does not discuss the questions that the other proposals are answering, given it considers that the normal Debian decision-making processes have not been exhausted.
In order to understand the three other proposals, it s useful to break them down into several questions.
Q1: support for the default init system on Linux
A1.1: packages MUST work with the default init system on Linux as PID 1.
(That is the case in both [iwj] and [lucas])
A1.2: packages SHOULD work with the default init system on Linux as PID 1.
With [dktrkranz], it would no longer be required to support the default init system, as maintainers could choose to require another init system than the default, if they consider this a prerequisite for its proper operation; and no patches or other derived works exist in order to support other init systems. That would not be a policy violation. (see this mail and its reply for details). Theoretically, it could also create fragmentation among Debian packages requiring different init systems: you would not be able to run pkgA and pkgB at the same time, because they would require different init systems.
Q2: support for alternative init systems as PID 1
A2.1: packages MUST work with one alternative init system (in [iwj])
(Initially, I thought that one here should be understood as sysvinit , as this mail, Ian detailed why he chose to be unspecific about the target init system. However, in that mail, he later clarified that a package requiring systemd or uselessd would be fine as well, given that in practice there aren t going to be many packages that would want to couple specifically to systemd _or_ uselessd, but where support for other init systems is hard to provide.)
To the user, that brings the freedom to switch init systems (assuming that the package will not just support two init systems with specific interfaces, but rather a generic interface common to many init systems).
However, it might require the maintainer to do the required work to support additional init systems, possibly without upstream cooperation.
Lack of support is a policy violation (severity >= serious, RC).
Bugs about degraded operation on some init systems follow the normal bug severity rules.
A2.2: packages SHOULD work with alternative init systems as PID 1. (in [lucas])
This is a recommendation. Lack of support is not a policy violation (bug severity < serious, not RC). A2.3: nothing is said about alternative init systems (in [dktrkranz]). Lack of support would likely be a wishlist bug.
Q3: special rule for sysvinit to ease wheezy->jessie upgrades
(this question is implicitly dealt with in [iwj], assuming that one of the supported init systems is sysvinit)
A3.1: continue support for sysvinit (in [lucas])
For the jessie release, all software available in Debian wheezy that supports being run under sysvinit should continue to support sysvinit unless there is no technically feasible way to do so.
A3.2: no requirement to support sysvinit (in [dktrkranz])
Theoretically, this could require two-step upgrades: first reboot with systemd, then upgrade other packages
Q4: non-binding recommendation to maintainers
A4.1: recommend that maintainers accept patches that add or improve
support for alternative init systems. (in both [iwj] and [lucas], with a different wording)
A4.2: say nothing (in [dktrkranz])
Q5: support for init systems with are the default on non-Linux ports
A5.1: non-binding recommendation to add/improve support with a high priority (in [lucas])
A5.2: say nothing (in [iwj] and [dktrkranz])
Comments are closed: please discuss by replying to that mail.

25 May 2014

I ve been playing with qemu-user-static a bit to create a set of porterboxes for my Deb-o-Matic build farm. After reading gregoa s post on how to create cross-chroot with qemu-debootstrap, I was immediately able to create armel, armhf and powerpc boxes with very little efforts.
I tried to extend the number of porterboxes available by adding mips* and s390x, in order to have all the Linux-based architectures supported in Jessie, but with no luck. Here s a summary of my attempts.
MIPS*
Chroot creation fails under both mips and mipsel trying to configure libuuid1. The problem is due to the fact libuuid1 s postinst script calls groupadd and useradd. Those two utilities rely on NETLINK sockets, which apparently are not handled by QEMU at the moment. I raised the question upstream to see whether it is possible to solve this problem.
s390x
Chroot creation used to fail with a SIGSEGV. This particular bug has been fixed recently, but it seems it s not enough to have a working chroot. It fails with some gzip errors, probably because some portions of dpkg-deb are not fully covered by qemu-s390x-static.
Preparing to unpack .../base-files_7.3_s390x.deb ...
Unpacking base-files (7.3) ...
dpkg-deb (subprocess): decompressing archive member: internal gzip read error: '<fd:5>: incorrect data check'
/bin/tar: Unexpected EOF in archive
/bin/tar: Unexpected EOF in archive
/bin/tar: Error is not recoverable: exiting now
dpkg-deb: error: subprocess tar returned error exit status 2
dpkg: error processing archive /var/cache/apt/archives/base-passwd_3.5.33_s390x.deb (--install):
subprocess dpkg-deb --control returned error exit status 2

19 September 2013

I ve been running several Deb-o-Matic instances on several servers since ages. This service, meant to provide a way to automate the build of Debian source packages, has been the starting point for a lot of Italian developers to start packaging, and eventually become Debian or Ubuntu developers. I can t exactly tell how many packages Deb-o-Matic servers have built so far, but it has been the primary build and test environment for at least three Debian Developers since four years.
This service was hosted for free since the beginning, but the good things always come to an end. My provider decided they cannot keep our servers running (its core business is totally unrelated to hosting servers), and asked me to shut them down. I started with the two kfreebsd-* machines, and soon I will shut down i386 and amd64 as well. When all services will be down, I ll take care of removing the debian.net domains I created for them.
Deb-o-Matic development won t stop, but developers who used Deb-o-Matic daily will have to find a different solution to build and test their packages.

11 March 2013

While reading the platforms for the upcoming DPL election, I read some statements about hearing from Project members about their experience within the Debian Project, so I took the chance to write some thoughts about my experience in one of our core teams, in which I have the privilege to be part of: the FTP Team.
During NM process, my Application Manager asked me about some activities I could have involved into once I became a Debian Developer. I was fascinated by the hard work of the FTP folks when processing NEW packages (and I guess everybody reading these lines agree with me, remembering the happines of reading that wonderful ACCEPTED mail for their very first contribution to Debian, I still have it in my mailbox), so I decided I wanted to help out those folks to share a portion of their hard work.
I committed errors, sometimes they were so silly I d be better to hide, but the team helped me to solve the mess I created. The thank you and kudos messages the FTP Team receives greatly overtake the frustration for my mistakes, and give the strength to keep up the good work
FTP Team is just one of the dozens teams which do an awesome work for the Debian sake, every day. Don t be shy, or worried of being unsuitable or not skilled enough for the role. You ll never know until you try. And it could happen you really like the job, and you could feel sorry for have waited so long

5 January 2013

1.3:
* Avoid failing on upload if a pre/post upload hook is missing from the
Filesystem.
* Fix "dcut raises FtpUploadException" by correctly initializing the uploader
classes from dcut (Closes: #696467)
1.2:
* Add bash completions for dput-ng (Closes: #695412).
* Add in a script to set the default profile depending on the building
distro (Ubuntu support)
* Fix a bug where meta-class info won't be loaded if the config file has the
same name.
* Add an Ubuntu upload target.
* Added .udeb detection to the check debs hook.
* Catch the correct exception falling out of bin/dcut
* Fix the dput manpages to use --uid rather then the old --dm flag.
* Fix the CLI flag registration by setting required=True
in cancel and upload.
* Move make_delayed_upload above the logging call for sanity's sake.
* Fix "connects to the host even with -s" (Closes: #695347)

30 December 2012

I don t like watching videos sitting in front of a computer, I usually store them into a USB key to plug into my Samsung LE37 TV set. It s not an easy task, though, as I have to fight with other components of my family who usually like watching something completely different than DebConf videos I could have invested 200 to buy a new one, but I tried a different approach first.
Recycling a netbook
I received a Tweety device as Christmas gift, and I tried to connect it to my netbook to see how it worked. I was quite impressed by the good quality of the sound (if compared to the really poor quality of the netbook speakers), and I realized I could have recycled the netbook as a brand new TV set with some little adjustments. I plugged my SyncMaster 920n PC monitor into the netbook VGA port, switched Totem to fullscreen mode, and here it is my TV set!
Remote control
OK, that was quite straightforward to accomplish. A real TV set has a remote control too, but my netbook is not equipped with an IrDA port, so I had to think about a different solution. My BlackBerry is equipped with a cool app called BBSSH, which allows you to connect to any SSH server, this could be a good starting point to implement a remote control device. Then, I had to write an interface to Totem using Python and DBus binding to allow my SSH remote control to issue commands to start and pause the player. I published a first working implementation: at the moment the only supported operations are play, pause, stop and volume adjustments, but there are a lot of opportunities by implementing full MPRIS specification. Just to avoid typing a long command each time, I finally created a set of bash aliases, one for each command, so I just have to type one letter to have the desired function, as much as a real remote control

17 September 2012

Today I ll be telling you about an interesting way to build your Debian packages using Deb-o-Matic, a tool developed and maintained by Luca Falavigna. Some more details about this tool from the package s description:

Deb-o-Matic is an easy to use build machine for Debian source packages based on pbuilder, written in Python.
It provides a simple tool to automate build of source packages with limited user interaction and a simple configuration. It has some useful features such as automatic update of pbuilder, automatic scan and selection of source packages to build and modules support.

The setup.
1. Download the package.

apt-get install debomatic

2. Modify the main configuration file as follows:

[default]
builder: pbuilder
packagedir: /home/john/debomatic # Take note of the following path since we'll need it for later use.
configdir: /etc/debomatic/distributions
pbuilderhooks: /usr/share/debomatic/pbuilderhooks
maxbuilds: 3 # The number of builds you can perform at the same time.
inotify: 1
sleep: 60 # The sleep time between one build and another.
logfile: /var/log/debomatic.log
[gpg]
gpg: 0 # Change to 1 if you want Deb-O-Matic to check the GPG signature of the uploaded packages.
keyring: /etc/debomatic/debomatic.gpg # Add the GPG Keys you want Deb-O-Matic to accept in this keyring.
[modules]
modules: 1 # A list of all the available modules will follow right after.
modulespath: /usr/share/debomatic/modules
[runtime]
alwaysupdate: unstable experimental precise
distblacklist:
modulesblacklist: Lintian Mailer
mapper: 'sid': 'unstable',
'wheezy': 'testing',
'squeeze': 'stable'
[lintian]
lintopts: -i -I -E --pedantic # Run Lintian in Pedantic mode.
[mailer] # You need an SMTP server running on your machine for the mailer to work. You can have a look at the 'Ssmtp' daemon which is a one-minute-setup MTA, check an example over here.
fromaddr: debomatic@localhost
smtphost: localhost
smtpport: 25
authrequired: 0
smtpuser: user
smtppass: pass
success: /etc/debomatic/mailer/build_success.mail-template # Update the build success or failure mails as you wish by modifying the relevant files.
failure: /etc/debomatic/mailer/build_failure.mail-template
[internals]
configversion = 010a

The available modules are:

Contents , which acts as a dpkg -c over the built packages.

DateStamp , which displays build start and finish times into a file in the build directory.

Lintian , which stores Lintian output on top of the built package in the pool directory.

Mailer , which sends a reply to the uploader once the build has finished.

PrevBuildCleaner , which deletes all files generated by the previous build.

Repository , which generates a local repository of built packages.

2. Configure dput to upload package s sources to your local repository, edit the /etc/dput.cf file and add this entry:

[debomatic]
method = local
incoming = /home/john/debomatic

or the following if you are going to upload the files to a different machine through SSH:

30 June 2012

Already chilled by Wheezy freeze? It s been a long ride since the release of Squeeze, and your beloved FTP Team tried to assist our tireless developers and contributors at its best. Here are some hot stats to give you a figure about what happened behind the scenes.
Since the release of Squeeze, 7462 .changes files with NEW components were processed by dak, with an average of 14.660 NEW packages per day. On the FTP Team side, we had 6877 accepts (13.511 per day), 641 rejects (1.259 per day) and 280 comments to maintainers (0.550 per day). This table represents the activity by single team member:

Login

Accepts

Rejects

Comments

ansgar

407 accepts (0.800 per day)

71 rejects (0.139 per day)

53 comments (0.104 per day)

dak

12 accepts (0.024 per day)

1 rejects (0.002 per day)

0 comments (0.000 per day)

dktrkranz

4319 accepts (8.485 per day)

381 rejects (0.749 per day)

104 comments (0.204 per day)

joerg

100 accepts (0.196 per day)

12 rejects (0.024 per day)

1 comments (0.002 per day)

mhy

214 accepts (0.420 per day)

14 rejects (0.028 per day)

5 comments (0.010 per day)

stew

67 accepts (0.132 per day)

16 rejects (0.031 per day)

7 comments (0.014 per day)

tolimar

1480 accepts (2.908 per day)

93 rejects (0.183 per day)

84 comments (0.165 per day)

twerner

278 accepts (0.546 per day)

53 rejects (0.104 per day)

26 comments (0.051 per day)

Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:

Clint rocks with tons of Haskell packages, followed by Jonas (mostly Perl packages), and Ben (kernel uploads). Italian cabal stands still, with Alessio and David respectively at 6th and 7th place
How long does a package stay in NEW? Some more, some less, but the average is 3 days, 15 minutes and 21 seconds. Now go and check your dak mails to see whether you had a fast processing or not
liblog4ada 1.2-1 surely had, as it was accepted after 30 seconds! gsoap 2.7.17-1 was not so lucky, it took 103 days, 8 hours, 20 minutes and 43 seconds to clear NEW, but then made its way to the archive. Better late than never
FTP Team is not just accepting NEW packages, but also removing obsolete ones. Here are some details about this task:

8 April 2012

If you are one of those brave soulschasing off spam from Debian mailing lists, you will probably get bored of clicking on the Spam radio buttons a lot of times, and will probably look for a quick shortcut. I submitted a patch in bug #640309, but as of today, it has not been applied yet.
Waiting for the patch to be applied, I prepared a little Greasemonkey script called Debian Mass Spam that allows to mark every message as Unsure , Ham , Inappropriate , or Spam by clicking the correspondent key on the keyboard:
Just press Z (for Unsure ), X (for Ham ), C (for Inappropriate ), or V (for Spam ), and you will see every radio button to change its status. I tested it on Chromium, Iceweasel (with xul-ext-greasemonkey package) and Epiphany (with epiphany-extensions package), and it seems to work properly.
Too many words and no actions so far, go and kill some spam now!

27 January 2012

Josselin Mouette is one the leaders of the pkg-gnome team, he takes sound technical decisions and doesn t fear writing code to work-around upstream issues. He deserves kudos for the work he has put into packaging GNOME over the years. He can also be very sarcastic (sometimes he even enjoys participating to flamewars on debian lists), and there are quite a few topics where we have long agreed to disagree. But this kind of diversity is also what makes Debian a so interesting place
Read on to learn more about the pkg-gnome team, its plans for Wheezy, Josselin s opinion on the GNOME 3 switch, and much more.
Raphael: Who are you?
Josselin: I am a 31 years old Linux systems engineer. I started in life with physics, which I studied at the ENS Lyon. I started a thesis on experimental and numerical models for optoelectronics, but when it became clear that research was not for me, I abandoned it and accepted a job at the CEA, which holds the largest computing center in Europe. Working on these machines has been the most awesome job ever (except for it being near Paris). After that I worked a bit on system monitoring technologies.
I am married, currently living in Lyon, and working for EDF (the French historical electricity company) on scientific workstations using Debian. EDF is using Debian on more than a thousand workstations and holds the fastest Debian supercomputer in the world (200 Tflops), which makes it another obvious place for Debian developers.
Raphael: How did you start contributing to Debian?
Josselin: I discovered Debian in 1999 while studying at the ENS, which is one of the biggest nests of Debian developers while being a small place, it is producing almost one Debian developer per year on average. After wondering for a while what it could be useful for, hacking on a slink snapshot made me think that it was for, well, everything except for gaming. Later, in 2002, when I was working on optoelectronics computing codes, I started to package them for Debian in order to make them easier to install, for us as well as other labs over the world. I started the NM process, and it was going smoothly but also going to take time. However, at that moment, the frozen-bubble game went out and made quite some buzz. Since I knew a guy who knew the game s developer, he asked me to package it. The package found 3 sponsors in a very short time and was fast-tracked into the archive at a speed that was unseen before. After which the NM process was completed very quickly.
At that time, I was a heavy WindowMaker user, but I didn t like the direction the project was taking (actually, I wonder if there was one). GNOME was starting to become attractive, but its packaging in Debian was very ineffective, with many inconsistent packages maintained by people who didn t ever talk to each other some of them didn t speak English, and some of them didn t talk at all. Together with awesome people, among which Jordi Mallach, Gustavo Noronha Silva, JHM Dassen, Ross Burton and S bastien Bacher, we started the GNOME team in 2003, introducing consistent packaging practices, and initiating synchronized uploads. Releasing a completely integrated GNOME 2.8 in sarge was a considerable achievement; proving (together with the Perl team) that a team was the best way to maintain large package sets changed the way people work on Debian.

Proving [ ] that a team was the best way to maintain large package sets changed the way people work on Debian.

Raphael: You re one of the most active contributors of the team which is packaging GNOME for Debian. What would you suggest to a new contributor who would like to help the team?
Josselin: There are several ways to contact the team, but the recommended one has always been IRC. We hang on #debian-gnome on the OFTC network, so just come around and ask for us. The real question is what you want to do in the team. Of course, most new volunteers want to help packaging the latest and greatest version of GNOME into unstable as soon as possible, but unless they already have Debian background, this is not the easiest task. Since there are already people working on this, the big packages are usually waiting on dependencies.
I used to direct newcomers towards bug triage, but it is a tedious task and I m now convinced that our huge bug backlog will never be dealt with. The most useful thing to do for newcomers now is probably to find a GNOME or GNOME-related package that needs improvement or is lagging behind, and simply try to work on it. You can also come and fix the bugs you find annoying. Find a patch on the GNOME bugzilla, or cook it yourself, propose it, and if it s worthy enough you ll soon get commit access.

Our huge bug backlog will never be dealt with.

At this point I feel worth mentioning that if no one answers in 10 minutes, it doesn t mean that no one will answer in 2 hours, so please stay on the channel after asking.
Raphael: There s been some controversy about GNOME 3 and the direction that the project is taking. What s your personal stance on GNOME 3? And what s the position of the pkg-gnome team?
Josselin: The controversy is not new to GNOME 3, but the large-scale changes made with it have put it more prominently. The criticism usually boils down to a few categories:

General lack of configurability

Strange design decisions

Red Hat centric development

Hardware requirements

Change resistance

The lack of configuration options has been an ongoing criticism since GNOME 2.0 has decided to rip off most of them. Of course, when the control center was redesigned again for 3.0, there was a surge of horrified exclamations from people who missed their favorite buttons. On this topic, I fully concur with GNOME developers. The configuration option that is useful for you is not necessarily useful for someone else. Of course, sometimes developers go a bit too far, but the general direction is right. At work, we found that only a minority of users actually configure anything on their desktops: they just want something that works to launch their applications. Apple and Google have sold millions of devices by making them the simplest possible and without any configuration.
Design decisions are, on the contrary, individual decisions, and each of them, while having reasons behind it, can be questioned. I remember seeing a lot of complaints when the OK and Cancel buttons were reversed in dialog boxes, something that nobody questions anymore. GNOME Shell is full of such changes; some are easy to get accustomed with, some others just make eyebrows raise. The most obvious example is the user menu in GNOME 3.2, which contains an entry to configure your Google account, but no entry to shutdown the computer. Both decisions were taken independently, each of them with (good or bad) reasons, but the result is simply ridiculous. The default configuration in Debian will contain an extension to make it a bit better, but on the whole we don t intend to diverge from the upstream design, on which a lot of good work has been done.

On the whole we don t intend to diverge from the upstream design, on which a lot of good work has been done.

Point 3 is more complex. Red Hat being the company spending the most on GNOME, it is obvious that their employees work on making things work for their distribution. An example is the recurring discussions about relying on system services that are currently only implemented by systemd. Since there is a lot of (mostly unjustified) resistance against systemd in Debian, and since it won t work on kFreeBSD anyway, someone needs to develop an alternative implementation of these services for upstart and sysvinit. Everything is in place for someone else to do the job but it has to be done, and this can be frustrating. Especially since it can also be hard to integrate changes needed for other distributions .
Hardware requirements are mostly a consequence of the previous criticism: there s hardware that most distributions just don t want to bother supporting. We ve seen it in squeeze with the introduction of a hard dependency on PulseAudio. The Debian GNOME team (together with the Gentoo maintainers) made this dependency optional, carrying heavy patches, in order to cover the cases where it does not work. Now that it has gained more maturity, making this effort obsolete, the new tendency is to require 3D acceleration. For various reasons, it is not available to everyone . On this matter, the position of the Debian GNOME team has always been to support as much different configurations as possible with reasonable effort. Thanks to efforts from the incredible Vincent Untz, upstream supports a so-called fallback mode , which is the GNOME panel from 2.x with a lot of its bugs fixed. We intend to support this mode for as long as reasonably possible in Debian, possibly even after upstream ends up dropping it. However, other applications are going to require 3D because GStreamer is moving to clutter too, affecting video playback performance on non-accelerated systems . For epiphany this is not a problem; only embedded video will be affected. But for totem, this is a major issue; because of that we will probably keep totem 3.0 in wheezy.
Finally, there is a natural human tendency to dislike change (I have it too), and it applies a lot to desktop users habits. Needless to say a change of such a scale as introducing GNOME Shell can trigger reactions. However, I don t think it is reasonable, because of this resistance, to keep gnome-panel 2.x in Debian. This would be a lot of work on obsolete technology, and would prevent the upcoming removal of a lot of deprecated libraries. This time is much better spent improving gnome-panel 3.x in Debian and keeping the fallback mode great. One of the change that was made in Debian was to make it easier to find, being available as GNOME Classic directly from the login manager, instead of having to find it in an obscure configuration panel. In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it. I had never been accustomed to a new environment as quickly ever before.

In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it.

Having seen several of my GDM patches reverted without a warning, I know we are not finished with carrying patches in Debian packages.
Scientific workstations are a non-trivial example, since there is a measurable effect of using 3D in the window manager on heavy 3D applications.
On the other hand, on accelerated systems, this feature should end up improving performance a lot.
Raphael: What are your plans for Debian Wheezy?
Josselin: The first goal of the GNOME team is, of course, to provide again a great desktop environment to work on. For wheezy it will probably be based on GNOME 3.4. There also needs to be some work on package management interfaces. Upstream bases everything on PackageKit, but it is not as featureful as the aptdaemon Ubuntu technology. If I have time, I would also like to improve HTTP proxy support, since currently it is based on a stack of terrible hacks.
Raphael: If you could spend all your time on Debian, what would you work on?
Josselin: Obviously I would like to make GNOME in Debian even better. That would imply working on underneath dependencies (what we now like to call plumbing) to make sure everything is working great. This would also imply working more as GNOME upstream to make it more suitable for our needs.
I would also work on large-scale improvements on the distribution, like conditional recommends which I d love to see implemented , or automatic build-dependency generation. I would also work on the installer to make it better for desktops machines.
The idea is to automatically install language packs, or glues between two packages when both packages are installed.
Raphael: What s the biggest problem of Debian?
Josselin: The obvious answer is the same as the one most people you interviewed before gave: not enough members in core teams. A lot of developers join Debian to work on a small number of pet packages, and don t necessarily want to be involved with existing teams. It is probably still not obvious enough that the primary way to start contributing to Debian is to join an existing team.
But if there is one thing that is preventing Debian from gaining more momentum now, it is a completely different one: the too short support timeframe. 3 years is really not enough for corporate users. One year to migrate from one version to another is too short, and it is not possible to skip a release. It is definitely possible to change that with reasonable effort: the long-term support after 3 years doesn t have to cover the same perimeter as the short-term one. For example, we could upgrade the kernel to the version in the current stable release, and stop fixing all non-remote security holes. The important thing is to cover the most basic needs: companies are ready to take the risk of having less support if it allows skipping a version, but not the risk of having no support at all. And even more important is to say that you do something. Red Hat says they support a release for 10 years, but of course after 5 years the supported perimeter is extremely small.

3 years [of support] is really not enough for corporate users.

Long-term support will not magically fix all problems in Debian, but it will bring more corporate users into the picture. And with corporate users come paid Debian developers, who can work on critical pieces of the system. Debian was built on the synergy between individuals and companies, and in recent years perhaps as a reaction against what happened with Ubuntu we ve kind of forgot the latter. A lot of individuals have joined the project, and they are actively working, for example, on shortening the release cycle, which goes against the interest of professionals. We should embrace again such users and developers, and that means adapting to the current needs of larger entities.
Raphael: You re the maintainer of python-support, a packaging helper that was competing with python-central. Both helpers are now deprecated in favor of dh_python2. Does this mean that the situation of Python in Debian is now sane? Or are there remaining problems?
Josselin: dh_python2 (and the Python3 version, dh_python3) has a sane enough design. It fixes a lot of issues in python-central and also python-support, at the expense of somehow reduced functionality for developers. However, just like the previous tools, it merely works around design mistakes in the Python interpreter. For example it is not possible to split binary modules, pure-Python modules and byte-compiled modules in different directory trees, like Perl does although PEP 3147 introduces a way to do so. There is still no sane and standardized way to deal with module versions. There is no difference made between the module (which is a part of language semantics) and the file containing it (an information which depends on the implementation). Developers heavily rely on introspection features and make assumptions based on the implementation, that make it impossible to work around problems with module files.
Such problems are not restricted to Python. Those who fought against Ruby gems could tell even worse stories. While introducing GObject introspection packages in Debian (they can be used in JavaScript and Python to provide modules based on GObject libraries), I was pleased to see a clear distinction between file and module, but I was again struck by the fact you are not forced to declare API versions in your Python/JS code. In all cases, there is no reliable way to detect runtime dependencies in a given Python or JavaScript file, which leaves the maintainer to declare them by hand, and of course, often be wrong about them. Add to that the fact that most errors cannot be detected before runtime. For all these reasons, and while still being fond of Python for scripts and prototyping, I ve become really skeptical of using purely interpreted languages to write real applications. Some GNOME developers are moving away from Python and JavaScript, mostly towards Vala; I can only approve of that move and hope the same happens to other projects.
Raphael: Is there someone in Debian that you admire for their contributions?
Of course there is the never-sleeping, never-stopping, Michael Biebl who can upload a whole GNOME release in a single week-end. But there are a lot of awesome people who make Debian something that simply works. I could talk about Cyril Brulebois from the X strike force, Julien Cristau from the release team, Sjoerd Simons for his sound advice and work on plumbing, Luca Falavigna who is so fast at processing NEW, to quote only a few of those I work with frequently. And of course, Jordi and Sam for their humor.
Thank you to Josselin for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that you can find older interviews on http://wiki.debian.org/PeopleBehindDebian.

1 November 2011

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (130.30 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects.
Dpkg work
The month started with fixing newly reported bugs to prepare the 1.16.1.1 release:

#644492: there was a flaw in a change I made to the trigger setup code. This resulted in packages being incorrectly marked as configured while they were only unpacked in a new chroot.

#644412: the Makefile snippet buildflags.mk did not respect the new maintainer specific environment variables (like DEB_CFLAGS_MAINT_APPEND) because make does not forward environment variable through $(shell ). Fixed that by manually exporting the required variables.

I also disabled dpkg-buildpackage s output of the build flags since it was confusing several maintainers. dpkg-buildpackage invokes debian/rules and it has no (clean) way to discover the build flags changes that maintainer request by setting the dedicated environment variables in debian/rules. Maintainers expect to see the build flags with the modifications they have requested and not just the default values set by the distribution.

With the help of Guillem, we decided on a proper fix for a race condition sometimes triggered by parallel builds when 2 concurrent dpkg-gencontrol try to update debian/files (see #642608). This ended up requiring a new package (libfile-fcntllock-perl) that the Debian perl team kindly packaged for us. With all this sorted, it was a rather easy fix.
Multiarch progress
I also spent lots of time on multiarch. I fixed an old bug that requested to support the multi-arch paths in case of cross-building (see #595144), the discussion was not really conclusive on which of the two proposed patches was better so I ended up picking my own patch because it was closer to how we currently deal with cross-building. Then I fixed 2 issues that have been reported on Ubuntu s dpkg. The first one (LP #863675) was rather severe since an installed package ended being disappeared in favor of its foreign counterpart that was removed (but that had some config files left). The second one (LP #853679) only affected dselect users (apparently there are still some!) who had a self-conflicting library (Provides: foo, Conflicts: foo) installed for multiple architectures.
But the bulk of the time spent on multiarch has been spent discussing with various parties on how to go forward with multiarch. The release team commented on the schedule of the merge to ensure it makes it into Wheezy, and the Debian project leader also commented on the problems encountered so far.
While not the best course of action I could have hoped for, it certainly helped since Guillem started pushing some reviewed commits. Out of the 66 commits that were in my pu/multiarch/full branch one week ago, 20 have been merged in the master branch already.
Python-django security update and RC bug
Since python-django s maintainer did not manage to prepare the required security updates, I stepped in and prepared version 1.2.3-3+squeeze2 for Squeeze and 1.0.2-1+lenny3 for Lenny. Unfortunately this security update is an example of how an inactive maintainer is likely to result in a severe delay for the release of security updates.
Furthermore in this specific case, the security team did not want to release the Squeeze security update until the Lenny one had been investigated (which required some time since upstream no longer supports the version in Lenny) but they did not make this very clear.
Later another release critical bug had been filed against the package (#646634) but after investigation, it turned out to be a local configuration problem so I downgraded it. I still forwarded the test suite failure to upstream authors since the test could be enhanced.
In any case, co-maintainers for python-django are welcome. I really preferred the situation where I can quietly sit down as backup maintainer
WordPress packaging
WordPress sounds similar to python-django. I m also only a backup maintainer but Giuseppe has been inactive for many months and I had to step in August because I wanted the new upstream version. I discovered a bit late that I was not subscribed to wordpress bugs and thus the release critical bug #639733 (that I introduced with my new upstream version) went unattended for a rather long time. Once aware, though, I quickly fixed it.
I also took the opportunity to start a discussion on debian-devel about how to deal with embedded javascript libraries and proposed a mechanism of opportunistic replacement with symlinks . WordPress is my testbed package for this mechanism, you can check out its debian/dh_linktree that implements the replacement logic.
The discussion has not been very interesting but at least I learned that Debian now requires that each source package shipping minified javascript files includes the original files too. It s somewhat of a pain since it s not a license requirement in many cases (many of those libraries are not under the GPL), but just a Debian requirement that many upstreams are not complying with. WordPress is affected and Jakub Wilk thus opened #646729 which is going to be a long-standing RC bug. To give good measures, I spent several hours investigating the case of each javascript file in the WordPress source package and I filed a new ticket on the upstream bugtracker.
Dropbox packaging work
A few months after the introduction of nautilus-dropbox to Debian and Ubuntu, I can say that the decision to only support the download of dropbox in the postinst has been a mistake. Because of this decision I had to make the postinst fail if the download failed. Even if the error message is relatively clear, this lead to many (mostly automated) bug reports on the Ubuntu side. Various other problems cropped up on top of this (trying to start dropbox while the package was not configured would result in an error because the user did not have the required rights to install the software, reinstalling the package while dropbox was running would result in a failure too, etc.).
I have fixed all those issues in the version 0.7.0-2 of the package. Now if the user has to install dropbox, it will use PolicyKit to request the root rights. The postinst will no longer fail if the dropbox download fails since it can be run later by the user. And I fixed the download code to remove the replaced file before unpacking a new file (insead of overwriting the existing file). All this work has been forwarded upstream.
The Debian Administrator s Handbook Update
I m glad to tell you that the translation will happen because we reached the minimal funding goal on October 22th with the help of 380 supporters.
Now the fundraising continues, but this time the goal is the liberation of the resulting book. For this to happen, we need to reach 25000 EUR in the liberation fund. So far we re at 37% of this goal with 9400 EUR in the liberation fund (which means that 59% of the money raised has been put in the liberation fund).

23 October 2011

We all love stats, don t we? So, here we go!
Let s start with a graph:
It shows the number of packages in the NEW queue since last year. You can see a big drop during April 2011, and a reasonably low rate during the last six months. You could think fellow Debian Developers stopped to upload NEW packages. Sorry, you re wrong!
Since Squeeze release, 3.832 .changes files with NEW components were processed by dak, with an average of 14,85 NEW packages per day. On the FTP Team side, we had 3.732 accepts (14,47 per day), 339 rejects (1,31 per day) and 178 comments to maintainers (0,69 per day).
Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:

Debian Haskell Group (362 packages)

Debian Perl Group (343 packages)

Debian Java Maintainers (161 packages)

Debian Ruby Extras Maintainers (124 packages)

Debian Multimedia Maintainers (100 packages)

Debian Fonts Task Force (96 packages)

Debian Med Packaging Team (79 packages)

Debian Install System Team (61 packages)

Debian Javascript Maintainers (54 packages)

Debian Python Modules Team (50 packages)

That s bad packaging teams cannot bake cookies!
Let s do the same with Changed By, this time:

Ben Hutchings (159 packages)

Joachim Breitner (138 packages)

Clint Adams (134 packages)

Jonas Smedegaard (124 packages)

TANIGUCHI Takaki (97 packages)

Nicholas Bamber (61 packages)

Alessio Treglia (60 packages)

maximilian attems (54 packages)

David Paleino (51 packages)

Torsten Werner (45 packages)

Much better now go and heat up your ovens, we know who you are
Another nice aspect to look at is the speed of NEW processing. Some maintainers were very happy for a fast NEW processing, someone even complained for having been too quick! So, let s find out which upload was the quickest ever. Try to gamble a bit before reading the answer, to see whether you are near to the real value
Alessio Treglia, you probably already know, because your gwc_0.21.16~dfsg-1 upload has been processed in 41 seconds (yes, forty-one seconds!). Here s an excerpt from ftp-master log to certify it:

7 October 2011

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (144.3 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects.
Dpkg work
While taking care of the last details for the hardening feature in dpkg 1.16.1, I have mailed debian-devel to find volunteers to handle a hardening release goal. The objective is to ensure a large number of packages have been converted/rebuilt to actually use the new hardening build flags.
Then I prepared the draft of the announce of the dpkg 1.16.1 upload (aka Bits of dpkg maintainers sent to debian-devel-announce) which got expanded by Guillem to also cover new features since dpkg 1.15.7.
update-alternatives got some refactoring by Guillem which resulted in a regression that has been fortunately discovered by Sven Joachim. I fixed that regression and did some further cleanup inspired by the root cause of this regression (see top 4 commits here).
Note that Sven is one of the few persons who are running the git version of dpkg. Hopefully the number of tester will increase since I recently documented the APT repositories with autobuilt versions of dpkg in the wiki.
At the end of the month, I started working on a bugfix release (what s going to be 1.16.1.1) by fixing some of the unavoidable problems discovered after an upload that accumulated more than 4 months worth of work (see top 4 commits here).
The Debian Administrator s Handbook
I spent countless hours finalizing the launch of the crowdfunding campaign for the Debian Administrator s Handbook and it went live on September 27th.
So far it s on good track with more than 63% of the base funding already secured. But we still have a long way to go to reach the liberation goal (we re at 21%). It s still worth nothing that more than 55% of the money raised has been put in the liberation fund so there are many persons who care about getting the book freed.
More than 250 persons are supporting the project currently with an average contribution of 38 EUR. I would have expected much less for the average contribution but many more supporters. I still hope we can get more people on board with the perspective of a good DFSG-free Debian ebook.
Did you order your copy? If not, click here and fix this! By the way Paypal used to be required but it s no longer the case, you can support the project just with your usual credit card.
Misc blog updates
Over time, I have written many useful articles for Debian users and Debian contributors. But scattered in the history, they are somewhat difficult to find. To fix this I have created some index pages listing them. Check them out:

Two new articles joined those pages this month: How to triage bugs in the Debian Bug Tracking System and Understand dpkg and don t get stuck with a maintainer script failure.
While writing the first article, I noticed we lacked a good page showing the most buggy packages so I quickly created it (with the help of UDD): http://qa.debian.org/cgi-bin/bugs-by-source
Misc packaging work
I did a small update to the developer s reference. Luca Falavigna submitted a patch to clarify how one is supposed to deal with meta-packages (cf #569219), I improved it and integrated the result in the SVN repository.
I upgraded nautilus-dropbox to version 0.6.9 and while doing this I discovered a bug in mergechanges (filed as #640782). I uploaded a new release of quilt mainly to add the Multi-Arch: foreign field so that it can satisfy dependencies of foreign packages (i.e. packages of a different architecture).
Django released some security advisories (tracked in #641405) and since the maintainer did not deal with the issue, I stepped up to the task (I m a backup maintainer) and released the fixed version 1.3.1 to unstable. I took the opportunity to switch from python-support to dh_python2, and do some misc improvements to the packaging (see changelog).
I wanted to update publican to a newer version but it turned out to be not possible because Debian doesn t have the latest version of docbook-xsl yet. I also discovered some bugs in the test suite and forwarded upstream the patch I created (see upstream bug). On top of this, fop was failing due to some java problem related to the introduction of multiarch. After having reported the bug, the java maintainers quickly released a fixed version.
So now publican is ready in the git repository but it s waiting on the docbook-xsl update. I got in touch with the maintainer who said he would have the time to take care of it by mid-october.
Thanks
See you next month for a new summary of my activities.