DebConf 10: Day 2

Stefano delivered an excellent address to the Debian project. As Project Leader, he offered a perspective on how far Debian has come, raised some of the key questions facing Debian today, and challenged the project to move forward and improve in several important ways.

He asked the audience: Is Debian better than other distributions? Is Debian still relevant? Why/how?

Having asked this question on identi.ca and Twitter recently, he presented a summary. There was a fairly standard list of technical concerns, but also:

A focus on quality, as defined by Debian’s highly modular approach. Each package maintainer is an expert on the software they package, and Debian as a whole offers a superior repository of packages.

The principles of software freedom, as embodied in Debian’s Social Contract. The Debian community’s current interpretation is a purist one, and Stefano cited the elimination of non-free firmware as a milestone in the upcoming Squeeze release. I wonder, though, how many of the audience, tapping away on WiFi-connected laptops, were able to do so without such firmware.

The project’s independent status, supported by donations and volunteers, which empowers it to make its own decisions, free of external impositions.

Debian’s ability to make decisions, as embodied in the constitution. This happens mostly through do-ocracy (individuals are empowered to decide questions concerning their own work), though larger scope issues are decided democratically. This one evoked a bit of a chuckle, as decision making in Debian is not always perceived as fully effective.

He pointed out some areas which we would like to see improve, including:

Developers accepting shared responsibility for the release as a whole. Making one’s own packages ready for release is necessary, but not sufficient. He cited evidence that the culture around NMUs is changing: historically, due to the do-ocratic system mentioned above, Debian developers have been somewhat territorial about their packages, and non-maintainer uploads were seen as stepping on their toes. However, recent experiments have indicated that this may no longer be the case, and Stefano encouraged more developers to help each other through NMUs.

When making decisions, we should seek consensus, not unanimity. In a project with thousands of contributors, whose operations are open to the public, there will never be unanimous support for a proposal, and seeking unanimity leads to stalled decisions.

In order to gain more contributors, Debian needs to welcome new and inexperienced contributors, as well as users (who can grow into contributors. He suggested reaching out to derivatives to find more of both. He decried the conventional wisdom that a “thick skin” should be a prerequisite for joining the project, pointing out that this attitude simply leads to fewer contributors. This point was met with applause by the DebConf audience.

All in all, I thought this was an accurate, timely and inspirational message for the project, and the talk is worth watching for any current or prospective contributor to Debian.

Russ facilitated a discussion about the Debian policy document itself and the process for managing it. He has recently put in a lot of time working on the backlog (down from 160+ to 120), but this is not sustainable for him, and help is needed.

There was a wide-ranging discussion of possible improvements including:

Editing the policy manual so that it is more readable start to finish as a document, rather than a reference

Creating a closer linkage between lintian and the policy manual, so that best practices from lintian get documented, and policy changes are accompanied by new checks

Separating the normative and informative parts of the policy manual

There was also some discussion in passing of the long-standing confusion (presumably among people new to the project) with regard to how policy is established. In Debian, best practices are first implemented in packages, then documented in policy (not the reverse). Sometimes, improvements are suggested at the policy level, when they need to start elsewhere. I’m not very familiar with how the policy manual is maintained at present, but listening to the discussion, it sounded like it might help to extend the process to include the implementation stage. This would allow standards improvements to be tracked all the way through from concept, to implementation, to documentation.

Torsten described the current state of Java packaging in Debian and the general problems involved, including licensing issues, build system challenges (e.g. maven) and dependency management. His slides were information-dense, so I didn’t take a lot of notes.

His presentation inspired a lively discussion about why upstream developers of Java applications and libraries often do not engage with Debian. Suggested reasons included:

They are not interested in Linux as a target platform

Although their code is released under a free license, they are not interested in meeting Debian standards for freedom and license correctness

They use Java because it is cross-platform, and so do not want to concern themselves with platform-specific issues

Because Java applications are easy to download and run manually, they perceive relatively little value in the Debian packaging system

Jorge talked about the connections between Debian and Ubuntu, how people in the projects perceive each other, and how to foster good relationships between developers.

He talked about past efforts to quantify collaboration between the projects, but the focus is now on building personal relationships. There were many good questions and comments afterward, and I’m looking forward to the Debian derivatives BoF session tomorrow to get into more detail.

Tonight is the traditional wine and cheese party. When this tradition started, I was one of just a handful of people in a room with some cheese and paper plates, but it’s now a large social gathering with contributions of cheese and wine from around the world. I’m looking forward to it.

Exactly, Java is crossplatfom, so very few applications provide a deb package. Even in those cases, the package contains all the libraries it needs, and requires no dependencies, not even the java runtime.
But the Java problem is a bit bigger than just that. Recreating the java library dependency information in deb package metadata is something nobody wants to do. Any simple Java app needs 10 libraries and each of them might need two or three additional ones. If you ever look at a Maven dependency graph you will not believe it.
And also the’s no way all the java apps will settle with the latest version of any common library. For example, all java apps use shared libraries like apache commons or log4j. If debian wants to package 2 big java apps like tomcat, jboss, netbeans, jedit, or squirrel I guess they will have to include 2 or 3 different versions of many of those small libraries, too.

Regarding the Java packaging nightmare, as someone who has used Java extensively for a long time, I’d like to suggest another reason why upstream developers of Java applications don’t engage with Debian (or any other Linux distro for that matter): packaging for Debian requires them to use a different tool chain than the one they are used to.

Java has evolved its own tool chain because the traditional UNIX tool chain (make in particular) was not well adapted to Java development. It started with Ant that was explicitly designed as a replacement for make that was adapted to the way Java works. It gradually evolved into the tools any Java developer takes for granted today: Ant, Maven and a collection of IDEs like Eclipse. Modern Java IDEs are like Emacs: operating systems that masquerade as text editors. Your average Java dev will start his IDE when he gets to work in the morning, shut it down when he goes home at night and do everything in between from the IDE: write code, build it, package it, deploy it to a remote server, commit it to svn repository, order pizza, start the coffee machine, etc.

So when you ask them to package for Debian, you ask them to leave their IDE and use a tool chain they don’t know and that, from afar, looks like a whole lot of pain: not a very tempting proposition.

I reckon that in order to solve this issue, the best way would be to create Debian packaging tools that are specialised for packaging Java applications and that leverage the standard Java packaging tools so that you don’t need those upstream developers to package specifically for Debian. Considering that Java packaging is quite consistent, it shouldn’t be too difficult to do (famous last words…) and I would be happy to help anybody who would want to do it: I can provide the Java expertise so if someone else can provide the Debian packaging expertise, I’m sure we can come up with a solution that reduces the pain.

Bruno is right. A good way to start it for debian to have a maven repo and then build an automatic tool that reads the maven dependency metadata and creates a .deb for the each application and it’s libs.

[…] unique set of issues because they tend to have their own packaging systems. Merely a month later he attended Debconf 2010 where a presentation was given on the issues that Java packages have on Dpkg based systems. Since […]