Search Results: "dktrkranz"

2 September 2015

I m receiving more requests for upload accounts to the Deb-o-Matic servers lately (yay!), but that means the resources need to be monitored and shared between the build daemons to prevent server lockups.
My servers are running systemd, so I decided to give systemd.resource-control a try. My goal was to assign lower CPU shares to the build processes (debomatic itself, sbuild, and all the related tools), in order to avoid blocking other important system services from being spawned when necessary.
I created a new slice, and set a lower CPU share weight:$ cat /etc/systemd/system/debomatic.slice
[Slice]
CPUAccounting=true
CPUShares=512
$
Then, I assigned the slice to the service unit file controlling debomatic daemons by adding the Slice=debomatic.slice option under the Service directive.
That was not enough, though, as some processes were assigned to the user slice instead, which groups all the processes spawned by users:
This is probably because schroot spawns a login shell, and systemd considers it belonging to a different process group. So, I had to launch the command systemctl set-property user.slice CPUShares=512, so all processes belonging to the user.slice will receive the same share of the debomatic ones. I consider this a workaround, I m open to suggestions how to properly solve this issue :)
I ll try to explore more options in the coming days, so I can improve my knowledge of systemd a little bit more :)

21 October 2014

This is an update of my previous attempt at summarizing this discussion. As I proposed one of the amendments, you should not blindly trust me, of course. :-)
First, let s address two FAQ:
What is the impact on jessie?
On the technical level, none. The current state of jessie already matches what is expected by all proposals. It s a different story on the social level.
Why are we voting now, then?
Ian Jackson, who submitted the original proposal, explained his motivation in this mail.
We now have four different proposals: (summaries are mine)

[iwj] Original proposal (Ian Jackson): Packages may not (in general) require one specific init system (Choice 1 on this page)

[lucas] Amendment A (Lucas Nussbaum): support for alternative init systems is desirable but not mandatory (Choice 2 on this page)

[plessy] is the simplest, and does not discuss the questions that the other proposals are answering, given it considers that the normal Debian decision-making processes have not been exhausted.
In order to understand the three other proposals, it s useful to break them down into several questions.
Q1: support for the default init system on Linux
A1.1: packages MUST work with the default init system on Linux as PID 1.
(That is the case in both [iwj] and [lucas])
A1.2: packages SHOULD work with the default init system on Linux as PID 1.
With [dktrkranz], it would no longer be required to support the default init system, as maintainers could choose to require another init system than the default, if they consider this a prerequisite for its proper operation; and no patches or other derived works exist in order to support other init systems. That would not be a policy violation. (see this mail and its reply for details). Theoretically, it could also create fragmentation among Debian packages requiring different init systems: you would not be able to run pkgA and pkgB at the same time, because they would require different init systems.
Q2: support for alternative init systems as PID 1
A2.1: packages MUST work with one alternative init system (in [iwj])
(Initially, I thought that one here should be understood as sysvinit , as this mail, Ian detailed why he chose to be unspecific about the target init system. However, in that mail, he later clarified that a package requiring systemd or uselessd would be fine as well, given that in practice there aren t going to be many packages that would want to couple specifically to systemd _or_ uselessd, but where support for other init systems is hard to provide.)
To the user, that brings the freedom to switch init systems (assuming that the package will not just support two init systems with specific interfaces, but rather a generic interface common to many init systems).
However, it might require the maintainer to do the required work to support additional init systems, possibly without upstream cooperation.
Lack of support is a policy violation (severity >= serious, RC).
Bugs about degraded operation on some init systems follow the normal bug severity rules.
A2.2: packages SHOULD work with alternative init systems as PID 1. (in [lucas])
This is a recommendation. Lack of support is not a policy violation (bug severity < serious, not RC). A2.3: nothing is said about alternative init systems (in [dktrkranz]). Lack of support would likely be a wishlist bug.
Q3: special rule for sysvinit to ease wheezy->jessie upgrades
(this question is implicitly dealt with in [iwj], assuming that one of the supported init systems is sysvinit)
A3.1: continue support for sysvinit (in [lucas])
For the jessie release, all software available in Debian wheezy that supports being run under sysvinit should continue to support sysvinit unless there is no technically feasible way to do so.
A3.2: no requirement to support sysvinit (in [dktrkranz])
Theoretically, this could require two-step upgrades: first reboot with systemd, then upgrade other packages
Q4: non-binding recommendation to maintainers
A4.1: recommend that maintainers accept patches that add or improve
support for alternative init systems. (in both [iwj] and [lucas], with a different wording)
A4.2: say nothing (in [dktrkranz])
Q5: support for init systems with are the default on non-Linux ports
A5.1: non-binding recommendation to add/improve support with a high priority (in [lucas])
A5.2: say nothing (in [iwj] and [dktrkranz])
Comments are closed: please discuss by replying to that mail.

25 May 2014

I ve been playing with qemu-user-static a bit to create a set of porterboxes for my Deb-o-Matic build farm. After reading gregoa s post on how to create cross-chroot with qemu-debootstrap, I was immediately able to create armel, armhf and powerpc boxes with very little efforts.
I tried to extend the number of porterboxes available by adding mips* and s390x, in order to have all the Linux-based architectures supported in Jessie, but with no luck. Here s a summary of my attempts.
MIPS*
Chroot creation fails under both mips and mipsel trying to configure libuuid1. The problem is due to the fact libuuid1 s postinst script calls groupadd and useradd. Those two utilities rely on NETLINK sockets, which apparently are not handled by QEMU at the moment. I raised the question upstream to see whether it is possible to solve this problem.
s390x
Chroot creation used to fail with a SIGSEGV. This particular bug has been fixed recently, but it seems it s not enough to have a working chroot. It fails with some gzip errors, probably because some portions of dpkg-deb are not fully covered by qemu-s390x-static.
Preparing to unpack .../base-files_7.3_s390x.deb ...
Unpacking base-files (7.3) ...
dpkg-deb (subprocess): decompressing archive member: internal gzip read error: '<fd:5>: incorrect data check'
/bin/tar: Unexpected EOF in archive
/bin/tar: Unexpected EOF in archive
/bin/tar: Error is not recoverable: exiting now
dpkg-deb: error: subprocess tar returned error exit status 2
dpkg: error processing archive /var/cache/apt/archives/base-passwd_3.5.33_s390x.deb (--install):
subprocess dpkg-deb --control returned error exit status 2

19 September 2013

I ve been running several Deb-o-Matic instances on several servers since ages. This service, meant to provide a way to automate the build of Debian source packages, has been the starting point for a lot of Italian developers to start packaging, and eventually become Debian or Ubuntu developers. I can t exactly tell how many packages Deb-o-Matic servers have built so far, but it has been the primary build and test environment for at least three Debian Developers since four years.
This service was hosted for free since the beginning, but the good things always come to an end. My provider decided they cannot keep our servers running (its core business is totally unrelated to hosting servers), and asked me to shut them down. I started with the two kfreebsd-* machines, and soon I will shut down i386 and amd64 as well. When all services will be down, I ll take care of removing the debian.net domains I created for them.
Deb-o-Matic development won t stop, but developers who used Deb-o-Matic daily will have to find a different solution to build and test their packages.

11 March 2013

While reading the platforms for the upcoming DPL election, I read some statements about hearing from Project members about their experience within the Debian Project, so I took the chance to write some thoughts about my experience in one of our core teams, in which I have the privilege to be part of: the FTP Team.
During NM process, my Application Manager asked me about some activities I could have involved into once I became a Debian Developer. I was fascinated by the hard work of the FTP folks when processing NEW packages (and I guess everybody reading these lines agree with me, remembering the happines of reading that wonderful ACCEPTED mail for their very first contribution to Debian, I still have it in my mailbox), so I decided I wanted to help out those folks to share a portion of their hard work.
I committed errors, sometimes they were so silly I d be better to hide, but the team helped me to solve the mess I created. The thank you and kudos messages the FTP Team receives greatly overtake the frustration for my mistakes, and give the strength to keep up the good work
FTP Team is just one of the dozens teams which do an awesome work for the Debian sake, every day. Don t be shy, or worried of being unsuitable or not skilled enough for the role. You ll never know until you try. And it could happen you really like the job, and you could feel sorry for have waited so long

30 December 2012

I don t like watching videos sitting in front of a computer, I usually store them into a USB key to plug into my Samsung LE37 TV set. It s not an easy task, though, as I have to fight with other components of my family who usually like watching something completely different than DebConf videos I could have invested 200 to buy a new one, but I tried a different approach first.
Recycling a netbook
I received a Tweety device as Christmas gift, and I tried to connect it to my netbook to see how it worked. I was quite impressed by the good quality of the sound (if compared to the really poor quality of the netbook speakers), and I realized I could have recycled the netbook as a brand new TV set with some little adjustments. I plugged my SyncMaster 920n PC monitor into the netbook VGA port, switched Totem to fullscreen mode, and here it is my TV set!
Remote control
OK, that was quite straightforward to accomplish. A real TV set has a remote control too, but my netbook is not equipped with an IrDA port, so I had to think about a different solution. My BlackBerry is equipped with a cool app called BBSSH, which allows you to connect to any SSH server, this could be a good starting point to implement a remote control device. Then, I had to write an interface to Totem using Python and DBus binding to allow my SSH remote control to issue commands to start and pause the player. I published a first working implementation: at the moment the only supported operations are play, pause, stop and volume adjustments, but there are a lot of opportunities by implementing full MPRIS specification. Just to avoid typing a long command each time, I finally created a set of bash aliases, one for each command, so I just have to type one letter to have the desired function, as much as a real remote control

17 September 2012

Today I ll be telling you about an interesting way to build your Debian packages using Deb-o-Matic, a tool developed and maintained by Luca Falavigna. Some more details about this tool from the package s description:

Deb-o-Matic is an easy to use build machine for Debian source packages based on pbuilder, written in Python.
It provides a simple tool to automate build of source packages with limited user interaction and a simple configuration. It has some useful features such as automatic update of pbuilder, automatic scan and selection of source packages to build and modules support.

The setup.
1. Download the package.

apt-get install debomatic

2. Modify the main configuration file as follows:

[default]
builder: pbuilder
packagedir: /home/john/debomatic # Take note of the following path since we'll need it for later use.
configdir: /etc/debomatic/distributions
pbuilderhooks: /usr/share/debomatic/pbuilderhooks
maxbuilds: 3 # The number of builds you can perform at the same time.
inotify: 1
sleep: 60 # The sleep time between one build and another.
logfile: /var/log/debomatic.log
[gpg]
gpg: 0 # Change to 1 if you want Deb-O-Matic to check the GPG signature of the uploaded packages.
keyring: /etc/debomatic/debomatic.gpg # Add the GPG Keys you want Deb-O-Matic to accept in this keyring.
[modules]
modules: 1 # A list of all the available modules will follow right after.
modulespath: /usr/share/debomatic/modules
[runtime]
alwaysupdate: unstable experimental precise
distblacklist:
modulesblacklist: Lintian Mailer
mapper: 'sid': 'unstable',
'wheezy': 'testing',
'squeeze': 'stable'
[lintian]
lintopts: -i -I -E --pedantic # Run Lintian in Pedantic mode.
[mailer] # You need an SMTP server running on your machine for the mailer to work. You can have a look at the 'Ssmtp' daemon which is a one-minute-setup MTA, check an example over here.
fromaddr: debomatic@localhost
smtphost: localhost
smtpport: 25
authrequired: 0
smtpuser: user
smtppass: pass
success: /etc/debomatic/mailer/build_success.mail-template # Update the build success or failure mails as you wish by modifying the relevant files.
failure: /etc/debomatic/mailer/build_failure.mail-template
[internals]
configversion = 010a

The available modules are:

Contents , which acts as a dpkg -c over the built packages.

DateStamp , which displays build start and finish times into a file in the build directory.

Lintian , which stores Lintian output on top of the built package in the pool directory.

Mailer , which sends a reply to the uploader once the build has finished.

PrevBuildCleaner , which deletes all files generated by the previous build.

Repository , which generates a local repository of built packages.

2. Configure dput to upload package s sources to your local repository, edit the /etc/dput.cf file and add this entry:

[debomatic]
method = local
incoming = /home/john/debomatic

or the following if you are going to upload the files to a different machine through SSH:

30 June 2012

Already chilled by Wheezy freeze? It s been a long ride since the release of Squeeze, and your beloved FTP Team tried to assist our tireless developers and contributors at its best. Here are some hot stats to give you a figure about what happened behind the scenes.
Since the release of Squeeze, 7462 .changes files with NEW components were processed by dak, with an average of 14.660 NEW packages per day. On the FTP Team side, we had 6877 accepts (13.511 per day), 641 rejects (1.259 per day) and 280 comments to maintainers (0.550 per day). This table represents the activity by single team member:

Login

Accepts

Rejects

Comments

ansgar

407 accepts (0.800 per day)

71 rejects (0.139 per day)

53 comments (0.104 per day)

dak

12 accepts (0.024 per day)

1 rejects (0.002 per day)

0 comments (0.000 per day)

dktrkranz

4319 accepts (8.485 per day)

381 rejects (0.749 per day)

104 comments (0.204 per day)

joerg

100 accepts (0.196 per day)

12 rejects (0.024 per day)

1 comments (0.002 per day)

mhy

214 accepts (0.420 per day)

14 rejects (0.028 per day)

5 comments (0.010 per day)

stew

67 accepts (0.132 per day)

16 rejects (0.031 per day)

7 comments (0.014 per day)

tolimar

1480 accepts (2.908 per day)

93 rejects (0.183 per day)

84 comments (0.165 per day)

twerner

278 accepts (0.546 per day)

53 rejects (0.104 per day)

26 comments (0.051 per day)

Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:

Clint rocks with tons of Haskell packages, followed by Jonas (mostly Perl packages), and Ben (kernel uploads). Italian cabal stands still, with Alessio and David respectively at 6th and 7th place
How long does a package stay in NEW? Some more, some less, but the average is 3 days, 15 minutes and 21 seconds. Now go and check your dak mails to see whether you had a fast processing or not
liblog4ada 1.2-1 surely had, as it was accepted after 30 seconds! gsoap 2.7.17-1 was not so lucky, it took 103 days, 8 hours, 20 minutes and 43 seconds to clear NEW, but then made its way to the archive. Better late than never
FTP Team is not just accepting NEW packages, but also removing obsolete ones. Here are some details about this task:

8 April 2012

If you are one of those brave soulschasing off spam from Debian mailing lists, you will probably get bored of clicking on the Spam radio buttons a lot of times, and will probably look for a quick shortcut. I submitted a patch in bug #640309, but as of today, it has not been applied yet.
Waiting for the patch to be applied, I prepared a little Greasemonkey script called Debian Mass Spam that allows to mark every message as Unsure , Ham , Inappropriate , or Spam by clicking the correspondent key on the keyboard:
Just press Z (for Unsure ), X (for Ham ), C (for Inappropriate ), or V (for Spam ), and you will see every radio button to change its status. I tested it on Chromium, Iceweasel (with xul-ext-greasemonkey package) and Epiphany (with epiphany-extensions package), and it seems to work properly.
Too many words and no actions so far, go and kill some spam now!

18 February 2012

At FOSDEM, John Sullivan
delivered an interesting talk titled Is
copyleft being framed? to verify alleged claims on the decline
of GPL-d software. (Slides
are available.) The crux of the talk is the analysis he performed
on the Debian archive to discover the amount of software we
distribute that is covered by GPL, LGPL, or AGPL ("GPL-d" for short
in the remainder).
John's talk steps in an interesting and long running debate (a
recent summary of which is available in this
ITWire article). The most interesting part is the discrepancy
among John's results and Blackduck's, which are
often used to
argue how the popularity of the GPL license is declining. That
might be the case. Or not. The more analyses we do to find it out,
the better.
The underlying assumption on John's work is that Debian is a
representative sample of the Free Software out there, which I think
is a reasonable assumption. I find the analysis presented in the
talk completely satisfactorily from a purely scientific point of
view. The same cannot be said about Blackduck's result: both their
methods and data are secret, making it impossible to reproduce
their experiments. Highly unscientific.
Still, John's results are surprising: as much as 87 percent of
Lenny's packages and 93 percent of Squeeze's are GPL-d. That seems
a lot. Puzzled about that, John discussed with me the
issue before his talk, in search for pitfalls in his methods or
data. Finding none, I pointed him to the almighty DktrKranz for some extra
review; who found nothing either. To stay on the safe side, even
during his talk John called for independent reviews of his results.
What could be wrong?
The tool used to gather the data is
license-count from the debian-policy package.
Input data are the debian/copyright files of all
Debian source packages. If license-count is not
bugged, our debian/copyright files might be. One thing
that occurred to me only a few days ago is the habit of
declaring a different license for Debian packaging (the
files under debian/) than the software being packaged
itself. That's a bad habit because it might cause unwanted license
mixtures via patches that live under debian/ but I've
seen several occurrences of it in the Debian archive. For name and
(self-)shame: I've also been guilty of it in the past, when I
was young .
Is that reason enough to skew results and overestimate
GPL-d software? I don't think so, I hope not, but
ultimately I don't know. It'd be nice to rule out the possibility
entirely. So if anyone is willing to do some sampling of affected
debian/copyright files and propose patches for
license-count to exclude those "false positives",
please shout. (As a bonus point: that would also help to take more
sound decision for the typical use case of
license-count, i.e. deciding when a license should be
added to /usr/share/common-licenses.)
Other independent reviews of the results are equally
welcome.
Note: the above, as well as John's analysis, would be a trivial
exercise if DEP-5
were already widely deployed in the Debian archive.
Update: add link to John's slides
Update 19/02/2012: Russ Allbery, author of
license-count, posted a
way more likely cause of data skew in John's analysis: double
counting among the different types of copyleft licenses

23 October 2011

We all love stats, don t we? So, here we go!
Let s start with a graph:
It shows the number of packages in the NEW queue since last year. You can see a big drop during April 2011, and a reasonably low rate during the last six months. You could think fellow Debian Developers stopped to upload NEW packages. Sorry, you re wrong!
Since Squeeze release, 3.832 .changes files with NEW components were processed by dak, with an average of 14,85 NEW packages per day. On the FTP Team side, we had 3.732 accepts (14,47 per day), 339 rejects (1,31 per day) and 178 comments to maintainers (0,69 per day).
Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:

Debian Haskell Group (362 packages)

Debian Perl Group (343 packages)

Debian Java Maintainers (161 packages)

Debian Ruby Extras Maintainers (124 packages)

Debian Multimedia Maintainers (100 packages)

Debian Fonts Task Force (96 packages)

Debian Med Packaging Team (79 packages)

Debian Install System Team (61 packages)

Debian Javascript Maintainers (54 packages)

Debian Python Modules Team (50 packages)

That s bad packaging teams cannot bake cookies!
Let s do the same with Changed By, this time:

Ben Hutchings (159 packages)

Joachim Breitner (138 packages)

Clint Adams (134 packages)

Jonas Smedegaard (124 packages)

TANIGUCHI Takaki (97 packages)

Nicholas Bamber (61 packages)

Alessio Treglia (60 packages)

maximilian attems (54 packages)

David Paleino (51 packages)

Torsten Werner (45 packages)

Much better now go and heat up your ovens, we know who you are
Another nice aspect to look at is the speed of NEW processing. Some maintainers were very happy for a fast NEW processing, someone even complained for having been too quick! So, let s find out which upload was the quickest ever. Try to gamble a bit before reading the answer, to see whether you are near to the real value
Alessio Treglia, you probably already know, because your gwc_0.21.16~dfsg-1 upload has been processed in 41 seconds (yes, forty-one seconds!). Here s an excerpt from ftp-master log to certify it:

8 June 2011

I just uploaded gdebi_0.8~exp1 into experimental.
This release adds Michael Vogt s support for GTK3 (Thanks a lot, Michael!):
Testers are welcome, expecially if you are trying GNOME3 already. Feel free to report any bug you find!

1 May 2011

6 October 2010

Many people know Ultimate Debian Database already, and do use of it to create useful QA tools, or other Debian related activities. There is another database which is widely used by all of us, every day, and runs behind the scenes without being noticed. It is called projectb, the huge database of the Debian archive.
projectb is the place where dak stores all the information it needs to run the archive (including but not limited to source packages, related binaries, suite definitions and associations, overrides, and so on), and it tells dak to hit you with a REJECT if you try to upload a package twice, or if you do something The Wrong Way. OK OK, it eventually allows your packages to be accepted too, sometimes it also behaves good
projectb is hosted on ftp-master.debian.org. Most fellow Debian Developers cannot login over there because access is restricted to FTP and Release people only, but can enjoy its clone on merkel.debian.org [1]. Please note that clone is synced on a regular basis, but it could be a little bit not up-to-date with the very latest changes. Just run the following:
psql -d projectb -p 5433
Now that you have access to it, you could want look at its structure to start looking at it. I just provide a couple of trivial examples of what you can do with it, your imagination is the limit [2]
This query displays which dpkg version known suites ship:SELECT s.source, max(s.version) AS version, su.suite_name AS suite
FROM source s
JOIN src_associations sa ON sa.source = s.id
JOIN suite su ON su.id = sa.suite
WHERE s.source = 'dpkg'
GROUP BY s.source, su.suite_name
ORDER BY version;
This query displays changelog entries for debhelper which versions are greater than 8:SELECT changelog
FROM changelogs
WHERE source = 'debhelper'
AND version > '8';
Working with projectb means working with real Debian archive data, so you are sure to always get solid information.
You can also experience a limited (but functional) dak session on merkel. You cannot do much, data is read-only, but this is not a limit for a couple of commands. The most interesting is dak rm, where you can simulate a package removal, and discover reverse dependencies in advance, to save some work to your lovely FTP Team
Just run the following:dak rm -n -R -m "Don't worry, it's a Dry Run" your_behated_package
Wait a little bit, and you will be prompted with removal report. Just remember to always use -n and -R flags, though.
Want even more? Join FTP Team, and you will given the chance to learn a lot more about Debian!
DISCLAIMER: if you plan to write a tool, or perform an activity which relies on current projectb implementation, PLEASE TALK TO FTP TEAM FIRST. Internal structure can change a lot over times, and it is way better let FTP Team to handle exporting/parsing its data for day-by-day tasks.
[1] This is probably supposed to change, as merkel is going to be decommissioned soon.
[2] If you need a new feature, just talk to FTP Team.

I recently invested some time to work on several cool features in Deb-o-Matic.
I plan to release 0.9 version soon, so I wanted something new for its users
I merged some patches by Alessio Treglia, who implemented a couple of nice things:

Mailer support, to send build status notifications to the uploader

Blacklist support, to temporarily limit builds for a given distribution

Then, I worked on some changes I really wanted to have:

Rebuild support, to redo build of a package already in distribution archive

Now that the big part of the planned features is committed, I am going to test them to spot major issues, and add some minor improvements to have a shiny release
Last but not least, your help is appreciated to complete translations! There are really a few strings to process, and several languages are already half-completed, so if you can spend five minutes (or even less!), I will be grateful