What is Miklos hacking - Tags: frugalware

Given that Frugalware was founded by me, and recently James recently took over
the project leadership, I think it makes sense to look back and summarize the
past 6 years.

The beginning

It all started with
this
announcement about 6 years ago. It was a one-man show before, there was a
manually
written ChangeLog but even then there were already quite a few packages, so
don’t ask me when I started hacking on this. Yes, normally there would be a
first commit in git, but remember this was before git existed, and I hated
centralized CVS so much that we didn’t use anything.

Looking back, it was all quite lame. :-) I used a mail address called "mamajom"
(English translation could be "momonkey"), tied to an ISP, with a lengthy
signature at the end of every mail I sent and was using my IRC nick instead of
my real one everywhere… OTOH, I made some decisions I’m happy about even
today. The first four developers (Ádám Zlehovszky, Krisztián Vasas, Zsolt
Szalai and me) were all Hungarian and despite of this, I forced every code,
test and documentation to be in English, to possibly turn the project into an
international one in the future. And that proved to very, very useful.

It was between 0.1 and 0.2 that we (Krisztián, Zsolt and me) showed up in a
Hungarian TV (video record)
explaining Frugalware. This is something never happened later for some reason.

The way we found a free hosting at a university I never attended may be also
interesting. In the grammar school, our IT teacher found my Linux distribution
idea interesting enough to support, and his friend was a teacher at the Eötvös
Loránd University, which is at the same Budapest city where I live. So, with my
friend Botond Balázs (who later was my witness at our wedding) we bought an
ultra-slow old PC (some Pentium 1 with a single HDD) and we were extremely
happy, as the uni offered to host it for free. I remember it ran some Slackware
version, as Frugalware didn’t support i586… ;-)

Early days

Then some more developers joined and we started to package all the usual free
software which is available in other distributions but I personally didn’t use.
Think of GNOME, Xfce, OpenOffice.org and so on.

Of course we were still lame, the announcements still were not spellchecked by
someone native and knowledgeable enough, ending up in words like "splitted" or
"optimalization". However, we started to use an SCM (Darcs, which was horribly
slow, but at least not the centralized CVS crap). Another developer I should
mention was Bence Nagy, he came up with the idea of how we should avoid
duplication in FrugalBuilds, leading to the various package templates under
source/include/ in the source tree.

Then we went multiarch, Krisztián Hamar contributed the x86_64 port to the 0.3
release. At the same time, I invented syncpkg, which avoided having to build
the same package on multiple architectures manually.

We also got Gábor Lőcsei (later we met IRL and completed several bike marathons
together, resulting in a great friendship) who helped users in general on IRC,
but more importantly, for some time he did quite some bug triaging, leading to
a much better bugzilla (actually it was running Flyspray at that time) state.

The constant questions of László Csécsy (who nowadays mostly hacks
Drupal core and its modules) generated more and more improving documentation.
We first started with a LaTeX documentation, then later turned it to an
asciidoc one, in the hope of attracting more non-math contributors. ;-)

Then we started to have some beautiful artwork, Viktor Gondor contributed some
really cool wallpapers.

Finally, I think the last two Hungarian guys who spent a hell of a time on the
project was András Vöröskői (we still link his
getting started HOWTO for new
contributors) and János Kovács. I remember for a long time I used the
home-hosted server of János when we didn’t have a fast dedicated i686 build
server and my desktop was still 32bit.

Going international

Then things started to happen fast. Developers outside Hungary showed up:
Gabriel Craciunescu from Germany and Michel Hermier from France worked a lot on
KDE, Gabriel also hacked a lot of the core OS, like kernel, glibc, etc.

Priyank Gosalia from India started to work on GTK tools like a package manager
frontend. David Kimpe contributed a PPC port years before I got my iBook (which
I used as my primary box for a few years).

And needless to say, we still made mistakes. We thought that the development of
the original Pacman package manager slowed down, its rewrite (providing a
separate library and a console frontend, instead of one monolithic binary)
seemed to never complete, and given that we were not Archlinux developers, we
never got access to its CVS. We started to use that library API in the
installer, testcases, GUI tools, while changing the API for them wasn’t an
issue, since officially it was still unreleased. All this frustration (and
underestimating the cost of maintaining a fork) resulted in our Pacman-G2
project. First it appeared to be shiny, but once Krisztián Hamar left
Frugalware, we constantly lacked manpower and today the consensus is that with
git, maintaining a patchset over pacman.git would be far better, just nobody
did the work of merging the two projects. Pacman-G2 is in maintenance mode
today, and it still well serves its purpose, but it didn’t see any new major
feature in years.

We also saw a rewrite of the installer. The original one was written in bash,
then the second was written in C, using libdialog. And currently James works on
a third one, hopefully fixing lots of instability, resulting from the design
errors of the current one. ;-)

More and more developers

And then somehow more and more developers came. Gourdin Gaetan (bouleetbil) and
Sébastien Vincent (Baste) worked a lot on the GNOME support, James Buren took
over the base system from me. Paolo Cretaro (Melko) does KDE bumps nowadays and
Marius Cirsta (mcirsta) did heroic work of including OpenJDK 7 into Frugalware.

Abdelmoumene Hamza (Slown) is a guy who volunteered to do a log of boring work.
When you can install a random Perl on Python module from a package, chances
that it was packaged by Slown are high. ;-)

Regarding infrastructure, Benjamin Nolmans (Xarkam) stepped up, and finally
converted the data from our Flyspray instance to the more mainstream and
maintained Trac format. Regarding architectures Boris Albar (Elentir)
contributed an ARM port. Our press noise was generated by Russell Dickenson
(phayz) for a long time: he wrote newsletters regularly, and these days he
works as a documentation writer for Red Hat.

For a long time, no more Hungarians joined the project. Then Bagdán Róbert
(kikadf) came, and even if he’s rarely present on IRC, he still contributes
from time to time, and these small, but valuable contributions are what keep
Frugalware alive. :-)

Regarding other big upgrades, Daniel Exner (dex) was the guy who stepped up and
finally packaged KDE4 for Frugalware. All KDE4 users meeting him should buy a
beer for him. ;-) Anthony Jorion (pingax) joined in 2010, and since then he
does XFCE updates.

Finally, I should not forget about Daniel Eledut (Devil505), who probably
helped me the most with my "the rest" task: he really helped out in various
areas, and he was a major Frugalware contributor for a long time.

The end of my leadership

Contributing to other projects is something I did regularly, since every time I
got a bugreport for a package, first I created a patch to fix the problem (in
case it could not be worked around in the buildscript), and then of course
tried to upstream it, knowing the maintenance cost of non-upstreamed patches.
Still, these were minor and my main project was Frugalware for years.

The bigger contributions came when I started to complete Google Summer of Code
projects, first SWIG, then git, finally OpenOffice.org — which turned into
LibreOffice. The motivation is easy: these were still Free Software hacking,
but given that they are not bash hacking (like most packaging work for
Frugalware), more interesting problem had to be solved, GSoC also provided nice
payments, and needless to say: was useful to many people not using Frugalware.

The idea of passing over my leadership to someone else first came when I
looked back, and I saw that I spend more time on LibreOffice than on
Frugalware, even after the paid GSoC ended (later resulting in joining SUSE to
work on LibreOffice fulltime). My focus somehow just shifted from hacking a
distribution to hacking something that is directly useful for average users as
well. It’s not worse or better, it’s just different, and after lots of years, I
wanted to do something different.

To sum up, I did not reach my expectations as a project leader and I waited for
a candidate I could teach and finally who could become as good as I was in
earlier years, or hopefully even better. And then James came, who first took
over the base system, and we agreed that after 1.7, he’ll do releases. So it
was not unexpected, but still I was a bit surprised when I saw his
role-changing
commit.
But needless to say, when thinking a bit more, it was completely logical, all I
really do these days is fixing things left and right regarding my arm port
needs, and maintaining the LibreOffice package.

Summary

The idea of writing this post is not mine. However, when I thought about how
many individuals helped this project so far, I realized I really need to create
a summary, thanking their work. Reading the above, I hope that in the past
years my leadership to Frugalware were useful and I realize that these days I
have to step back, given a better leader appeared. I honestly wish the project
to see at least as many releases as we already have, now that fresh energy
leads it. :-)

I tried to point out all our heroes during the last years, but it’s possible I
forgot someone. If you think you’re missing from this post, please leave a
comment. Thanks.

Now the question is how does this config deal with the GRUB2 upgrade. First,
don’t miss the official upgrade howto, it
covers most cases. What I want to detail here is how did I avoid starting from
scratch and creating a proper partition table using fdisk.

Here are the steps I needed:

Backup. Yes, I did screw up for the first time, so it’s really needed.

Resize parts of the / RAID1 (/dev/sda1 and /dev/sdb1) with gparted livecd (resizing ext3 from cmdline parted didn’t work for me).

mdadm re-creation as described in the upgrade howto, from fw install cd, so you’ll have the required 1.0 metadata.

Reinstall GRUB1 to sda and sdb, since the physical location of GRUB1’s stage* changed.

Boot back to 1.6, run pacman-g2 -Syu, and grub-install — again, see the upgrade howto for details.

Now given that serial console needs a custom GRUB config and kernel parameters, you need to modify GRUB2’s /etc/default/grub. Here is my diff:

I recently got a GuruPlug. It has Debian
by default, and it’s apt config is set to stable, while in fact at the moment
what’s the factory default is considered as oldstable by upstream. So if you
blindly do a few apt-get install foo, soon you’ll have newer userspace than
kernel, and your device will no longer boot (based on true story - and yes,
this is not Debian’s fault). Moreover, I was interested in how to install
Frugalware on this device, so here is a quick howto.

Install rootfs

First you need to bootstrap Frugalware from Debian. It’s a good idea to install
Frugalware on a USB stick, so you can switch back to Debian in case you messed
up something and start from scratch again.

Partitioning is up to you, you’re recommended to have a small FAT (type: 0x0b)
partition (32MB for example) at the beginning, we’ll use that later. The second
can be the rest, ext4 or so.

Format and mount it (your device name may differ!):

mkfs.ext4 /dev/sda2
mkdir -p /mnt/sda2
mount /dev/sda2 /mnt/sda2

Then install our pacman-g2 binary to the Debian system, so you can bootstrap:

Installing the required packages is a single command, as described
here:

pacman-g2.static --noconfirm -Sy core base -r /mnt/sda2/

Upgrade the bootloader

Once the rootfs is ready, you need a new bootloader that will be able to boot
our vanilla kernel.

You need a JTAG
Board, so you can access the serial console. If you connect the USB cable to
you PC, you can use for example

screen /dev/ttyUSB0 115200

to access the device.

Given that we want to boot a vanilla kernel, we need a vanilla bootloader as
well. Before you mess with the bootloader, it’s a good idea to make a backup of
its config (there is a 3 second timeout during boot - if you press any key
there, you get the Marvell prompt). Here is my config:

The only semi-unique part is the MAC address of the network interface(s).

If you want to update the bootloader, a possible way is to put the new binary
to a pendrive. Given that the default bootloader does not support ext*, we need
a fat filesystem. So format the first small partition we created already (the
device name may be different in your case!):

mkdosfs /dev/sda1

Till Frugalware 1.6 is released, support for GuruPlug is available in
Frugalware -current only, so download the
binary
package from there, extract the u-boot.kwb file from the guruplug directory,
put it to the new partition. (A few other models are explained
here).

Before you reboot, copy also /boot/uImage to the fat partition, you may have
problems problems with reading the kernel from the ext4 partition with u-boot.

I did not care about boot speed for a long time, since servers usually
restarted only for kernel upgrades, laptops suspend to ram all the time,
and I have no desktop machine at home for years now. Though one use
case started to motivate me recently: I use virtual machines a lot, and
waiting for them to boot up is boring.

So I looked at the systemd-analyze blame output, and it turned out
that for a base-only install the only service that takes a lot of time
is netconfig. It’s because it was a oneshot service, so the whole
boot process waited for dhcpcd to get an IP. A much better solution is
to just start netconfig in the background and move on — and that’s
exactly frugalwareutils in git does now.

Finally I can run systemd on my desktop, and not in a virtual machine
only, since there is
a patch for the
sched/cgroup issue I had. It was not a systemd bug, but systemd
triggered a kernel issue, which was hidden so far.

I just
updated
our bluez package, and bus activation works fine. This means bluetoothd
is not started till somebody connects to it. Given that I sometimes do
not log in to KDE to use kbluetooth, but just want to do something on
the console, where having bluetoothd running is totally unnecessary, I
consider this really nice.

It came up on #bitlbee that there is a
Fedora bug where
they ask for systemd unit files for BitlBee. So I
created a patch, and the
nice thing is that the second iteration now seems to be fine for
Frugalware/Fedora/Debian, so we managed to build something cross-distro
here. (I think I mentioned that’s the #1 feature I like in systemd.)

I just found
this
podcast, it’s a bit dated since it happened in August and things have
been improved a bit since then, but still I was happy to listen it. (The
interesting part starts at 1:10:50; 17 minutes.)

I already had a short post about systemd, and I
mentioned the killer feature for me is the compact distro-independent
service file (initscript for sysvinit) format it uses, since even if the
initscript format we use in Frugalware right now is mostly
bullshit-free, it’s still more bloated than the systemd service files.

Now back to why I began writing this post. :) So there is
the post about systemd,
but it’s rather long, and it’s easy to miss the point. Since the
previous post, I think there are a few more killer features in systemd:

Every service is started in a separate control group (cgroup). Do you
remember the situation when you wanted to know who the hell started a
given particular process? This is typically a problem if the process is
running as root. Now this is no longer a problem, systemd-cgls will show
you if it was started by a user or a given service, etc. (Yes, I
consider this a security feature.)

Upstart already had this "restart if it crashed" feature, but systemd
does it better: if the daemon supports socket activation, then
messages sent to the socket are buffered by the kernel, and no message
will be lost during the restart.

systemd provides this on-demand feature, which is pretty much like
inetd, so hopefully we can get rid of inetd, which was a constant
problem previously. (We inherited a fork of the OpenBSD inetd from
Slackware, and being a fork it did not improve since years, causing a
constant problem.)

Why writing this post today? Because this morning systemd
appeared in -current, so you can
even try it
out without installing any unofficial package.

So we have this systemd wip
repo and one of the remaining todo items was to fix up splashy to work
with systemd.

First, why doesn’t it out of the box of systemd is a drop-in
replacement? Because splashy is not a simple init script, it hooks
itself to rc.sysvinit, so it was specific to sysvinit.

Now given that plymouth was already ported to systemd, it served as a
good example. You may also ask: why don’t we just switch to plymouth?
Because:

we don’t want to introduce multiple changes at the same time

it
requires
kms, while Splashy did not require it - so as long as it’s not true
that most video cards support kms (think of via or anything else which
supports vesafb, but not kms), such a switch would be a regression

Okay, enough anti-Plymouth rant. As you can see, right now the task is to fix Splashy.

The steps I did were:

updated the Splashy theme to remove the progressbar: given that now we
start services parallel and on-demand, it does not really make too
much sense

introduced unit files: to start on boot, to stop before getty/xorg, to
start before kexec/reboot/halt/poweroff

added the usual compatibility symlinks to avoid starting the old
rc.splash (which would happen anyway)

added a simple wrapper that fires up splashy and once it’s ready it
sends the Booting…/Rebooting…/etc text to it

The last problem is a bit more complex: now that xorg startup wants to
stop splashy, a dependency has to be declared. xorg startup is handled
by prefdm.service, but in case it depends on splashy-quit.service,
what will happen if you don’t have or don’t want a splash?

The solution for now is to declare an After= relation: that means xorg
startup will happen after the quit of splashy, if it happens at all.

This way:

you can disable the splash, it won’t cause any dependency errors

if you don’t have prefdm.service enabled, you can even have splashy
uninstalled

(this one is a bit ugly) you can disable the splash, but if you
uninstall it and have prefdm.service enabled, you’ll get a
dependency error, as systemd will have no idea what
splashy-quit.service is.

To test the above:

you need to enable the systemd wip repo (see the link above)

run pacman-g2 -Sy systemd

boot with init=/bin/systemd

Stay tuned, we still have a few blocker issues before merging the
(disabled by default, as you need that kernel parameter) systemd support
to current.