WARNING: You might be first to follow this guidecorrections, comments and improvements welcome.

DISCLAIMER: You follow this guide at your own risk. If your system breaks as a result, you get to keep both pieces.
Allow plenty of time - this is a major change. For help, post to this thread. >=udev-182 is only in ~ARCH at the time of writing.
If you run ~ARCH you are supposed to know what you are doing.

Aim of this guide
To keep you in control of the boot process on your system.

You need this guide if you have a separate /usr or /var partition and want to understand and have control of the boot process on your system after the upgrade to >=udev-182
The use of raid and lvm is covered as I had to make that work but encrypted filesystems are not as I have no way to test.
An initrd is required but you build it with the aid of a kernel provided script. Neither dracut nor genkernel are required, well if they were needed you would not
be in control would you?
The idea is to have a system that when it fails to boot, you can fix it yourself. Its likely that during the executopn of this procedure you will get to practice
that, so have a bootable USB stick, or CDROM handy. You will not be able to boot into an old kernel.

Prerequisties
A Gentoo system with a separate /usr and or /var partition, optionally, with everything except /boot on lvm on raid. >=udev-182 should be in your package.mask, or
you have seen emerge wanting to upgrade to it. If you already have udev-182, you box won't boot again until you make arrangeents to mount /usr and /var before udev
is started.

Overview
Preparation - you can do this now and test it. These steps are required before you reboot useing you new initrd but they can be tested and debugged before you
make the leap to >=udev-182

Initrd preparation
This can be done but not tested until your have >=udev-182 installed.

The leap to >=udev-182
Going back after this stage is pointless, long term the only way is on.

The reboot
If it doesn't work first time, fetch the boot media you hoped you would not need, get into your chroot and fix it.

Preparation
As the initrd will contain /sbin/mount change /etc/fstab to mount everything by UUID. This is a robustness improvement you get for free. You cannot yet use a UUID
in grub.conf to mount root. That's covered later, once the system boots using the initrd.
To disciver your UUIDs run the command blkid

The Automount is not required as our initrd init script will mount it anyway. If you have a kernel with the automount option missing, this guide will still work.

If you are using kernel raid auto assembly, turn it off

Code:

[ ] Autodetect RAID arrays during kernel boot

This will break rebooting meanwhile.

Rebuild and reinstall your kernel. If you need to reboot, raid autoassembly users cannot reboot into this kernel without following the rest of this guide.

At the outset, I mentioned a kernel provided script. The kernel sources provide the gen_init_cpio and gen_initramfs_list.sh utilities. The gen_init_cpio utility
does not come prepackaged and needs to be built.

Code:

cd /usr/src/linux/usr
make gen_init_cpio

Check these files are execuatable

Code:

chmod +x gen_init_cpio
chmod +x ../scripts/gen_initramfs_list.sh

Initrd Preparation

With the aid of the kernel scripts, there are two files to prepare. The init script itself and a list of files to be included in the initrd.
Old habits die hard an I like to prepare the initrd in /root/initrd
The /root/initrd/initramfs_list contains

# / (root) I wimped out of root on lvm for this box
/sbin/mdadm --assemble /dev/md126 --uuid=ad5fe0cb-775d-38b4-7169-e221fc96089f || rescue_shell
# if root won't assemble, we are stuck

# LVM for everything else
/sbin/mdadm --assemble /dev/md127 --uuid=52be4797:edab2349:eb21497e:52035eaa || rescue_shell
# and if the LVM space won't assemble there is no /usr or /var so we are really in a mess
# TODO could auto cope with degraded raid operation

# lvm runs as whatever its called as and we need vgchange
ln -s /sbin/lvm.static /sbin/vgchange

# start the vg volume group - we only have one volume group
/sbin/vgchange -ay vg || rescue_shell
# if this failed we have no /usr or /var

Fill in your own mdadm parameters and volume group names.
Remove the mdadm and lvm items from the init script if you don't use mdadm or lvm
Read the comments in the init file - you may need to make other adjustments.

Notice that if you make a mess of your included libraries, there is a 20 second pause then fsck returns as if it had worked but without doing anything.
This avoids the need to get into your chroot to fix your initrd, provided everything else works. Fix it. Mounting unchecked filesystems is ok for debugging but not
reccomenred.

Mount /boot, so the script gen_initramfs_list.sh can install the initrd there.
Run /usr/src/linux/scripts/gen_initramfs_list.sh -o /boot/initrd.cpio.gz /root/initrd/initramfs_list

Fix grub.conf. Mount root using the UUID of the root filesystem and add in the initrd line.

The leap to >=udev-182
Update udev and anything else that is manually masked due to udev. Check your /etc/portage/package.mask
be sure you have >=openrc-0.9.9.3
run

Code:

etc-update

Reboot to test.

TODO: Put all the files needed in the initrd in a filesystem in /root/initrd so they don't get messed up by random system updates.

TODO: Done - Use UUIDs in the mdadm --assemble commands post updated above but initscript examples copy/pasted from a second box.
UUIDs are unique, so readers need to use their own anyay. Not the the UUIDS here are the UUIDS foing using

Code:

mdadm -E <componet device>

not the UUIDs of the filesystem on the /dev/md device.

TODO: Test with a static /dev in the initrd and DEVTMPFS off in the kernel. If that works, this becomes a complex static initrd that can be hand rolled with cpio.

--- edit 1 ---
Well that should have booted but not cleanly. The filesystems that the initrd now checks and mounts the normal init system tryes to check and mount. Edit /etc/fstab to stop that.

The filesystems that are mounted by the initrd need to be set to noauto, so that the normal init system does not attempt to mount them. Do not remove the entries as the init script in the initrd uses them.
The pass number, thats th final field on the line needs to be set to zero, which means don't check this filesystem.

Which of those directories in the initramfs directory list (the first list of directories) are necessary, and are there others?

emerging busybox results is:

Code:

* You cannot have USE='static pam'. Assuming static is more important.

Do I need to rebuild busybox, or does this mean that it has automatically dropped the pam USE-flag, and built static?

What is the difference between an initramfs and an initrd? Also, this relates to the graphics options you have on your first kernel option in your grub.conf. I remember explicitly disabling framebuffer support and directfb a couple years ago (due to some problem with the X server which may not even exist anymore; support for Radeon? Xcomposite? I can't even remember =/ ). Is there instruction somewhere where I can set that up, besides the framebuffer wiki that's here?

Why does the "--uuid=..." line for your large lvm use colons innstead of dashes? And what do you mean you wimped out of root on lvm?

What is /mnt/root? I don't have one on my system. Is that / ?

Thanks for the guide. Adventure!

Cheers,

EE

Last edited by ExecutorElassus on Mon Apr 30, 2012 10:21 pm; edited 1 time in total

You need mdadm in the boot runlevel if your root is not on raid and the initrd does not assemble yur raid sets. (It does.)
If you have mdadm set up to monitor your raid sets and send you emails you should add it to the defualt runlevel unless you need it started earlier.
The initrd only assembles your raid sets - it does not start the mdadm service.

Search in menuconfig is your friend. press / and enter devtmpfs.

Busybox must be built with stactic, as its libraries are not in the initrd. This meanes busybox does not have pam support.

The difference between and initrd and initramfs is one of internal format. An initrd is an old term the really refers to an ext2 filesystem in a gzipped file.
An initramfs is a compressed cpio archive ove other cpio achives. The compression is user defined at kernel build time ... at least, the decompression support is.

I still use a framebuffer console as I use the nvidia binary blob for Xorg. You continue to use hwatever you have been using. This initrd/initramfs does not affect any video options._________________Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

The initrd is a little like a boot CD , when you installed originally.
You attached your root partition at /mnt/gentoo to being the install, untarred the stage3 then chrooted into it.
The initrd is the boot CD, your install is the already untarred stage3 and the init script in the initrd lists the steps requited to mount things, check them and get into the chroot.

The initrd attaches it at /mnt/root because it needs to read /etc/fstab to get the UUIDs of /usr and /var
It also needs to switch_root right at the very end, so your root filesystem has to be mounted somewhere before that._________________Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

ah, okay. then I'll delete that directory once I can mount / again. For the moment, the initrd is not mounting any of the / partitions, and is consequently unable to find the kernel's root (that is, the block device specified by "root=…" in grub.conf). Neither "/dev/md126" (which is the raid device built out of the three mirrored partitions from each of the three HDDd), nor the equivalent UUID of that array device, seem to work. Is there some other UUID I should be giving? Just the UUID from one of the partitions?

If I boot the system, only symlinks for the gentoo PV's containing LVs are created in /dev/disk/by-label(uuid), I assume by udev.
Despite eg /dev/arch/ and /dev/ubuntu/ are present, the /dev/mapper/ only contains the gentoo LVs.

If I start after boot the lvm service, still no symlinks.
However, if I restart the lvm service, the symlinks are created.

I have basically the same setup for arch and ubuntu, and it works there.

Also, I don't get the system started without the "dolvm" kernel parameter, but I guess it is genkernel init script related

Any help would be appreciated.

PS: I probably give your cpio "arch" approach to initramfs a try and see if that works better.

Last edited by udeved on Sun May 06, 2012 5:50 pm; edited 1 time in total

I don't get any symlinks at all for my logical volumes.
In fact, I get error messages from lvm saying that udev was supposed to make the links but didn't ... falling back in direct linke creation.

genkernel will do something similar to my initrd but it will do a lot more too as genkernel builds you a fully modular kernel, therfore, it needs to load the kernel modules needed to boot your box too. My initrd is not tied to any partitular kernel as it does not contain bits of the kernel. With genkernel, you need a new initrd with every kernel build.

Are your arch and ubuntu logical vlomes started by genkernels initrd?
I would be surprised if they were._________________Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

I don't get any symlinks at all for my logical volumes.
In fact, I get error messages from lvm saying that udev was supposed to make the links but didn't ... falling back in direct linke creation.

genkernel will do something similar to my initrd but it will do a lot more too as genkernel builds you a fully modular kernel, therfore, it needs to load the kernel modules needed to boot your box too. My initrd is not tied to any partitular kernel as it does not contain bits of the kernel. With genkernel, you need a new initrd with every kernel build.

Are your arch and ubuntu logical vlomes started by genkernels initrd?
I would be surprised if they were.

Yes, perhaps I didn't get the problem along.

I run a custom kernel, without many modules, almost all is built in, but the initrd is generated with

Code:

genkernel --lvm --install initramfs --kernname=kernel

Only eg ext3 fs module in initrd, its about 5 modules, and very small in size

# To activate volumegroups on all devices in the cache
lvm_commands="${lvm_commands} \nvgchange -ay --sysinit"

# To create symlinks so users can use real_root=/dev/vg/root
# This needs to run after vgchange, using vgchange --mknodes is too
# early.
lvm_commands="${lvm_commands} \nvgmknodes --ignorelockingfailure"

# Iterate sysfs and fire off everything; if we include a rule for it then
# it'll get handled; otherwise it'll get handled later when we do this again
# in the main boot sequence.
( /sbin/udevadm trigger --action=add --subsystem-match=block; \
/sbin/udevadm trigger --action=add --subsystem-nomatch=block; ) &

If you follow the bug report I posted, iirc post 54 gives the solution.
It says to comment out some lines in /lib/rcscripts/addons/lvm-start-sh, which I bvelieve is used in initrd.
I will generate a new initrd, after I commented out the vgchange line as well, and test it.

I've been following the udev-182 saga, but not dared touch anything so far. My box has a 4-disk RAID 5 array partitioned into /, /var, /home and /var/tmp, so /usr is part of rootfs, but /var isn't. The system is an amd64 Gentoo stable setup, kernel 3.2.12 (no initramfs) sys-fs/udev-171-r5. The mdadm service is in the default runlevel, and doesn't start until quite late (after cups). The kernel assembles the RAID array from command-line parameters (not auto-assemble, but the explicit version of the same). Here are the md-related lines from syslog:

. md127p1 is /var, md127p2 is /var/tmp (and /tmp bind-mounted from within that), md127p3 is rootfs, and md127p4 is /home. /etc/fstab locates the partitions by LABEL=, which may not be as precise as UUID, but it's pretty similar in function.

My question is, will this need an initramfs after udev-182? I'd have thought not, as the kernel is already assembling the RAID array without udev's or mdadm's assistance. It will need something to mount /var and so forth before udev starts, such as greyspoke's script, or maybe just a one-liner to mount /var.

If it's got a reasonable chance, I'll try upgrading udev to test it, but not if I'm almost certain to need to build an initramfs._________________Greybeard

You will need something to mount /var before udev starts, thats normally in the sysinit runlevel and well before local-mount.
An initrd or one possible solution. A mount /var early in the in init script is not a good idea. The filesystem should be checked before its mounted, so now you need the right fsck too.

There are two or three other approaches, greyspokes script. SteveLs changes to the existing init scrips and a latecommer, vapiers busybox with the USE=sep-usr flag set.
I'm not sure how that works but I guess its intended for use in an initrd._________________Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

You will need something to mount /var before udev starts, thats normally in the sysinit runlevel and well before local-mount.
An initrd or one possible solution. A mount /var early in the in init script is not a good idea. The filesystem should be checked before its mounted, so now you need the right fsck too.

There are two or three other approaches, greyspokes script. SteveLs changes to the existing init scrips and a latecommer, vapiers busybox with the USE=sep-usr flag set.
I'm not sure how that works but I guess its intended for use in an initrd.

Thanks a lot for mentioning the buybox approach.
Google found me the thread, and it states it works with busybox 1.20.0 without initrd.
Exactly what I was looking for, besides the earlymount script.

I don't think busybox with sep-usr works with /usr on LVM since it runs before init (thus before lvm service):

vapier wrote:

[the] new applet has a hand written set of commands to automatically mount /dev /proc /sys /usr and seed /dev, and then execute
the real init (defaulting to /sbin/init)

I'd like openrc using busybox in any case, so I'll play with that sometime in next couple of weeks.

You could also try earlymounts or the patched scripts, which I prefer as they seem to have less issues with other stuff, although they are a bit trickier to setup due to the patching needed, and switching udev runlevel. (They just make udev run after localmount, so don't require any configuration beyond a single variable in udev.conf, eg for fsck, lvm or mdadm, last of which I don't use, so haven't tested.)

I can confirm, that busybox sep-usr does not work with root on lvm.
I tried to configure busybox with mdev support, no difference, LVM volumes don't get activated.

I also messed with different initrd structures.
I took arch's initrd created with mkinitcpio, customized it a bit, and the thingy boots. Probably I post the init script here, after I got everything like demons to work properly.

After closer inspection of arch's initrd, I found that arch includes the udevd in lib.
This seems to solve the problem of LVs not getting properly symlinked and thus refuse to show in filemanager.

I also simply copied the whole arch kernel with ramdisk to gentoo boot partition.
If I boot arch's kernel with gentoo, everything LV related works as intended in desktop environment.

Atm, arch's mkinitcpio seems to have the better approach with their "hooks" for ramdisk than genkernel.

Root on LVM has always required an inird. LVM is a userspace tool and you need it working to mount root.
Root on a real partition or autodetected raid with everything else in LVM did not used to need an initrd but will with >=udev-182 as /usr and /var are in LVM, so LVM must be started to mount /usr and /var before udev starts.

I'm not sure if steveLs patched scripts will start LVM early enough. I think they will as local-mount used to mount /usr and /var anyway, so LVM had to be running before local-mount._________________Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Root on LVM has always required an inird. LVM is a userspace tool and you need it working to mount root.
Root on a real partition or autodetected raid with everything else in LVM did not used to need an initrd but will with >=udev-182 as /usr and /var are in LVM, so LVM must be started to mount /usr and /var before udev starts.

I'm not sure if steveLs patched scripts will start LVM early enough. I think they will as local-mount used to mount /usr and /var anyway, so LVM had to be running before local-mount.

Yes, you are totally right. I completely forgot about userspace when made my enthusiast post.
I assumed wrongly that ginit also allows the kernel parameter dolvm, which produces a kernel panic.
I forgot to mention, I moved my root to a non LVM partition as well and tested busybox sep-usr

The result was still unmounted partitions with that ginit kernel paramenter for non root LVM partitions.
I tested a non LV /usr and /var as well.