Note: if you are upgrading udev beware that from 204 on, the ebuild silently adds udev to the sysinit runlevel. So before you reboot, make sure to run rc-update del udev sysinit or you will get what looks like dependency errors, with fsck (and udev) starting before lvm/localmount.
There were no changes for 204: it just took me 3 hours of chasing initscripts, before I was ready to give up and lo and behold, adding it to sysinit (after removal from boot) failed. :/
Note to self: always check: rc-update show and scan across. ;)

Warning: If you are on lvm see the notes below.

As many of you will be aware, udev upstream has moved to requiring /usr mounted before it starts. Upstream (including kernel developers) advocates an initramfs across the board for Linux distributions, but definitely in the case where /usr is on a separate partition. This is not down to udev itself, but helper scripts for devices which often live in /usr or sometimes /var, and aiui the hardware database is also on /usr (i guess that bit is down to udev itself;)

This has caused some controversy on the gentoo-user and gentoo-dev mailing lists, as many of us are used to keeping /usr on a separate partition for security and backup purposes (other use-cases for a separate partition include mounting /usr over NFS, or a common partition for virtual machines) and don't want an initramfs, both for startup times and the additional maintenance task of recompiling it at every kernel upgrade/recompile (yes this could be scripted, but we don't see the need.) I've read several times that you can't use LVM on /usr etc without an initramfs, but this has never been an issue for several of us. (I don't have root on LVM, just most other partitions.)

Anyhow, leaving aside the arguments, Gentoo is about choice and in that spirit, I'd like to share the setup I've been using for the last 2 years (so I know it works;) to make udev start after localmount.

NB: Do not use this if you don't know what this is about, don't have /usr or /var on a separate partition, or need some device started by udev in order to mount local filesystems, or start your system (eg a bluetooth keyboard was given as an example on the dev mailing list.)

Having said that, it is simple enough to revert (comment line in /etc/rc.conf, and move udev to sysinit runlevel again,) so that could be done from a rescue shell (boot with init=/bin/sh and then edit file and run rc-update) or live disk [at worst-- if you can't get a shell] for setups that need udev to mount local drives.

As usual, you are responsible for your own system. This works for me and is easy enough to revert, but don't complain if you have an unusual setup and find you need to boot a live disk and chroot in to fix it (although you're welcome to ask for help, or share the experience. :) This is for people who know they have all the modules built-in the kernel to mount local filesystems, have a separate /usr and/or /var, and are happy with their current setups, apart from possible future issues with udev starting before localmount, and find the requirement for an initramfs sufficiently annoying to tweak their setups, and are willing to deal with keeping the lines in the initscripts during etc-updates.

Basically it consists of patches to 2 initscripts (these are all tested against stable openrc: 0.11.8, and udev: 204. eudev-1.1 is confirmed working as well) to support a new option initramfs which defaults to no/0.
There was a bug in openrc-0.11.5 which meant we had to tell localmount to umount /usr when it stops, which meant we needed a cross-initscript variable. See the discussion in this post if you're curious about it. That's been fixed, so we again only need patches to udev scripts. It's still more convenient to keep the setting in /etc/rc.conf: after all it affects more than one initscript, and issues like this may come up again, where it's much easier just to patch a file quickly using the setting we have easily available.

Config: /etc/rc.confNote: There is only one line with any content:

Code:

+#initramfs="YES"

The rest is all instructions and comments on the setup.

Code:

--- /etc/rc.conf
+++ /etc/rc.conf
@@ -75,7 +75,7 @@

##############################################################################
# MISC CONFIGURATION VARIABLES
-# There variables are shared between many init scripts
+# These variables are shared between many init scripts

# Set unicode to YES to turn on unicode support for keyboards and screens.
unicode="YES"
@@ -89,6 +89,10 @@
# own fstypes to the following variable.
#extra_net_fs_list=""

+# UNSUPPORTED: Allow people with a separate /usr and /var to live without an
+# initramfs. See EOF for explanation and rc-update setup. Default="YES"
+#initramfs="YES"
+
##############################################################################
# SERVICE CONFIGURATION VARIABLES
# These variables are documented here, but should be configured in
@@ -160,3 +164,40 @@
# so if you turn this off, be aware that you may not be able to use that
# feature.
#rc_controller_cgroups="YES"
+
+##############################################################################
+# SPLIT PARTITIONING WITHOUT INITRAMFS (UNSUPPORTED by systemd-udev)
+# Allow people with a separate /usr and /var to live without an initramfs.
+# The problem here is that udev helpers live in /usr and /var, so udev has
+# to start after they have been mounted.
+# To use udev, you MUST have CONFIG_DEVTMPFS set in kernel
+# If you are not using an initramfs (and are *sure* that you don't need one
+# NOR udev to mount /usr and /var) you can set initramfs="NO" to make
+# udev-mount provide dev for devfs and udev wait for localmount.
+# This also means localmount will umount /usr when it stops.
+# NB: you MUST first have applied the patches from:
+# http://forums.gentoo.org/viewtopic-t-901206.html
+
+## NOTE: to make this work you MUST:
+# rc-update add udev-mount sysinit
+## - this is now required by newer udev
+# rc-update add sysfs sysinit
+# rc-update del udev sysinit
+# rc-update add udev boot
+
+## If you have alsasound in boot then move it to default:
+# rc-update del alsasound boot
+# rc-update add alsasound default
+## - you can also add it to nonetwork:
+# rc-update add alsasound nonetwork
+
+# If you are using LVM, you MUST use: /dev/mapper/<vg>-<lv>
+# in /etc/fstab, and not: /dev/<vg>/<lv>
+# CONFIG_DEVTMPFS_MOUNT is recommended for LVM.
+
+## If you are resetting initramfs to its default, you MUST:
+# rc-update del udev boot
+# rc-update add udev sysinit
+## - sysfs can also be removed if you wish, and it will be
+## called by udev in sysinit:
+# rc-update del sysfs sysinit

Again: The only line of significance here is:

Code:

+#initramfs="YES"

which you have to change to "NO" (or "no" or 0) in order to use the new setup; this should be done after the changes to runlevels (and kernel first, if needed: --reboot before making any of the other changes) outlined in the comments.

Basically all you are doing is moving udev to boot, not sysinit, so it can start after localmount. Additionally sysfs and udev-mount need to be added to sysinit; before the change they would have been started up in sysinit as dependencies of udev.
Moving alsasound to default allows network drivers to start in the correct order, as alsasound starts after hotplug, and on my system at least triggers net.eth0 starting before net.lo. This hasn't made any difference on my machine afaict, but it doesn't look right, and is counter to the dependencies in the init.d net script, so I figure it's best to move alsasound. After all, it's really not needed to get the machine to boot is it?

The other two changes are to the init scripts for udev and udev-mount, to support the new option:
/etc/init.d/udev-mount

As you can see this just does the default setup, unless we're using the initramfs="no" option, when we should start after localmount to make sure /usr and /var etc are mounted. The before line makes sure we start as soon as possible after localmount (immediately after, here.)

NB: If you are using lvm, you will need to use /dev/mapper/vg-lv in your /etc/fstab rather than /dev/vg/lv. The /dev/mapper/vg-lv links are made by the lvm vgscan, in our case at a time when udev is not running to make symlinks.
Note:

Code:

/dev/vg/lv => vg-lv under: /dev/mapper

(ie with a hyphen.)

For >=sys-fs/lvm2-2.02.100 you'll need these set in /etc/lvm/lvm.conf:

[Thanks to saellaven for working these out.]
The first, sysfs_scan is default; I mention it in case you have it set differently.

CONFIG_DEVTMPFS_MOUNT is also useful, should you ever need to boot into a rescue shell. It means /dev will automatically be mounted by the kernel, which means the /dev/mapper links will be available however you start your machine, IME. While I've seen the links unavailable in sysinit (for 204 upgrade, when udev had been added back to sysinit by the ebuild) whenever I've booted into a rescue shell they have been present (presumably from the dm kernel module.) So you can usually get mount -a to work: if it's a dependency ordering issue, lvm will have already have activated the volumes, albeit too late for fsck; otherwise run:

Code:

pvscan; vgscan --mknodes; vgchange --sysinit -a ly

to activate your local volumes.
--

So, if you're sure you don't need udev to mount your local drives, you can simply add the new option and change a couple of runlevels via rc-update to get udev to start after they've been mounted. This means full correct udev support, wherever the helper scripts or the hardware database live.

Note: you won't be able to use an encrypted rootfs, or rootfs on LVM, without an initramfs using this, but then you can't do either of those without an initramfs at all. Also, bear in mind that udev-181+ is not supposed to support booting without an initramfs, so be careful upgrading! Remember the note at the top about udev being added to sysinit, without notice.

Effectively, this is not so much a technical requirement as a distribution packaging issue, since all this happens in an initramfs in any case (or how else can it support lvm on /usr?) That's why the original technical specification given, was that "/usr and all file-systems where supporting scripts, binaries, plugins or libraries might live, be mounted and available before udev starts" (to summarise Greg K-H, in case I've not got the words exact.) So when I've had some rest I'll look at filing a bug, for upgrades.

I'd recommend backing up your current udev package with qpkg from portage-utils, if you're not using FEATURES=buildpkg (highly recommended, in combination with eclean from gentoolkit), before you upgrade.

Er, the user ML; I was hesitant about posting to dev, at very least until it's been tested by more people (and changes for unstable are worked out) and there seemed to be quite a few people on user ML who wanted an alternative. (The fake initscript thing was interesting in terms of showing something new, but seemed like a bit of a hack to me, as well as an unnecessary extra step; dependency tracking via the init system feels like the correct approach, since that is the issue-- udev needs localmount and for most of us localmount doesn't need udev, although in a generic system it might-- and an interesting area to explore as I'd never played with it.)

Thank you, I will try this as soon as possible. Since a few weeks I'm using a minimal initramfs to mount /usr (this was mostly to learn how to build an initramfs and use it), but I'm not entirely satisfied with it.
Since a few days I thought about removing it, your topic is coming just in the nick of time

It will be the weekend before I can test as I can't afford to be without my main box until then.
Then its /boot on raid1, root on raid5 and everything else in LVM on raid5, with separate /usr and /var._________________Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

No worries, Neddy; I look forward to hearing how you get on whenever you do get some time. I haven't used RAID (mdadm?) myself, so I don't know anything about it, but if you were able to get your machine booting without a ramdisk before (as many people appear to have been able to) then you should be fine.

As I said, resetting it is pretty easy; I kept switching back and forth when I was playing with the patches in August.
To turn it off, you just comment out the initramfs="no" line in /etc/conf.d/udev and run:

Code:

rc-update del udev boot
rc-update add udev sysinit

..which only requires root fs.

netfab, glad it will be of use (hopefully;) Let us know how you get on.

My system has nothing special, just a separate /usr, this seems to work fine.
Just a small typo, in /etc/conf.d/udev

Ah thanks netfab; I've edited the post to reflect that.

I also have CONFIG_DEVTMPFS_MOUNT which isn't needed, in that udev-mount mounts /dev as a tmpfs (devtmpfs if in /proc/filesystems which is what we want and configure) but it doesn't hurt, as udev-mount just einfo's that /dev is already mounted. If things change in the initscripts we can fall back on the kernel option (it might also be safer for more esoteric setups in that it's mounted straight after rootfs by the kernel itself.)

I really don't want CONFIG_DEVTMPFS so I'll play with a cut down static /dev to get my raid sets assembled.
It might justwork for me anyway as my install is old enough to have come with a full static /dev from the stage3.

Users who have installed since May 2011, when baselayout2 was added to the stage3 will not have this static /dev

More news at the weekend._________________Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Users who have installed since May 2011, when baselayout2 was added to the stage3 will not have this static /dev

According to bug 368597 there was a problem with device nodes not being installed in stages for a few months. While it's not a full static /dev, users should now be getting ~250 static device nodes to fall back on; note that these are hidden once udev-mount has mounted a (dev)tmpfs on /dev (or indeed if kernel has CONFIG_DEVTMPFS_MOUNT set) (correct me if I'm wrong, as ever.) If you should be in that situation, comment 97 gives a good approach to recovering the standard setup for recovery situations.

The bug is useful reading for others looking at using static devices, who should consider MAKEDEV (from sys-apps/makedev but available on live CD) which can be used to make individual device nodes (eg MAKEDEV /dev/ttyS0) (add -S flag to see what it'll do first) as well as the traditional MAKEDEV generic (which would be run from within the target /dev.) MAKEDEV will take care of permissions and ownership, as well as device types, for you.

I'm not sure how static device nodes interact with udev; presumably it asks the kernel what devices are currently in use to populate its new /dev directory?

My setup use partitions everywhere (/var /tmp /usr /home /var/portage /usr/src, etc). So, this means, I am in trouble. Although, I hate to use an initrid, before experimenting your method, I want to create an initrd.
I have never used or created it before, so I ask here, what is the current way of creating an initrd without genkernel for grub2? I don't want to create a big initrd, but only to enough to get over separete /var /usr problem.
There seems to be a new tool dracut which obsoletes the mkinitrd.
Will this work?

Code:

dracut /boot/initrd-3.1.1-gentoo.img 3.1.1

Also what about kernel options that need to enable? only initramfs is enough to enable? Are there other packages needed to install?

Heh so do I, and don't worry you're not. Like you I've partitioned as much as I can, for differing file usage, stopping runaway processes running in a temp directory, and one I like is not allowing device nodes anywhere but root; unfortunately I found that stopped stuff working a while back, so /usr has lax permissions. You can't have 'noexec' on /var/tmp as scripts and built programs need to run from there during emerges.

Here's my fstab; it's easier to paste it than describe it and should prove to you that this setup works for complex partitioning schemes-- just not if you need services started, or scripts run, to mount root. Simply put, if you didn't need an initrd before, you won't now, and udev scripts will all start up properly, since the requisite filesystems are all there before it gets started.

/dev/mapper/gent2 and /dev/mapper/var are LVM volume groups; I also have a debian group I no longer use anymore, as well as plenty of hard-disk space for when I get round to experimenting :) All of this was setup using the cfdisk, lvm tools etc, via the gent2 installer script we started a few years ago; it definitely needs some love. (I was out of action for quite a while and update needed the work first to get up to speed with emerge after such a long hiatus.)

Quote:

Although, I hate to use an initrid, before experimenting your method, I want to create an initrd.

That's commendably sensible.

Unfortunately I've never used one since bad old days trying to compile a kernel in mandrake about a decade ago. Bear in mind that initrd is old style, initramfs is newer and supposed to be easier to handle. But then you appear to have got there already :)

Thanks for the tip; anything like this is welcome afaic. The more options people have, the better.

/me looks at Neddy for some info on setting up static dev ;)
(it's been mentioned on user ML as well; people are making mdev work again, and also playing with static /dev on vms.)

(Yeah I know I should have /boot read-only for security, but I figure if something malicious can write there, it can read and write anywhere, so my box would already be compromised.)

edit: Updated fstab listing to reflect usage with new lvm which relies on udev to create the /dev/vg/lv mapping of /dev/mapper/vg-lv.

Last edited by steveL on Mon Mar 19, 2012 7:03 am; edited 1 time in total

Hi, steveL. It's a pleasure to see you, here, too. Thanks for the info you shared. I will try when I can afford time.
Btw, I see, your update script advanced wonderfully. That is really appreciated. Best way to update. I don't know what I would do, if you guys weren't here._________________Anyway it's all the same at the end...
Need help to get it working: "x-fi surround 5.1"

What are the patches for udev-171-r5? I applied the ones for 164 as best I could but gnome-terminal became unusable (don't know what else broke), luckily I was able to ssh into the box and revert the changes.

what is the gain in placing /usr in separate partition?_________________Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein
ProjectFootball

Sorry about delay responding, I was offline for a while. When I upgraded my machine after a couple of months, the whole thing broke ;) but it turned out to be due to a recent change in lvm2, since my LVM partitions were being mounted as eg /dev/gent/usr instead of /dev/mapper/gent2-usr, and lvm startup now expects the nodes to be made by udev. I had to use vgscan --mknodes when booting from a rescue shell, and couldn't quite integrate that into the lvm scripts (which iirc have the same option) so I switched back to initramfs=yes. Then X didn't work and it took me a while to realise I'd upgraded xorg-server so needed the drivers rebuilding. After that I didn't bother playing with it, though I realised I could just use the mapper device names instead of the udev-provided friendly names, so I switched my fstab to those to check they function, and got on with other work.

Trog Dog wrote:

What are the patches for udev-171-r5? I applied the ones for 164 as best I could but gnome-terminal became unusable (don't know what else broke), luckily I was able to ssh into the box and revert the changes.

After the above saga, I've tested the updated patches in the first post, which I'm running on my machine now. The two major points are lvm usage as above (it simply won't work unless you use /dev/mapper/..) and additionally references to /etc/conf.d/udev were removed from the udev and udev-mount initscripts; that's not an issue for the former, as openrc sources it itself, but we need it to build the depend() info for udev-mount.

Curiously, on my machine, when I was testing the upgraded patches, not having the initramfs info in udev-mount didn't hurt, possibly as I already have CONFIG_DEVTMPFS_MOUNT, not sure.

Quote:

Additionally, will this workaround still be valid for udev 181?

I don't know yet, but I'd plan for it not to be. There's a recent thread on dev ml about a news item for 181 hitting unstable tree, stating that udev will no longer support starting without an initramfs. It'd be a shame if they actively put in stuff that requires more than local drives being mounted, but I'd advise getting a basic initramfs built-in to your kernel that does support udev with the standard setup (initramfs=yes) before doing that upgrade, so you can switch to it before installing, or in case there are any other problems later.

I'd understand if you felt you might as well just chuck in the towel and switch to an initramfs.

I just realised my home server has /var on its own partition... I don't really need hotplug on there, would it be easier for me to make the kernel non-modular and use devtmpfs on its own maybe?

edit: I've installed sys-fs/static-dev - a few things broke initially, but I fixed them all:

/etc/init.d/udev* didn't get unmerged due to config-protect and all hell broke loose on the next reboot until I removed them by hand

I needed to `rc-update add sysfs boot` since it no longer got autostarted

I was using kexec, which no longer autodetected the root partition (no /dev/root symlink), so I had to set that manually to avoid kernel panics

/dev/{ptmx,tty*,vcs*} need manually chown'ing to root:tty and chmod g+rw on vcs*. Paludis in particular complains if they aren't, other programs might too.

wait, do I need initramfs if my /var is on a different partition?_________________Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein
ProjectFootball

I just realised my home server has /var on its own partition... I don't really need hotplug on there, would it be easier for me to make the kernel non-modular and use devtmpfs on its own maybe?

edit: I've installed sys-fs/static-dev - a few things broke initially, but I fixed them all:

/etc/init.d/udev* didn't get unmerged due to config-protect and all hell broke loose on the next reboot until I removed them by hand

I needed to `rc-update add sysfs boot` since it no longer got autostarted

I was using kexec, which no longer autodetected the root partition (no /dev/root symlink), so I had to set that manually to avoid kernel panics

/dev/{ptmx,tty*,vcs*} need manually chown'ing to root:tty and chmod g+rw on vcs*. Paludis in particular complains if they aren't, other programs might too.

Great info, Ant, and nicely done :)

One thing: maybe you should have sysfs in sysinit runlevel? I'm not sure how vital it being there is, but I know that's where it starts on a standard system, as udev is normally in sysinit and requires sysfs and udev-mount.

DaggyStyle wrote:

what is the gain in placing /usr in separate partition?

As I said in the first post "many of us are used to keeping /usr on a separate partition for security and backup purposes (other use-cases for a separate partition include mounting /usr over NFS, or a common partition for virtual machines)." It's also easier to use lvm (and iirc other technologies like drive encryption and mdraid, though that may be less true nowadays if you have no initramfs) if you have a small-ish root and mount other partitions from lvm, which is my use-case. I also like to set different mount options on a partition basis; nodev, nosuid and noexec are the main security ones, though as you'll see from my fstab above, /usr and / have the same options. I certainly don't want /var, which is also required for some udev scripts, to have lax settings though.

But you don't have to use this to get separate partitions: you can do that with an initramfs, and have full support. I'm personally not happy at being told an initramfs is the rescue system that root used to be, at the same time as I'm told that I only need a small initramfs with at most a few hundred kilobytes in it, but I would recommend anyone new to partitioning to go the supported route.

Well, it needs to be mounted before udev starts (and pragmatically all standard mountpoints do, ie the equivalent of localmount.)

imho this is completely stupid, is there any way to avoid this and still update udev

but if there is any relation to the news about the udev upgrade, than the msg was talking about /usr only, thus the gentoo devs got lazy and may cause unneeded serious computer breakage for users, this must be corrected at once and notify all that /var on separate mounts is affected too._________________Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein
ProjectFootball

After the above saga, I've tested the updated patches in the first post, which I'm running on my machine now. The two major points are lvm usage as above (it simply won't work unless you use /dev/mapper/..) and additionally references to /etc/conf.d/udev were removed from the udev and udev-mount initscripts; that's not an issue for the former, as openrc sources it itself, but we need it to build the depend() info for udev-mount.

Curiously, on my machine, when I was testing the upgraded patches, not having the initramfs info in udev-mount didn't hurt, possibly as I already have CONFIG_DEVTMPFS_MOUNT, not sure.

Cheers Steve - I'll give it a shot on my boxes tonight.

Quote:

Quote:

Additionally, will this workaround still be valid for udev 181?

I don't know yet, but I'd plan for it not to be. There's a recent thread on dev ml about a news item for 181 hitting unstable tree, stating that udev will no longer support starting without an initramfs. It'd be a shame if they actively put in stuff that requires more than local drives being mounted, but I'd advise getting a basic initramfs built-in to your kernel that does support udev with the standard setup (initramfs=yes) before doing that upgrade, so you can switch to it before installing, or in case there are any other problems later.

I'd understand if you felt you might as well just chuck in the towel and switch to an initramfs.