It would be best to configure software that has some sort of state/database to use unique state/database storage directories for each host. If you wanted to run [http://puppetlabs.com/ puppet], for example, you could simply use the {{ic|%H}} specifier in the puppet unit file:

It would be best to configure software that has some sort of state/database to use unique state/database storage directories for each host. If you wanted to run [http://puppetlabs.com/ puppet], for example, you could simply use the {{ic|%H}} specifier in the puppet unit file:

−

{{hc|# vim "$root/etc/systemd/system/puppetagent.service"|2=<nowiki>

+

{{hc|# vim "$root/etc/systemd/system/puppetagent.service"|<nowiki>

[Unit]

[Unit]

Description=Puppet agent

Description=Puppet agent

Line 279:

Line 250:

[Install]

[Install]

−

WantedBy=multi-user.target</nowiki>}}

+

WantedBy=multi-user.target

+

</nowiki>}}

Puppet-agent creates vardir and ssldir if they do not exist.

Puppet-agent creates vardir and ssldir if they do not exist.

Line 289:

Line 261:

{{Accuracy|The initramfs should contain only the hooks required to mount the root filesystem and nothing more.}}

{{Accuracy|The initramfs should contain only the hooks required to mount the root filesystem and nothing more.}}

−

It is possible to avoid systemd unit files entirely by mounting /var (or subdirectories) in early boot with initcpio. You can use the following files as guides:

+

It is possible to avoid systemd unit files entirely by mounting /var (or subdirectories) in early boot with initcpio. You can use the following files as guides:

{{hc|/usr/lib/initcpio/install/var|2=

{{hc|/usr/lib/initcpio/install/var|2=

Line 322:

Line 294:

}}

}}

−

You will then need to add "var" to the end of HOOKS in {{ic|/etc/mkinitcpio.conf}} on the client, and re-run {{ic|mkinitcpio -p linux}}.

+

You will then need to add {{ic|var}} to the end of HOOKS in {{ic|/etc/mkinitcpio.conf}} on the client, and re-run {{ic|mkinitcpio -p linux}}.

== Client boot ==

== Client boot ==

Line 334:

Line 306:

This makes things particularly interesting when it comes to kernel updates. You can't have your client filesystem mounted while you're booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.

This makes things particularly interesting when it comes to kernel updates. You can't have your client filesystem mounted while you're booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.

−

You'll need to first copy $root/boot from the client installation to your tftp root (i.e. /srv/boot).

+

You'll need to first copy {{ic|$root/boot}} from the client installation to your tftp root (i.e. {{ic|/srv/boot}}).

Note: next-server should be the address of the TFTP server; everything else should be changed to match your network

RFC 4578 defines the "Client System Architecture Type" dhcp option. In the above configuration, if the PXE client requests an x86_64-efi binary (type 0x7), we appropriately give them one, otherwise falling back to the legacy binary. This allows both UEFI and legacy BIOS clients to boot simultaneously on the same network segment.

TFTP

The TFTP server will be used to transfer the bootloader, kernel, and initramfs to the client.

Set the TFTP root to /srv/arch/boot. See Tftpd server for installation and configuration.

Network storage

The primary difference between using NFS and NBD is while with both you can in fact have multiple clients using the same installation, with NBD (by the nature of manipulating a filesystem directly) you'll need to use the copyonwrite mode to do so, which ends up discarding all writes on client disconnect. In some situations however, this might be highly desirable.

NBD

Note: Set copyonwrite to true if you want to have multiple clients using the same NBD share simultaneously; refer to man 5 nbd-server for more details.

Start nbd systemd service.

Client installation

Next we will create a full Arch Linux installation in a subdirectory on the server. During boot, the diskless client will get an IP address from the DHCP server, then boot from the host using PXE and mount this installation as its root.

Directory setup

Create a sparse file of at least 1 gigabyte, and create a btrfs filesystem on it (you can of course also use a real block device or LVM if you so desire).

Though poorly documented, GRUB also supports being loaded via PXE. Because of an endianness bug in grub-core/net/tftp.c not fixed until bzr revision 4548 (grub-bios 2.00 is revision 4542), you will need to build grub-bios-bzrAUR; it makes the most sense to do this on the host installation. In addition, due to a bug in grub-core/net/drivers/efi/efinet.c not fixed until bzr revision 4751 you'll need to build grub-efi-x86_64-bzrAUR as well. The Arch User Repository article describes how to build AUR packages.

GRUB dark-magic will set root=(tftp,10.0.0.1) automatically, so that the kernel and initramfs are transferred via TFTP without any additional configuration, though you might want to set it explicitly if you have any other non-tftp menuentries.

Pxelinux

Note: Syslinux at present has no UEFI networking stack, so you'll be unable to use syslinux-efi-gitAUR (as is possible with #GRUB) and still expect to be able to tftp your kernel and initramfs; pxelinux still works fine for legacy PXE booting

We also created the pxelinux.cfg directory, which is where pxelinux searches for configuration files by default. Because we don't want to discriminate between different host MACs, we then create the default configuration.

Note: You will need to change nbd_host and/or nfsroot, respectively, to match your network configuration (the address of the NFS/NBD server)

The pxelinux configuration syntax identical to syslinux; refer to the upstream documentation for more information.

The kernel and initramfs will be transferred via TFTP, so the paths to those are going to be relative to the TFTP root. Otherwise, the root filesystem is going to be the NFS mount itself, so those are relative to the root of the NFS server.

Additional mountpoints

NBD root

In late boot, you'll want to switch your root filesystem mount to both rw, and enable compress=lzo, for much improved disk performance in comparison to NFS.

# vim "$root/etc/fstab"

/dev/nbd0 / btrfs rw,noatime,discard,compress=lzo 0 0

Program state directories

The factual accuracy of this article or section is disputed.

Reason: systemd can apparently be told to not use persistent logging, and does this by default anyway then /var/log is tmpfs (Discuss in Talk:Diskless system#)

You could mount /var/log, for example, as tmpfs so that logs from multiple hosts don't mix unpredictably, and do the same with /var/spool/cups, so the 20 instances of cups using the same spool don't fight with each other and make 1,498 print jobs and eat an entire ream of paper (or worse: toner cartridge) overnight.

It would be best to configure software that has some sort of state/database to use unique state/database storage directories for each host. If you wanted to run puppet, for example, you could simply use the %H specifier in the puppet unit file:

If neither of these approaches are appropriate, the last sane option would be to create a systemd generator that creates a mount unit specific to the current host (specifiers are not allowed in mount units, unfortunately).

Using initcpio to mount /var in early-boot

The factual accuracy of this article or section is disputed.

Reason: The initramfs should contain only the hooks required to mount the root filesystem and nothing more. (Discuss in Talk:Diskless system#)

It is possible to avoid systemd unit files entirely by mounting /var (or subdirectories) in early boot with initcpio. You can use the following files as guides:

#!/usr/bin/ash
run_latehook() {
local HOSTNAME=$(/usr/bin/hostname)
msg ":: mounting /var for $HOSTNAME"
nfsmount -o v3,nolock,nocto 192.168.0.1:/srv/nfs/cluster-store/vars/$HOSTNAME /new_root/var
if [ $? -gt 0 ]; then
err "There is no /var directory for this node on the server. The node will not work correctly without one."
while true; do
sleep 1m
done
fi
}

You will then need to add var to the end of HOOKS in /etc/mkinitcpio.conf on the client, and re-run mkinitcpio -p linux.

Client boot

NBD

The factual accuracy of this article or section is disputed.

Reason: When using COW on the server, the clients all effectively have read-only mounts of the original filesystem; it should theoretically be safe to do a read-write mount on the NBD server (Discuss in Talk:Diskless system#)

If you're using NBD, you'll need to umount the arch.img before/while you boot your client.

This makes things particularly interesting when it comes to kernel updates. You can't have your client filesystem mounted while you're booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.

You'll need to first copy $root/boot from the client installation to your tftp root (i.e. /srv/boot).

# cp -r "$root/boot" /srv/boot

You'll then need to umount $root before you start the client.

# umount "$root"

Note: To update the kernel in this setup, you either need to mount /srv/boot using NFS in fstab on the client (prior to doing the kernel update) or mount your client filesystem after the client has disconnected from NBD