* A [[Tftpd_server|TFTP]] server to transfer the boot image (a requirement of all PXE option roms).

+

* A [[Tftpd_server|TFTP]] server to transfer the boot image (a requirement of all PXE option roms).

* A form of network storage ([[NFS]] or NBD) to export the Arch installation to the diskless node.

* A form of network storage ([[NFS]] or NBD) to export the Arch installation to the diskless node.

{{Note|{{pkg|dnsmasq}} is capable of simultaneously acting as both DHCP and TFTP server. For more information, see the [[dnsmasq]] article.}}

{{Note|{{pkg|dnsmasq}} is capable of simultaneously acting as both DHCP and TFTP server. For more information, see the [[dnsmasq]] article.}}

−

===DHCP===

+

=== DHCP ===

−

Install ISC {{pkg|dhcp}}.

+

Install ISC {{Pkg|dhcp}} and configure it:

−

{{bc|# pacman -Syu dhcp}}

+

{{hc|/etc/dhcpd.conf|<nowiki>

−

+

allow booting;

−

Configure ISC DHCP.

−

−

{{hc|# vim /etc/dhcpd.conf|2=

−

<nowiki>allow booting;

allow bootp;

allow bootp;

Line 51:

Line 47:

range 10.0.0.128 10.0.0.254;

range 10.0.0.128 10.0.0.254;

}

}

−

}</nowiki>}}

+

}

+

</nowiki>}}

{{Note|{{ic|next-server}} should be the address of the TFTP server; everything else should be changed to match your network}}

{{Note|{{ic|next-server}} should be the address of the TFTP server; everything else should be changed to match your network}}

Line 57:

Line 54:

RFC 4578 defines the "Client System Architecture Type" dhcp option. In the above configuration, if the PXE client requests an x86_64-efi binary (type 0x7), we appropriately give them one, otherwise falling back to the legacy binary. This allows both UEFI and legacy BIOS clients to boot simultaneously on the same network segment.

RFC 4578 defines the "Client System Architecture Type" dhcp option. In the above configuration, if the PXE client requests an x86_64-efi binary (type 0x7), we appropriately give them one, otherwise falling back to the legacy binary. This allows both UEFI and legacy BIOS clients to boot simultaneously on the same network segment.

−

Start ISC DHCP.

+

Start ISC DHCP [[systemd]] service.

−

{{bc|# systemctl start dhcpd4.service}}

+

=== TFTP ===

−

−

===TFTP===

The TFTP server will be used to transfer the bootloader, kernel, and initramfs to the client.

The TFTP server will be used to transfer the bootloader, kernel, and initramfs to the client.

Line 67:

Line 62:

Set the TFTP root to {{ic|/srv/arch/boot}}. See [[Tftpd server]] for installation and configuration.

Set the TFTP root to {{ic|/srv/arch/boot}}. See [[Tftpd server]] for installation and configuration.

−

===Network storage===

+

=== Network storage ===

The primary difference between using NFS and NBD is while with both you can in fact have multiple clients using the same installation, with NBD (by the nature of manipulating a filesystem directly) you'll need to use the {{ic|copyonwrite}} mode to do so, which ends up discarding all writes on client disconnect. In some situations however, this might be highly desirable.

The primary difference between using NFS and NBD is while with both you can in fact have multiple clients using the same installation, with NBD (by the nature of manipulating a filesystem directly) you'll need to use the {{ic|copyonwrite}} mode to do so, which ends up discarding all writes on client disconnect. In some situations however, this might be highly desirable.

{{Note|If you're not worried about data loss in the event of network and/or server failure, replace {{ic|sync}} with {{ic|async}}--additional options can be found in the [[NFS]] article.}}

−

−

====NBD====

−

−

Install {{pkg|nbd}}.

−

−

{{bc|# pacman -Syu nbd}}

−

−

Configure nbd.

{{hc|# vim /etc/nbd-server/config|2=

{{hc|# vim /etc/nbd-server/config|2=

Line 118:

Line 91:

copyonwrite = false}}

copyonwrite = false}}

−

{{note|Set {{ic|copyonwrite}} to true if you want to have multiple clients using the same NBD share simultaneously; refer to {{ic|man 5 nbd-server}} for more details.}}

+

{{Note|Set {{ic|copyonwrite}} to true if you want to have multiple clients using the same NBD share simultaneously; refer to {{ic|man 5 nbd-server}} for more details.}}

−

−

Start nbd.

−

{{bc|# systemctl start nbd.service}}

+

Start '''nbd''' systemd service.

−

==Client installation==

+

== Client installation ==

Next we will create a full Arch Linux installation in a subdirectory on the server. During boot, the diskless client will get an IP address from the DHCP server, then boot from the host using PXE and mount this installation as its root.

Next we will create a full Arch Linux installation in a subdirectory on the server. During boot, the diskless client will get an IP address from the DHCP server, then boot from the host using PXE and mount this installation as its root.

−

===Directory setup===

+

=== Directory setup ===

Create a [[Wikipedia: Sparse file|sparse file]] of at least 1 gigabyte, and create a btrfs filesystem on it (you can of course also use a real block device or [[LVM]] if you so desire).

Create a [[Wikipedia: Sparse file|sparse file]] of at least 1 gigabyte, and create a btrfs filesystem on it (you can of course also use a real block device or [[LVM]] if you so desire).

−

{{bc|1=

+

# truncate -s 1G /srv/arch.img

−

# truncate -s 1G /srv/arch.img

+

# mkfs.btrfs /srv/arch.img

−

# mkfs.btrfs /srv/arch.img

+

# export root=/srv/arch

−

# export root=/srv/arch

+

# mkdir -p "$root"

−

# mkdir -p "$root"

+

# mount -o loop,discard,compress=lzo /srv/arch.img "$root"

−

# mount -o loop,discard,compress=lzo /srv/arch.img "$root"}}

{{Note|Creating a separate filesystem is required for NBD but optional for NFS and can be skipped/ignored.}}

{{Note|Creating a separate filesystem is required for NBD but optional for NFS and can be skipped/ignored.}}

−

===Bootstrapping installation===

+

=== Bootstrapping installation ===

−

−

Install {{pkg|devtools}} and {{pkg|arch-install-scripts}}, and run {{ic|mkarchroot}}.

You will then need to append {{ic|nbd}} to your {{ic|HOOKS}} array after {{ic|net}}; {{ic|net}} will configure your networking for you, but not attempt a NFS mount if {{ic|nfsroot}} is not specified in the kernel line.

You will then need to append {{ic|nbd}} to your {{ic|HOOKS}} array after {{ic|net}}; {{ic|net}} will configure your networking for you, but not attempt a NFS mount if {{ic|nfsroot}} is not specified in the kernel line.

−

==Client configuration==

+

== Client configuration ==

In addition to the setup mentioned here, you should also set up your [[HOSTNAME#Set_the_host_name|hostname]], [[Timezone#Time_Zone|timezone]], [[Locale#Setting_system-wide_locale|locale]], and [[KEYMAP|keymap]], and follow any other relevant parts of the [[Installation Guide]].

In addition to the setup mentioned here, you should also set up your [[HOSTNAME#Set_the_host_name|hostname]], [[Timezone#Time_Zone|timezone]], [[Locale#Setting_system-wide_locale|locale]], and [[KEYMAP|keymap]], and follow any other relevant parts of the [[Installation Guide]].

−

===Bootloader===

+

=== Bootloader ===

−

====[[GRUB]]====

+

==== GRUB ====

−

{{Merge|GRUB2|}}

+

{{Merge|GRUB|}}

−

Though poorly documented, grub also supports being loaded via PXE. Because of an [https://savannah.gnu.org/bugs/?36755 endianness bug] in {{ic|grub-core/net/tftp.c}} not fixed until bzr revision [http://bzr.savannah.gnu.org/lh/grub/trunk/grub/revision/4548 4548] ({{pkg|grub-bios}} 2.00 is revision [http://bzr.savannah.gnu.org/lh/grub/trunk/grub/revision/4542 4542]), you will need to build {{aur|grub-bios-bzr}}; it makes the most sense to do this on the host installation. In addition, due to a bug in {{ic|grub-core/net/drivers/efi/efinet.c}} not fixed until bzr revision [http://bzr.savannah.gnu.org/lh/grub/trunk/grub/revision/4751 4751] you'll need to build {{aur|grub-efi-x86_64-bzr}} as well. The [[Arch User Repository]] article describes how to build AUR packages.

Next, flip to the {{aur|grub-efi-x86_64-bzr}} package you built, and run {{ic|grub-mknetdir}} exactly as above a second time.

+

Luckily for us, grub-mknetdir creates prefixes for all currently compiled/installed targets, and the {{pkg|grub}} maintainers were nice enough to give us both in the same package, thus grub-mknetdir only needs to be run once.

[[Syslinux|Pxelinux]] is provided by {{Pkg|syslinux}}, see [[Syslinux#Pxelinux|here]] for detail.

−

{{Note|syslinux at present has no UEFI networking stack, so you'll be unable to use {{aur|syslinux-efi-git}} (as is possible with [[#GRUB]]) and still expect to be able to tftp your kernel and initramfs; pxelinux still works fine for legacy PXE booting}}

+

=== Additional mountpoints ===

−

Install {{pkg|syslinux}}.

+

==== NBD root ====

−

{{bc|1=

+

In late boot, you'll want to switch your root filesystem mount to both {{ic|rw}}, and enable {{ic|<nowiki>compress=lzo</nowiki>}}, for much improved disk performance in comparison to [[NFS]].

−

# pacman -Syu syslinux}}

−

−

Copy the pxelinux bootloader (provided by the syslinux package) to the boot directory of the client.

−

{{bc|1=

+

{{hc|# vim "$root/etc/fstab"|<nowiki>

−

# cp /usr/lib/syslinux/pxelinux.0 "$root/boot"

+

/dev/nbd0 / btrfs rw,noatime,discard,compress=lzo 0 0

−

# mkdir "$root/boot/pxelinux.cfg"}}

−

−

We also created the {{ic|pxelinux.cfg}} directory, which is where pxelinux searches for configuration files by default. Because we don't want to discriminate between different host MACs, we then create the {{ic|default}} configuration.

{{Note|You will need to change {{ic|nbd_host}} and/or {{ic|nfsroot}}, respectively, to match your network configuration (the address of the NFS/NBD server)}}

−

−

The pxelinux configuration syntax identical to syslinux; refer to the upstream documentation for more information.

−

−

The kernel and initramfs will be transferred via TFTP, so the paths to those are going to be relative to the TFTP root. Otherwise, the root filesystem is going to be the NFS mount itself, so those are relative to the root of the NFS server.

{{Accuracy|systemd can apparently be told to not use persistent logging, and does this by default anyway then /var/log is tmpfs}}

{{Accuracy|systemd can apparently be told to not use persistent logging, and does this by default anyway then /var/log is tmpfs}}

Line 287:

Line 198:

It would be best to configure software that has some sort of state/database to use unique state/database storage directories for each host. If you wanted to run [http://puppetlabs.com/ puppet], for example, you could simply use the {{ic|%H}} specifier in the puppet unit file:

It would be best to configure software that has some sort of state/database to use unique state/database storage directories for each host. If you wanted to run [http://puppetlabs.com/ puppet], for example, you could simply use the {{ic|%H}} specifier in the puppet unit file:

−

{{hc|# vim "$root/etc/systemd/system/puppetagent.service"|2=<nowiki>

+

{{hc|# vim "$root/etc/systemd/system/puppetagent.service"|<nowiki>

[Unit]

[Unit]

Description=Puppet agent

Description=Puppet agent

Line 300:

Line 211:

[Install]

[Install]

−

WantedBy=multi-user.target</nowiki>}}

+

WantedBy=multi-user.target

+

</nowiki>}}

Puppet-agent creates vardir and ssldir if they do not exist.

Puppet-agent creates vardir and ssldir if they do not exist.

Line 306:

Line 218:

If neither of these approaches are appropriate, the last sane option would be to create a [http://www.freedesktop.org/wiki/Software/systemd/Generators systemd generator] that creates a mount unit specific to the current host (specifiers are not allowed in mount units, unfortunately).

If neither of these approaches are appropriate, the last sane option would be to create a [http://www.freedesktop.org/wiki/Software/systemd/Generators systemd generator] that creates a mount unit specific to the current host (specifiers are not allowed in mount units, unfortunately).

−

==Client boot==

+

== Client boot ==

−

===NBD===

+

=== NBD ===

{{Accuracy|When using COW on the server, the clients all effectively have read-only mounts of the original filesystem; it should theoretically be safe to do a read-write mount on the NBD server}}

{{Accuracy|When using COW on the server, the clients all effectively have read-only mounts of the original filesystem; it should theoretically be safe to do a read-write mount on the NBD server}}

Line 316:

Line 228:

This makes things particularly interesting when it comes to kernel updates. You can't have your client filesystem mounted while you're booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.

This makes things particularly interesting when it comes to kernel updates. You can't have your client filesystem mounted while you're booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.

−

You'll need to first copy $root/boot from the client installation to your tftp root (i.e. /srv/boot).

+

You'll need to first copy {{ic|$root/boot}} from the client installation to your tftp root (i.e. {{ic|/srv/boot}}).

−

{{bc|# cp -r "$root/boot" /srv/boot}}

+

# cp -r "$root/boot" /srv/boot

You'll then need to umount $root before you start the client.

You'll then need to umount $root before you start the client.

−

{{bc|# umount "$root"}}

+

# umount "$root"

{{Note|To update the kernel in this setup, you either need to mount {{ic|/srv/boot}} using [[NFS]] in [[fstab]] on the client (prior to doing the kernel update) or mount your client filesystem after the client has disconnected from NBD}}

{{Note|To update the kernel in this setup, you either need to mount {{ic|/srv/boot}} using [[NFS]] in [[fstab]] on the client (prior to doing the kernel update) or mount your client filesystem after the client has disconnected from NBD}}

Note: next-server should be the address of the TFTP server; everything else should be changed to match your network

RFC 4578 defines the "Client System Architecture Type" dhcp option. In the above configuration, if the PXE client requests an x86_64-efi binary (type 0x7), we appropriately give them one, otherwise falling back to the legacy binary. This allows both UEFI and legacy BIOS clients to boot simultaneously on the same network segment.

TFTP

The TFTP server will be used to transfer the bootloader, kernel, and initramfs to the client.

Set the TFTP root to /srv/arch/boot. See Tftpd server for installation and configuration.

Network storage

The primary difference between using NFS and NBD is while with both you can in fact have multiple clients using the same installation, with NBD (by the nature of manipulating a filesystem directly) you'll need to use the copyonwrite mode to do so, which ends up discarding all writes on client disconnect. In some situations however, this might be highly desirable.

NBD

Note: Set copyonwrite to true if you want to have multiple clients using the same NBD share simultaneously; refer to man 5 nbd-server for more details.

Start nbd systemd service.

Client installation

Next we will create a full Arch Linux installation in a subdirectory on the server. During boot, the diskless client will get an IP address from the DHCP server, then boot from the host using PXE and mount this installation as its root.

Directory setup

Create a sparse file of at least 1 gigabyte, and create a btrfs filesystem on it (you can of course also use a real block device or LVM if you so desire).

Luckily for us, grub-mknetdir creates prefixes for all currently compiled/installed targets, and the grub maintainers were nice enough to give us both in the same package, thus grub-mknetdir only needs to be run once.

GRUB dark-magic will set root=(tftp,10.0.0.1) automatically, so that the kernel and initramfs are transferred via TFTP without any additional configuration, though you might want to set it explicitly if you have any other non-tftp menuentries.

Pxelinux

Additional mountpoints

NBD root

In late boot, you'll want to switch your root filesystem mount to both rw, and enable compress=lzo, for much improved disk performance in comparison to NFS.

# vim "$root/etc/fstab"

/dev/nbd0 / btrfs rw,noatime,discard,compress=lzo 0 0

Program state directories

The factual accuracy of this article or section is disputed.

Reason: systemd can apparently be told to not use persistent logging, and does this by default anyway then /var/log is tmpfs (Discuss in Talk:Diskless system#)

You could mount /var/log, for example, as tmpfs so that logs from multiple hosts don't mix unpredictably, and do the same with /var/spool/cups, so the 20 instances of cups using the same spool don't fight with each other and make 1,498 print jobs and eat an entire ream of paper (or worse: toner cartridge) overnight.

It would be best to configure software that has some sort of state/database to use unique state/database storage directories for each host. If you wanted to run puppet, for example, you could simply use the %H specifier in the puppet unit file:

If neither of these approaches are appropriate, the last sane option would be to create a systemd generator that creates a mount unit specific to the current host (specifiers are not allowed in mount units, unfortunately).

Client boot

NBD

The factual accuracy of this article or section is disputed.

Reason: When using COW on the server, the clients all effectively have read-only mounts of the original filesystem; it should theoretically be safe to do a read-write mount on the NBD server (Discuss in Talk:Diskless system#)

If you're using NBD, you'll need to umount the arch.img before/while you boot your client.

This makes things particularly interesting when it comes to kernel updates. You can't have your client filesystem mounted while you're booting a client, but that also means you need to use a kernel separate from your client filesystem in order to build it.

You'll need to first copy $root/boot from the client installation to your tftp root (i.e. /srv/boot).

# cp -r "$root/boot" /srv/boot

You'll then need to umount $root before you start the client.

# umount "$root"

Note: To update the kernel in this setup, you either need to mount /srv/boot using NFS in fstab on the client (prior to doing the kernel update) or mount your client filesystem after the client has disconnected from NBD