Several weeks of testing Netboot for Raspberry Pi with /boot and root (/) on NFS went fine until I hit a problem with home-assistant due to container didn’t want to start at all with strange message failed to reserve container name is reserved for.
I knew that there is iSCSI feature on my Synology, but I never tried it before. So the time has come to try.
Plan was quite simple - partially to use what already was done with PXE and booting from TFTP, but now root (/) should be on iSCSI disks, not on NFS mount as before, /boot can be still left on NFS mount.
After reading several good articles (all of them are left at the end) I’ve understood that in addition to what was done initrd has to be recompiled with iscsi modules as by default they are not there so OS won’t start without it from iscsi disk.
So at the end I’ve got the following release plan:
- Configure the DHCP service on your router (the same as Netboot for Raspberry Pi with /boot and root (/) on NFS)
- Configure the NFS and TFTP services on your NAS (only /boot will be on NFS now)
- Configure iSCSI and test it (on storage and RPi ends)
- Recompile initrd with iscsi modules on RPi
- Copy /boot to NFS and update boot configuration for iSCSI
- Copy / to iSCSI disk
On NAS rpi-tftpboot folder should be created, /boot file systems for each RPi will be stored there.
NFS-service has to be active and RW-access should be provided for RPi-nodes to rpi-tftpboot folder.
TFTP-service has to be enabled either and rpi-tftpboot folder should be specified as TFTP root folder.
Configure iSCSI and test it
From storage side volumes (disks/LUNs) have to be created and mapped to targets (one target per each RPi server), I use 16GB volumes (one for each RPi) and simple CHAP authentication (not mutual). On examples below 192.168.8.99 is my Synology.
From server side open-iscsi packet has to be installed and configured
apt-get install open-iscsi -y
In /etc/iscsi/iscsid.conf file the same authentication credentials should be specified RPi to be able to login to iSCSI server to see available LUN. Restart service after changing configuration.
# egrep -v "^#" /etc/iscsi/iscsid.conf | egrep "(node.session.auth)|node.startup" node.startup = automatic node.session.auth.authmethod = CHAP node.session.auth.username = rpi1 node.session.auth.password = mysupersecurepassword # systemctl restart iscsid.service
Now you can try to run discovery and see what is available on iscsi-server (storage)
iscsiadm -m discovery -t st -p 192.168.8.99 192.168.8.99:3260,1 iqn.2000-01.com.synology:storage.rpi1.ebab984136
Let’s try to login:
# iscsiadm --mode node --targetname "iqn.2000-01.com.synology:storage.rpi1.ebab984136" --portal 192.168.8.99 --login Logging in to [iface: default, target: iqn.2000-01.com.synology:storage.rpi1.ebab984136, portal: 192.168.8.99,3260] (multiple) Login to [iface: default, target: iqn.2000-01.com.synology:storage.rpi1.ebab984136, portal: 192.168.8.99,3260] successful.
Now we should be able to see new disk available (/dev/sda) like usual local drive:
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 16G 0 disk
It can be formatted as ext4, just remember UUID of new created FS - it will be needed configuring boot later.
mkfs.ext4 /dev/sda mkdir /mnt/iscsi mount /dev/sda /mnt/iscsi blkid /dev/sda /dev/sda: UUID="de208c76-4d29-467f-b6c5-4d7c9f0e0827" TYPE="ext4"
Recompile initrd with iscsi modules on RPi
Pretty simple and fast procedure, just don’t forget to update system before to do that.
apt-get update && apt-get upgrade -y touch /etc/iscsi/iscsi.initramfs update-initramfs -v -k `uname -r` -c ls -l /boot/initrd.img-5.10.17-v7l+ -rwxrwxrwx 1 1024 users 8792648 Jun 12 12:31 /boot/initrd.img-5.10.17-v7l+
Copy /boot to NFS and update boot configuration for iSCSI
The same way as it was mentioned in Netboot for Raspberry Pi with /boot and root (/) on NFS post /boot should be copied to NFS rpi-tftpboot folder, each RPi should have their own boot sub-folder there (based on RPi s/n).
vcgencmd otp_dump | grep 28: | sed s/.*://g 47102626 mkdir /mnt/boot sudo mount 192.168.8.99:/volume1/rpi-tftpboot /mnt/boot sudo cp -r /boot/* /mnt/boot/47102626
Add to config.txt details about what initrd should be used
cat >> /mnt/boot/47102626/config.txt initramfs initrd.img-5.10.17-v7l+ followkernel [Ctrl+D]
To update cmdline.txt the following details need to know, all of them should be specified instead of current details about root device:
- ISCSI_INITIATOR=iqn.1993-08.org.debian:01:3f3ea6e86ba8 (can be taken from /etc/iscsi/initiatorname.iscsi)
So cmdline.txt should look like that:
cat /mnt/boot/47102626/cmdline.txt console=serial0,115200 console=tty1 ip=dhcp root=UUID=de208c76-4d29-467f-b6c5-4d7c9f0e0827 ISCSI_USERNAME=rpi1 ISCSI_PASSWORD=mysupersecurepassword ISCSI_INITIATOR=iqn.1993-08.org.debian:01:d52b48145cc ISCSI_TARGET_NAME=iqn.2000-01.com.synology:storage.rpi1.ebab984136 ISCSI_TARGET_IP=192.168.8.99 ISCSI_TARGET_PORT=3260 rw rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory
Copy / to iSCSI disk
Last step is to copy all data from current / to iscsi disk which is still mounted as /mnt/iscsi
rsync -xa --progress --exclude /mnt / /mnt/iscsi
And edit copied /etc/fstab to mount /boot from NFS and specify new /
cat > /mnt/iscsi/etc/fstab proc /proc proc defaults 0 0 192.168.8.99:/volume1/rpi-tftpboot/47102626 /boot nfs defaults,vers=3,proto=tcp 0 0 UUID=de208c76-4d29-467f-b6c5-4d7c9f0e0827 / ext4 defaults 1 1
After all these steps run reboot and check how it goes. System should boot via PXE, login to iSCSI server, get root disk visible and start from it as usually. After reboot you’ll see something like:
# df -h Filesystem Size Used Avail Use% Mounted on udev 1.8G 0 1.8G 0% /dev tmpfs 383M 17M 367M 5% /run /dev/sda 16G 5.0G 9.9G 34% / tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup 192.168.8.99:/volume1/rpi-tftpboot/47102626 5.5T 4.2T 1.3T 77% /boot
And now home-assistant container works as expected in K3s.