Have you wondered about that weird last boot option on the boot loader page? Wife getting on you about the server in the house and need to move to a datacenter? Well it’s time to learn about network booting.
Network booting ditches your standard boot USB with a local server. This TFTP server delivers your boot files to your 3 nodes. This can be useful in bigger home farms, but is all but mandatory in a datacenter setup.
Network boot setup is quite easy and is centered about configuring a TFTP server. There are essentially 2 options for this, a small dedicated server such as a raspberry pi, or piggybacking on your pfsense or opnsense router. I would recommend the latter as it eliminates another piece of equipment and is probably more reliable.
You should familiarize yourself with this link.
https://github.com/threefoldtech/zos/blob/main/docs/network/setup_farm_network.md
Setting Up Your Router to Allow Network Booting
These steps are for OPNsense, PFsense may differ. These set are required regardless of where you have your TFTP server.
Services>DHCPv4>LAN>Network Booting
Check “Enable Network Booting”
Enter the IP address of your TFTP server under “Set next-server IP” This may be the router’s IP or whatever device you are booting from.
Enter “pxelinux.0” under Set default bios filename
Ignore the TFTP Server settings.
TFTP server setup on a debian machine such as Ubuntu or Raspberry Pi
apt-get update
apt-get install tftpd-hpa
cd /srv/tftp/
wget http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/pxelinux.0
wget https://bootstrap.grid.tf/krn/prod/ --no-check-certificate
mv ipxe-prod.lkrn
tar -xvzf netboot.tar.gz
rm version.info netboot.tar.gz
rm pxelinux.cfg/default
chmod 777 /srv/tftp/pxelinux.cfg (optional if next step fails)
echo ‘default ipxe-prod.lkrn’ >> pxelinux.cfg/default
TFTP Server on a OPNsense router
(I have not used on PFsense, steps are probably similar, but the directory or other small things may differ)
The first step is to download the TFTP server plugin. Go to system>firmware>Status and check for updates, follow prompts to install. Then click the Plugins tab and search for tftp, install os-tftp. Once that is installed go to Services>TFTP (you may need to refresh page). Check the Enable box and input your router ip (normally 192.168.1.1). Click save.
Turn on ssh for your router. In OPNsense it is System>Settings>Administration. Then check the Enable, root login, and password login. Hop over to Putty and connect to your router, normally 192.168.1.1. Login as root and input your password. Hit 8 to enter the shell.
In OPNsense the tftp directory is /usr/local/tftp
cd /usr/local
mkdir tftp
cd ./tftp
fetch http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/pxelinux.0
fetch https://bootstrap.grid.tf/krn/prod/
mv ipxe-prod.lkrn
tar -xvzf netboot.tar.gz
rm version.info netboot.tar.gz
rm pxelinux.cfg/default
echo ‘default ipxe-prod.lkrn’ >> pxelinux.cfg/default
You can get out of shell by entering exit or just closing the window.
3Node Setup
Set the server to BIOS boot and put PXE or network boot as the first choice. At least on Dell machines, make sure you have the network cable in plug 1 or it won’t boot.
Most credit goes to Micheal who sent the linux commands to me, Scott who sent it to him, and some mystery person who sent it to scott. I managed to figure out the server and router settings somehow and adapted the commands to ubuntu and OPNsense.
