PXEboot with OpenWRT

How to boot/install a bunch of servers over the network with a router ? This is how ...

Openwrt is awesome and free, and I'm sure that your router will like it as I did. Also I'm sure that you have some free-spare-conference-usb-keychain (1GB will be enough), and of course internet connection is needed.

In this setup we were using TP-Link WDR-4300 router with latest Openwrt trunk version (Bleeding Edge, r47723) + 8GB USB keychain with awesome grml distribution. At the end, I was able to achieve 8 MB/s read rate over NFS (faster USB keychain - much happiness) 

So, let's go with the setup ... 

Local setup:

Create working directory:

mkdir ~/pxeboot
cd ~/pxeboot

Download grml netboot version:

wget http://download.grml.org/grml_netboot_package_grml64-full_2014.11.tar.bz2

Extract image:

tar -xvf grml_netboot_package_grml64-full_2014.11.tar.bz2
mv grml_netboot_package_grml64-full_2014.11/* ~/pxeboot
rm -fr grml_netboot_package_grml64-full_2014.11
rm grml_netboot_package_grml64-full_2014.11.tar.bz2

Download and mount grml liveCD image:

wget http://download.grml.org/grml64-full_2014.11.iso
mkdir /tmp/grml
sudo mount -o loop grml64-full_2014.11.iso /tmp/grml

Sync with working directory:

rsync -avP /tmp/grml/ ~/pxeboot/tftpboot

Edit pxelinux.cfg:

vim tftpboot/pxelinux.cfg/default

It should look like this:

default grml
label grml
  menu DEFAULT
  menu label grml64-full ^Standard (2014.11, amd64)
  kernel vmlinuz
  append initrd=initrd.img root=/dev/nfs rw nfsroot=192.168.1.1:/mnt/tftpboot/ boot=live live-media-path=/live/grml64-full/ bootid=grml64full201411 apm=power-off nomce noprompt noeject vga=791 

  text help
                                        Grml is a Debian based Linux live
                                        system for system administrators
                                        and users of text tools.

                                                         http://grml.org/
  endtext

Prepare USB keychain and sync files

Format and mount USB keychain partition:

mkfs.ext4 -L pxeboot /dev/sdn1 # replace sdn1 with your partition
mkdir /tmp/pxebootusb
mount /dev/sdn1 /tmp/pxebootusb
rsync -avP ~/pxeboot/ /tmp/pxebootusb

Prepare OpenWRT router

Install necessearry packages:

opkg update
opkg install nfs-kernel-server kmod-fs-ext4 kmod-usb-storage block-mount

Insert and mount USB keychain :

mount /dev/sda1 /mnt

Make permanent/automount:

vim /etc/config/fstab

config 'global'
        option  anon_swap       '0'
        option  anon_mount      '0'
        option  auto_swap       '0'
        option  auto_mount      '1'
        option  delay_root      '5'
        option  check_fs        '0'

config 'mount'
        option 'target'   '/mnt'
        option 'label'    'pxeboot'
        option 'fstype'   'ext4'
        option 'options'  'rw,sync'
        option 'enabled'  '1'
        option 'enabled_fsck' '0'

/etc/init.d/fstab enable

Create NFS export

vim /etc/exports

/mnt/tftpboot   *(rw,sync,no_subtree_check)

Restart nfs service:

/etc/init.d/rpcd restart
/etc/init.d/nfsd restart

Modify network configuration:

vim /etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config interface 'lan'
        option type 'bridge'
        option ifname 'eth0.1'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option ifname 'wlan1'
        option proto 'dhcp'

config interface 'wan6'
        option ifname 'eth0.2'
        option proto 'dhcpv6'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '1 2 3 4 0t'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '5 0t'

Restart network service:

/etc/init.d/network restart

Modify dnsmasq configuration:

vim /etc/config/dhcp

config dnsmasq
        option domainneeded '1'
        option boguspriv '1'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.auto'
        option localservice '1'
        option enable_tftp '1'
        option tftp_root '/mnt/tftpboot'
        option dhcp_boot 'pxelinux.0'

config dhcp 'lan'
        option interface 'lan'
        option start '100'
        option limit '150'
        option leasetime '12h'
        option dhcpv6 'server'
        option ra 'server'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'

Restart dnsmasq service:

/etc/init.d/dnsmasq restart