view · edit · attach · print · history

The contents of this website are Copyright (c)2004 by Brian Manning <brian at antlinux dot com>. Please do not reuse any of the content on this website without permission from the author.

Background

I purchased a Soekris net4501 in 2002. The net4501 uses a CompactFlash card for storage, so every time I made a change to my test Linux system, I had to pull the card out of the Soekris, mount it on another machine, and copy down my changes. I figured there had to be a better way.

There is.

You can get the Soekris (and a lot of other embedded devices it seems) to do what's called PXE (Pre-execution Environment) boot, or in other words, to boot off of the network and download all of the software that will be used to run off of the network as well. Intel created the PXE specification, I'm sure if you search their website you can find the technical documentation on how PXE is supposed to work.

Note that PXE does not require the bootp protocol, so you don't need to have bootp in your kernel or running on the machine, only DHCP and TFTP.

I also have a page with My AntLinux.LinuxOnSoekris notes, where the below notes about PXEbooting originally appeared.

TFTP/Server Security Notes

WARNING: PXE booting a machine requires extra software to be running on your machine in the form of a TFTP server; it's strongly recommended that you run TFTP on a private network, or behind a firewall, as most TFTP servers have no access controls beyond tcpwrappers. This means that if someone on the Internet were to find your TFTP server, they could start sending files to it and receiving files from it.

Here's how I blocked access to tftp from the big bad Internet by using the following ipchains rules:

 /sbin/ipchains -A input -s 192.168.0.0/24 -d 192.168.0.1/32 69 -p udp -j ACCEPT 
 /sbin/ipchains -A input -s 0/0 -d 192.168.0.1/32 69 -p udp -j REJECT

Assuming your server's IP is 192.168.0.1, the first rule allows traffic from any host on the 192.168.0.0/24 network to pass to the TFTP server that's listening for UDP traffic on port 69. The second rule will basically drop traffic that the does not match the first rule, so for any host that's NOT on the 192.168.0.0/24 network, traffic sent to that port on the server will be sent to the Great Bit Bucket In The Sky.

Here's the equivalent iptables version:

 /sbin/iptables -t filter -A input -s 192.168.0.0/24 -d 192.168.0.1/32 
 -p udp --dport 69 -j ACCEPT
 /sbin/iptables -t filter -A input -s 0/0 -d -d 192.168.0.1/32 
 -p udp --dport 69 -j DROP

TFTP also requires that DHCP be set up and properly configured; this means that if your TFTP and DHCP server is set up to span two networks (say your internal private network and an external network such as most ISP's that use cable modems and DHCP), your DHCP server must be configured correctly, or you will disrupt service on the shared ISP network and possibly get your service cut.

If you don't understand either of the above two ipchains/iptables commands nor my warning about rogue DHCP servers, then please don't play with tftpd, you're not ready for it.

PXEboot Setup

Install the following Debian packages on the server:

  • dhcp3-server - ISC's DHCP server
  • tftpd-hpa ( H. Peter Anvin's TFTP server)
  • syslinux, also from H. Peter Anvin. syslinux contains a precompiled binary for x86 machines called pxelinux, which basically is what bootstraps (or starts) the system after PXE hands off control to it. pxelinux will download a Linux kernel and initrd image from a central server, then boot the downloaded kernel. pxelinux may also boot other UNIX-y type operating system kernels as well.

Red Hat/Fedora/SuSE users could install from source, or there may be similar packages as part of the distributions themselves.

dhcpd Setup

Set up dhcpd so that it listens to the network that you have the Soekris set up on. Here's a good sample dhcpd.conf configuration file:

 # dhcpd.conf
 # Copyright (c) 2004 by Brian Manning <brian (at) antlinux dot com>
 # Originally taken from Debian dhcpd3-server package maintained by 
 # Eloy A. Paris <peloy (at) debian dot org>
 # using the source provided by ISC <http://www.isc.org/sw/dhcp/>

 # change 'somedomain.com' to match your domain, or your ISP's domain name
 option domain-name "somedomain.com";
 # list of DNS servers for clients, separate multiple servers with commas only,
 # no other whitespace is allowed
 option domain-name-servers xxx.xxx.xxx.xxx,xxx.xxx.xxx.xxx;

 # lease times in seconds
 default-lease-time 600;
 max-lease-time 7200;

 # the subnet we'll be doing development on
 # note that this is a block declaration, it's enclosed in opening and 
 # closing curly braces '{}'.  If you miss a closing curly brace, DHCPd 
 # may not start
 subnet 192.168.0.0 netmask 255.255.255.0 {
    range 192.168.0.10 192.168.0.20;
    option broadcast-address 192.168.0.255;
    option routers 192.168.0.1;
    option subnet-mask 255.255.255.0;
 }

 # any other IP's on this machine need to be added with an empty block
 # this basically tells dhcpd to ignore DHCP requests from the network card
 # bound to this IP address.  A good example of using this would be a host
 # that's connected to a cable modem that obtains it's own IP address using
 # DHCP as a client; you set the below IP for your cable modem network
 # and that way you're not running a rogue DHCP server, and have your
 # service cut as a result.  If your external address is also a DHCP address,
 # then most likely what you would do is to script your DHCP client to change
 # this file with every change in IP that your ISP gives you via DHCP.  It's 
 # ugly, yes, but your only alternative is to find a way to get a static
 # IP address
 subnet xxx.xxx.xxx.xxx netmask 255.255.255.0 {
 }

 # PXE clients will live inside of a group{} declaration
 group {
  # the next-server is the IP address of your tftp server
  next-server 192.168.0.1;

  # individual hosts within this group
  # again note the opening and closing curly braces '{}'
  host pulgas {
    # MAC address of the port on the soekris that you have plugged
    # into your test network most likely port 0.  Get the MAC address
    # from your terminal window when you boot the Soekris
    hardware ethernet 00:00:24:c0:95:78;

    # the IP address to give to the client when it makes it's DHCP request
    fixed-address 192.168.0.90;

    # the filename to serve to the client.  Below I serve a raw 
    # kernel image, which works, but barfs as the kernel itself
    # will panic without some direction as to where it can find 
    # some init style program (/sbin/init, /etc/init.d/rcS,
    # /linuxrc, etc.). That's the benefit to using pxelinux, it 
    # can do multistage boots, where it downloads both the kernel 
    # and an initrd ramdisk image

    #filename "vmlinuz.soekris";

    # this is how you get pxelinux to run, the file pxelinux.0
    # is the PXE 'bootloader', which will then download it's own
    # config file 'pxelinux.cfg' and run with the configuration
    # parameters listed in that file
    filename "pxelinux.0"
  }
 } # group

pxelinux Setup

  1. In the directory that tftp uses to keep files (it's listed in /etc/inetd.conf, it will be the last argument on the 'tftpd' line in that file), copy the pxelinux.0 file from the syslinux distribution.
 cd /var/
 sudo mkdir tftpd
 cd tftpd
 sudo cp /usr/lib/syslinux/pxelinux.0 .
  1. Create a pxelinux.cfg directory in the same directory you put the pxelinux.0 file into.
    • sudo mkdir pxelinux.cfg
  2. Copy your kernel image and initrd image to the same directory as pxelinux.0.
    • sudo cp /tmp/vmlinuz.somehost /var/tftpd
    • sudo cp /tmp/somehost.initrd.gz /var/tftpd
  3. Build a config file for the host; the name of the config file will be the IP address in hexadecimal of the host you are PXEbooting; for example, the IP address 192.168.0.90 is C0.A8.01.5A hexadecimal, which makes the config filename for that host C0A8015A. You can use the ConvertingToHexadecimal page to learn how to use bc from the command line to convert decimal IP octets to hexadecimal.
    • sudo vi /var/ftpd/pxelinux.cfg/C0A8015A
  4. Inside your config file, put all of your syslinux/pxelinux commands (see the PXELINUX page for much more information). Here's an example config file:
 SERIAL 0,9600
 DEFAULT 1
 TIMEOUT 30
 PROMPT 1
 DISPLAY banner.hostname.txt
 LABEL 1
        KERNEL vmlinuz
        APPEND console=ttyS0,9600n8 root=/dev/ram0 load_ramdisk=1
initrd=initrd.gz ramdisk_size=4096

If you really want to be tricky, you can create a directory under $TFTP_ROOT for each host that you will be booting via PXE. So the above example config file would look like this for host somehost:

 SERIAL 0,9600
 DEFAULT 1
 TIMEOUT 30
 PROMPT 1
 DISPLAY /somehost/banner.hostname.txt
 LABEL 1
        KERNEL /somehost/vmlinuz
        APPEND console=ttyS0,9600n8 root=/dev/ram0 load_ramdisk=1
 initrd=/somehost/initrd.gz ramdisk_size=4096

tftpd setup

  • For security reasons, you might think about changing the tftp command line in /etc/inetd.conf by placing -s (switch to tell tftpd to chroot) before the directory that is to host all of your TFTP files. chroot makes it so that if someone breaks your tftp server, the only files they will have access to are the files inside of the tftp root directory, the directory specificed in the /etc/inetd.conf file.
  • Edit /etc/inetd.conf, and uncomment the tftp line. Make sure that the directory that tftpd will use is the same as what you create above.
  • Restart inetd on your server by issuing the following command as root:
    • kill -HUP `cat /var/run/inetd.pid`

PXEBooting your Soekris

Power up the Soekris, then hit Ctrl-P when prompted to enter the PROM. From the PROM prompt, enter boot f0 to PXEboot the device.

Once the pxelinux.0 bootloader runs, it will download the config file from the server. pxelinux knows where the config file will be on the tftp server based on the IP address of the machine that pxelinux is running on (see the above configuration if this is not clear). Running the above config file will cause pxelinux and the kernel to display output on the serial port COM1: in DOS or /dev/ttyS0 in Linux. When the kernel is booted, it load the named initrd image into ramdisk automatically. You should be able to keep multiple kernel images around just by changing name of the file on the KERNEL line in the above pxelinux config file, or adding multiple LABEL stanzas to the pxelinux config file, and then choosing which image you want to boot when pxelinux prompts you.

If you run into any issues, just watch the log files on the server (/var/log/daemon.log and /var/log/messages), as well as the screen display of the machine that is PXEbooting.

view · edit · attach · print · history
Page last modified on July 05, 2006, at 05:07 PM