More Horms Solutions Projects

Disk-Less Linux with Intel Lan Desk

Horms (Simon Horman)
horms@verge.net.au

February 2002
http://horms.org/projects/diskless/

Abstract

Disk-Less nodes are a useful way of bring up machines on a network. Applications include workstations or cluster nodes. This short paper outlines how to get such a system up and running. Caveats on doing this with Intel Ether Express Pro hardware are covered in detail.

Introduction

The disk in one of machines broke so I decided it was time to try disk-less again. Initially I was going to try and boot of a floppy disk, but I soon discovered that the Intel Ether-Express Pro 100 has an on-board DHCP and TFTP server. Here is my journey.

Kernel

The vmlinuz-2.4.13.mosix was configured with the following to allow an nfs-root to be established:

Networking Options
  TCP/IP networking (CONFIG_INET)
    IP: kernel level autoconfiguration (CONFIG_IP_PNP)
      IP: DHCP support (CONFIG_IP_PNP_DHCP)

File Systems
  /dev file system support (CONFIG_DEVFS_FS)
    Automatically mount at boot (CONFIG_DEVFS_MOUNT)

  Network File Systems
    NFS file system support (CONFIG_NFS_FS)
      Root file system on NFS (CONFIG_ROOT_NFS)

In addition the following option was used to allow tmp to be kept in virtual memory, rather than on an nfs mount.

File Systems
  Virtual memory file system support (CONFIG_TMPFS)

Root File System

On the NFS server a directory was populated with a root file system for the disk-less machine. In the end I made a separate copy of / and a shared copy of /usr and /home. This suited my needs at the time but is by no means the best solution.

As we will be using tmpfs the following symlinks were established: /tmp -> /dev/shm /var/tmp -> /dev/shm

The after this the fstab on the disk-less node was as follows.

proc            /proc           proc    defaults                0 0
tmpfs           /dev/shm        tmpfs   defaults                0 0
/dev/fd0        /floppy         auto    defaults,user,noauto    0 0
/dev/cdrom      /cdrom          iso9660 defaults,ro,user,noauto 0 0
nfs-server:/export/nfs/loretta  / nfs   defaults                0 0
nfs-server:/usr  /usr nfs               defaults,ro             0 0
nfs-server:/home /home nfs              defaults                0 0

As devfs is being used some modification to the system configuration is required. In particular I modified the /etc/inittab lines for virtual terminals to the following:

1:2345:respawn:/sbin/getty 38400 vc/1
2:23:respawn:/sbin/getty 38400 vc/2
3:23:respawn:/sbin/getty 38400 vc/3
4:23:respawn:/sbin/getty 38400 vc/4
5:23:respawn:/sbin/getty 38400 vc/5
6:23:respawn:/sbin/getty 38400 vc/6

DHCP

Setting up DHCP was straight forward, a static entry as follows was inserted into the dhcp.conf entry on the DHCP server.

host loretta {
        filename "/tftpboot/pxelinux.0";
        next-server 192.168.7.14;
        hardware ethernet 00:90:27:3E:EA:60;
        fixed-address 192.168.7.15;
}

Where 192.168.7.14 is the IP address of the TFTP server.

TFTP

Setting up the TFTP server was quite painful as in their infinite wisdom Intel insist on Lan Desk shipping with a broken TFTP server. In short you need to disable MTU discovery for the TFTP server and set the daemon to disable blksize[1]. The former can be achieved by running the following on the TFTP server:

echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc 

Alternatively add the following line to /etc/sysctl.config and then run sysctl -p. This will ensure that this option is set on reboot.

net.ipv4.ip_no_pmtu_disc=1

To disable blksize you need a TFTP daemon that allows you to do this. I used tftp-hpa[2], I believe atftp[3] will allow you to do the same. If your TFTP server runs windows try tftp32[4] The /etc/inetd.conf entry for tftp-hpa with blksize disabled is as follows:

tftp            dgram   udp     wait    nobody    /usr/sbin/tcpd /usr/sbin/in.tftpd -r blksize /tftpboot

pxelinux

Instead of the tftp server supplying a linux Kernel, Pxelinux[5], which is part of Syslinux[6] is used. This is done for two reasons. Firstly, Lan Desk's TFTP client has a limit on the size of file that it is able to download. I am not sure of the size of the limit but it seems to be significantly smaller than a kernel image. Fortunately Pxelinux is small enough for Lan Desk to be able to download. Secondly, using Pxelinux allows us to pass boot-time options to the kernel.

To do this I populated the /tftpboot directory as follows:

vmlinuz-2.4.13.mosix
pxelinux.cfg/default
pxelinux.0

Where:

Pxelinux searches for configuration files by taking the hexadecimal representation of the host's IP address and removing lest significant hex digits until a match is found. If no match is found then default is used. [6]

For example 192.168.0.15 is C0A8000F.

Thus the following configuration files will be requested. The first one found will be used.

pxelinux.cfg/C0A8000F
pxelinux.cfg/C0A8000
pxelinux.cfg/C0A800
pxelinux.cfg/C0A80
pxelinux.cfg/C0A8
pxelinux.cfg/C0A
pxelinux.cfg/C0
pxelinux.cfg/C
pxelinux.cfg/default

This configuration file was used to specify the name of the image, the image to be used, and nfs root information needed. ipappend 1 tells syslinux to set the IP address of the node passed to the kernel to that supplied to Lan Desk by DHCP. The configuration file is as follows:

label linux
  kernel vmlinuz-2.4.13.mosix
  append nfsroot=192.168.7.14:/export/nfs/loretta
  ipappend 1

Conclusion

Getting TFTP up and running proved to be much more painful than it should have been due to bugs and limitations in Lan Desk. Thanks Intel for wasting my time. I'll be sure to return the favour in the future.

In the end it all works quite well. The disk-less node boots up over a 100Mb Ethernet link quite quickly as there are no disk checks. Provided you aren't doing disk-intensive work performance shouldn't be an issue.

References

The majority of these references are written by, or supplied by H. Peter Anvin.

  1. http://syslinux.zytor.com/hardware.php
  2. http://www.kernel.org/pub/software/network/tftp/
  3. http://www.vekoll.vein.hu/~bazsi/aftpd/
  4. http://tftpd32.jounin.net/
  5. http://syslinux.zytor.com/
  6. http://syslinux.zytor.com/pxe.php

More Horms Solutions Projects

All software and trade marks are the property of their respective owners.

| Horms Solutions | Simon Horman's Personal Page |

Copyright © 1995-2015 Simon Horman
Copyright © 2010-2015 Horms Solutions Ltd.
All rights reserved unless otherwise noted