Linux Diskless HOWTO

Dovie Adler, dadler@hobbes.jct.ac.il

Version 0.0.30 December, 13 1998

This document is a step by step guide to setting up and configuring linux in a diskless environment. It covers setting up the boot server running linux and the linux diskless clients.

Contents

1 Preamble

1.1 Copyright

1.2 Disclaimer

1.3 Acknowledgements

2 Introduction

2.1 Who should read this document?

2.2 What are we trying to do here?

2.3 New Versions of this Document

2.4 Feedback

2.5 General Overview

3 Installing the server

3.1 Needed software

3.2 Configuring the server

3.2.1 Configuring the DHCP server

3.2.2 Configuring the bootp server

3.2.3 Setting up the tftp server

3.3 Exporting file systems

4 Setting up the clients

4.1 Clients general overview

4.2 Creating the client's root directory

4.2.1 Creating the directory tree

4.2.2 The /etc directory

4.2.3 The /var directory

4.2.4 The /dev directory

4.3 Configuring X

4.4 Configuring the DHCP client (DHCPcd)

4.5 building the bootrom

4.6 Adding more diskless clients

4.7 Things to look out for

5 Software

5.1 Server Software

5.2 Client Software

5.3 Useful scripts

6 Appendix B - Other sources of information

1.1 Copyright

(c) Copyright 1998 Dovie Adler. Permission to distribute freely is granted as long as this copyright message is attached intact. Trademarks are owned by their owners. There is no warranty about the usefulness of the information in this document. Use and distribute at your own risk.

1.2 Disclaimer

No liability for the contents of this document can be accepted. Use the concepts, examples and other content at your own risk. Additionally this is still an early version of this document filled with possible admissions and errors. Although almost every effort was made to ensure that everything here works as it should.

1.3 Acknowledgements

The writing of this HOWTO was a long process and it is thanks to many individuals that it is finally close to completion. I would like to thank the following people for their contributions and help.

* All the people who are responsible for the Linux operating system, and all the other free software packages that are out there.

*

2.1 Who should read this document?

2.2 What are we trying to do?

The fundamental idea is to use older machines that are lying around holding doors open, as diskless workstations much the same way that system administrators use Sun(tm) workstations to boot diskless off of a SPARC(tm) server, and have them run as X servers. Since we are using different machines with different hardware configurations the decision was made to have them all mount everything from a NFS server. If some of the machines happen to have working hard disk they will still boot from the net and use the local disk for swap and /tmp. Since I did not have a ROM burner handy at the time I started using the boot code from a floppy disk. We simply pushed the 3.5" drive back in to the drive bay and closed it off from the front. This has worked out so well that we have so far not moved at all to real boot ROMS. A second plus to keeping the boot code on floppies is that I did not have to write boot ROM code for all the different types of NIC cards that I have. In the end we wanted to have machines that boot diskless mount there root directories through NFS from the boot server, get all there account information from NIS, and all their mail from the mail server.

For the examples in this HOWTO, the following should hold true through out. The boot server is called nfs_srv and its IP address is 157.151.7.30. Its running the DHCP deamon. Home directories are mounted from a NFS server called hd_srv and its IP address is 157.151.7.20. The mail server is mail_srv and its IP address is 157.151.7.15. The diskless clients are called client01 and up and have IP addresses starting from 157.151.7.220.

2.3 New versions of this document

New versions of this document are available at the normal ftp sites including

ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO

Finally the latest version of this document should also be available in various formats at:

http://hobbes.jct.ac.il/

2.4 Feedback

All comments, error reports, omissions, additional information and criticism of any sort can be sent to:

dadler@hobbes.jct.ac.il

3 Installing the Server

I will not go into details of installing the linux server here. Slackware and Red Hat both come with more then enough information on this subject. I will just recommend installing the full Red Hat or the Slackware distribution to save time now and later on, when you will have to look for packages that you need but did not install.

We are also assuming that you have networking configured correctly, and that you can ping, telnet, and export file systems with NFS from your machine to the rest of your network. If this is not the case I recommend reading the NET-3-HOWTO, NFS-HOWTO first, and then come back here.

3.1 Needed software

It goes with out saying that you will need a machine running linux to use as a boot server, and three additional packages

1. A dhcp or bootp server

the dhcp package is available at:

Primary-site: ftp://ftp.kobe-u.ac.jp/pub/PC-UNIX/Linux/network/dhcp

32kB dhcpcd-0.65.tar.gz

Alternate-site: ftp://sunsite.unc.edu/pub/Linux/system/network/daemons

32kB dhcpcd-0.65.tar.gz

or at:

http://www.isc.org/dhcp.html

2. The netboot package

the netboot package is available at:

http://www.han.de/~gero/netboot

and as a RPM package from:

http://www.beowulf.org/software/RPMS/netboot-07.2-1.i386.rpm

3. The dhcp client

4. The crynwr packet driver collection available at http://www.crynwr.com

Which to use, bootp or dhcp? The choice is yours. I went with dhcp because it lets be supply information to my win95 machines from the same server. There are two versions of the dhcpd software. One is version 2 which is currently in beta testing but is reported to be fairly stable. The second version is 1.0, which has had its feature freeze in November 1996 and is recommended for running in a production environment. The only reason to use version 2 is that it supports multiple network interfaces, something that version 1.0 does not support.

3.2 Configuring the server

The kernel should have MULTICAST enabled. It goes with out saying that you need to have TCP/IP enabled also. If you are planning on having more then one NIC in your server and you will be using DHCP, than you will have to have a version of the linux kernel higher then 2.0.31. This is because earlier versions did not have support for SO_BINDTODEVICE. In order to take advantage of this you also have to have the 2.0.31 system headers installed before you compile DHCPd.

ifconfig -a should return you something like this:

eth0 Link encap:10Mbps Ethernet HWaddr 00:00:D0:C1:C4:12

inet addr:183.217.19.43 Bcast:157.151.5.251 Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU: 1500 Metric:1

RX packets: 2875542 errors: 0 dropped: 0 overruns: 0

TX packets: 218647 errors: 0 dropped: 0 overruns: 0

Interrupt: 11 Base address: 0x210

If there is no MULTICAST on the line with UP BROADCAST you should reconfigure your kernel to

add multicast support (and reboot, of course). You don't need to do anything else to enable multicasting.

3.2.1 Configuring the DHCP server

Configuring dhcp is usually done through its configuration file, usually /etc/dhcpd.conf

the format of the files

# example dhcpd.conf file
#========================================================================
default-lease-time 600;
max-lease-time 7200;
option subnet-mask 255.255.255.0;
option broadcast-address 157.151.7.0;
option routers 157.151.7.10;
option domain-name-servers 157.151.1.4, 157.151.1.15;
option domain-name "test.edu";

host client01 {
  hardware ethernet 00:00:D0:C4:24:D6;
  fixed-address 157.151.7.220;
  filename "/tftpboot/client01";
} 
host client02 {
  hardware ethernet 00:00:C0:5B:24:D6;
  filename "/tftpboot/cl";
  option root-path "/export/root/client02";
  option nis-domain "test";
  option nis-servers 157.151.1.4;
}

# end of dhcpd.conf

To startup DHCP

touch /etc/dhcpd.leases

add this line to one of your rc scripts I added this to the end of rc.M

if [ -f /usr/sbin/dhcpd ] then

/usr/sbin/dhcpd ; echo "Starting up the DHCP server"

fi

/usr/sbin/dhcpd

You only have to touch the dhcpd.leases file once. If the file exists every time dhcpd starts it will over write the existing file.

3.2.2 Configuring the bootp server

Bootp is configured through its configuration file usually /etc/bootptab:

You must include the IP address of your gateway, DNS server and the ethernet addresses and IP address of your diskless workstations. An example /etc/bootptab file looks like this:

# example /etc/bootpdtab file
# the global section
global:\
:sm= 255.255.255.0:\
	:ds=157.171.1.4:\
	:gw=157.171.5.10:\
	:ht=ethernet:\
	:hd=/tftpboot:\

# hosts
client01:tc=global:ha=00:00:C0:12:B1:01:ip=157.151.7.200:bf=client01:
client02:tc=global:ha=00:00:C0:C1:B1:11:ip=157.151.7.201:bf=client02:

To start up the bootpd server from the command prompt type

bootpd -s

or from inetd through the inetd configuration file usually at /etc/inetd.conf. Uncomment the following line in inetd.conf that looks like this:

bootps dgram udp wait root /usr/sbin/tcpd bootpd -d 4

The line might also look like this:

bootps dgram udp wait root /usr/sbin/bootpd bootpd -I

3.2.3 Setting up the tftp server

The Trivial File Transfer protocol or tftp, is used to give the clients their kernels. The reason why its used over regular ftp is that its easier to code and does not require a password.

By default tftp uses the /tftpboot directory. You can specify on the command line any other directory that you like. Just note that its really not advisable to export your whole directory tree to the world through tftp.

In the /etc/inetd.conf you have to uncomment (remove the hash mark) the line:

# tftp dgram udp wait nobody /usr/sbin/in.tftpd tftpd

Insert or uncomment the following two lines in /etc/services:

bootps 67/tcp # BOOTP server

tftp 69/udp # TFTP server

Then restart inetd by typing

kill -HUP <INETD_PROCESS_ID>

4 Setting up the clients

Because on a disk-less client all system files are actually on the server, this is where the entire client configuration will be done.

4.1 General Overview

In order for the client workstation to boot there are several problems that it must over come

* It must find the boot server

* It must find out its own IP address

* It then needs to get it kernel

* then it has to find it's NFS server and the mount point to its root file system

*

Compiling the kernel

First you will need a kernel to boot the clients from. The kernel should be compiled with support for the following options. These can't be selected as modules since they are needed before the file systems are mounted.

* your network interface card (NIC),

* TCP/IP networking

* NFSROOT

* the NFS filesystem

* RARP

You do not need to compile in support for the ext2 file system, this can be loaded later on through a module.

To start configuring:

make menuconfig

from the /usr/src/linux directory. After you have configured all the options that you need you can start compiling the kernel. Issue the following comands:

make dep ; make clean ; make zImage

To create the kernel for each client make sure that your linux kernel in in the current directory and then run mknbi-linux:

mknbi -linux -a nfsroot=<SERVER_IP>:/export/root/<MACHINE_NAME> -i rom  -k zImage -o <MACHINE_NAME>

so that if your machine name is client01 and it's root directory is /export/root/client01 on boot NFS server nfs_srv whose IP address is 151.157.1.1, you would type:

mknbi -linux -a nfsroot=151.157.1.1:/export/root/client01 -I rom -k zImage -o client01

the -a option is the parameters that you are passing to the linux kernel on boot. In this case it's the machine IP address and the directory to mount as the root filesystem.

You can then copy the file <MACHINE_NAME> to the /tftpboot directory and change its mode to 755

from here you can go onto section 4.5 to make the boot rom.

As a side note here, If you are not using the Netboot package and want the kernel on the floppy disk you can use this little trick from the NFS-Root Mini-HOWTO

mknod /dev/nfsroot b 0 255
cd /usr/src/linux/arch/i386/boot
rdev zImage /dev/nfsroot

and to copy the image onto the floppy disk.

dd if=zImage of=/dev/fd0

4.2 Creating the clients root directory

In creating the clients root directory structure there are two general types of files. One are files that are unique to each client, and as such you will have to have a separate copy for each one. And the second type of files are ones that are the same for everyone. Files that are the same for every one can of course be linked from the servers copy.

In the clients root directory we will need the following directories:

/bin, /dev, /etc, /lib /proc, /sbin, /tmp, /var

since there are no client spicific file in /usr, I just mounted the whole thing for each client. You might need to also create the mount point for your home directories.

4.2.2 The /etc directory

Since /etc is where all the system specific configuration files should be, all of the static unique files will (should) be here. Here is a list of files that will be different for each machine:

/etc/hosts

/etc/inittab

/etc/rc.d/*

/etc/fstab

/etc/passwd

/etc/groups

/etc/XF86config

The /etc/fstab file should mount the clients root directory first

then swap, /usr and home directories. An example /etc/fstab file is show here:

# fstab file for client1
nfs_srv:/export/root/client01	/	nfs	defaults	1	1
home_dirs:/home  	       /home	nfs	defaults	1	1

4.2.3 The /var directory

The var directory should be separate for each client. If you want to try to share var amoung the different workstations and the server at least the following directories should be seperate for each station

* /var/run

* /var/spool

* /var/lock

* /var/adm

* /var/preserve

/var/preserve and /var/tmp have to be writable by everyone and have the sticky bit set.

chmod 777 /var/preserve
chmod 777 /var/tmp
chmod  g+s /var/preserve
chmod g+s /var/tmp

4.2.4 The /dev directory

Although the /dev directory can be shared among the different clients, its better to create a separate one for each client. While its possible to create each client;s /dev/directory with the appropriate MAKEDEV script. In most cases its simpler just to copy it from the server:

cp -a /dev/ /export/root/client01/dev

Keep in mind that /dev/mouse and /dev/cdrom are symbolic links and should be set to point to the proper actual device file.

4.3 Configuring X

On a standalone Slackware system, using Xfree /usr/X11R6/lib/X11 is a link to /var/X11R6/lib. This is because X writes to /var/X11R6/lib/xdm and /var/X11R6/lib/xkb, every time that it starts. I wanted everything in /usr to be static and the same for all machines. So first I removed the link to /var/X11R6/lib and copied all the files and directories from /var/X11R6/lib/X11 to really be in /usr/X11R6/lib/X11. Then I created two links from /usr/X11R6/lib/X11/xdm to be linked to /var/X11R6/xdm and /usr/X11R6/lib/X11/xkb/compiled to be a link to /var/X11R6/compiled. To use the proper VGA card driver, /usr/X11R6/bin/X should be a link to /var/X11R6/XF86_[PROPER_CARD_DRIVER]. For example in my setup most of the computers use the MACh64 graphics card so /usr/X11R6/bin/X is a link to /var/X11R6/XF86_MACH64.

If you are going to be using these machines as X servers and want them to use xdm from a X client then add to the end of your rc.M file following line:

4.4 Configuring DHCPcd

Configuring the DHCP client under Red Hat 5.0 and up, is a snap. All you have to do is start the control panel by typing control-panel then select "Network Configuration" under the eth0 interface define the DHCP protocol. leave the other paramets blank.

If you are not using Red Hat you will need to get the DHCP client demon by your self.

4.5 Building the bootrom

The first thing to do is to set up the network interface card (NIC). This usually involves playing with jumpers or using some type of setup program that most often has to be run from DOS. For more information on how to do this curectly please consult with the manual that came with your network interface card. After configuring your interface card it's a good idea to write down someplace safe all the necessary hardware configurations such as the interrupt number memory address and so on. It's also a good idea to keep the configuration consistent among all your machines. This saves lots of guesswork later in life.

If you have installed the full netboot package then you can just type:

makerom

From any directory. If you have not installed the complete netboot package then type:

make bootrom

From the netboot directory.

Makerom will then ask you which bootrom kernel you want to build (don't get this confused with the Linux kernel). Unless you have any special requirements, you should use the standard kernel. You then have to specify which network driver you want to use. You can pick one from the given list of supply one of your own. If you are providing your own you have to give the full path name to where the driver file resides on your system and all the necessary options to run it.

Note that the netboot comes with lots of options that I did not list here. Please read the README file that comes with the netboot package for more information.

When the makerom configuration script has finished you will find two new files in the current directory:

image.flo

this file can be written onto a floppy using dd

and one of the following:

image.rom

image.hex

image.flash

You can now copy the image.flo file onto a floppy using:

dd if=image.flo of=/dev/fd0

4.6 Adding more diskless clients

To add another diskless client you need to add the appropriate entry in the /etc/dhcpd.conf or the /etc/bootbtab file. To do that you needs its hardware address. You also need to add entries for the machine in you DNS tables and the /etc/exportfs file. After that you can copy one of your exsisting clients for the new one for example

cp -a /export/root/client01 /export/client/root/client20

you then have to change the following files:

/etc/rc.d/rc.inet1 the proper IP address

/etc/rc.d/rc.local the proper hostname

/etc/hostname

4.7 Things to look out for

When booting up the diskless machines the netboot code would successfully use the network interface card (NIC). However, the kernel would not find the card. The solution to this was not to use the soft config option on the NIC but one of its jumpered settings.

Using DHCPD with two NIC cards caused some problems, according to the documentation, the newest version should be fixed by now.

If you are running a Linux 2.1 kernel, this does not guarantee that you have SO_BINDTODEVICE. Linux kernel version 2.0.31 was released quite a while after 2.1kernel development began. The earliest Linux kernel in the 2.1 development stream with SO_BINDTODEVICE is version 2.1.68.

5.3 Some useful Scripts

Slackware seems to lack the exportfs command. So here is a little script that accomplishes the same thing:

#!/bin/sh

# FILE: exportfs

#--------------------------------------------------------------

killall -HUP /usr/sbin/rpc.mountd

killall -HUP /usr/sbin/rpc.nfsd

echo file systems re-exported

# end of script

6 Appendix B - Other sources of information

* /usr/src/linux/Documentation/nfsroot

* NFS Root Client HOWTO

* NFS-Root HOWTO

* NFS HOWTO

* DNS HOWTO

* NIS HOWTO

* DHCPd mini-HOWTO

* Linux Security HOWTO

*