UDP Cast

UDPcast is a file transfer tool that can send data simultaneously to many destinations on a LAN. This can for instance be used to install entire classrooms of PC's at once. The advantage of UDPcastover using other methods (nfs, ftp, whatever) is that UDPcast uses Ethernet's multicast abilities: it won't take longer to install 15 machines than it would to install just 2.

UDPcast can be started from the included linuxrc boot disk for OS installations, or from the command line when using it for other purposes.

New alpha version of Udpcast for Cygwin (Unix emulation layer for Windows).

New features in 20011231 version

Fixed socklib such as to no longer require the presence of the in_addr_t type (useful for compiling udpcast on older Linux distributions)
Fixed --mcast-addr flag, which was ignored in previous versions
Added new --mcast-all-addr flag, in order to override the default broadcast/multicast address used for finding peers

New features in 20011211 version

Ported it to SuSE 7.3 bootdisk
Half-duplex mode is now default it turns out that performance impact of using full duplex protocol on a network only capable of half duplex is far worse (10%) than in the inverse situation (1%)
Optimized retransmit timer handling (is now adaptative: a sliding average is computed for the observed response time, and if no response is received within that time, plus 10 percent, a second Reqack is sent out)
Hardwired full-duplex slize size to 112 (ideal setting for 100Mbps half-duplex network)
Point-to-point mode is now automatically selected if only one receiver is present.
Bug fix for FEC mode

New features in 20010930 version

Forward error correction:

Transmit redundant information, to enable receivers to locally reconstruct data which has been lost in transit, without requesting it again from the sender.

The sender calculates error correction blocks, which are a kind of checksum of all data blocks in the slice, and sends those to the receivers too. By using those, the receivers can locally reconstruct missing packets without requesting them again from the server

With this, it is even possible to operate in asynchronous mode, with no need to have a communication channel going back from receiver to sender!

Multicast mode:

Rather than broadcasting the data to all machines on the ethernet, whether interested or not, multicasting allows to send data only those machines which are actually interested. Effectiveness of this depends on the Ethernet switch used:

With a hub, or in Tbase2 networks, all machines still get the data, even if multicast can be filtered out at a lower level (i.e. more cost efficiently)
With a switch supporting IGMP, data is only sent out to those ports which have subscribed to it, easying the load on the rest of the network. Note: Some switches may block multicast data altogether (not good), or still forward it to all ports...

Full duplex data scheduling

The sender can now transmit a new slice of data while it is waiting for the receiver's to acknowledge the previous slice. Old versions of udpcast had to wait for the current slice to be acknowledged before transmitting more data.

Point-to-point mode

Useful for further diminishing the load when only sending data to a single destination. This can also obtain the full 100Mbps on mixed speed networks, which would force the lowest speed (10Mbps) on multicast and/or broadcast.

Command line version

Basic usage

udp-sender -f outfile

Starts the sender program. The program waits for connections from the receivers. As soon as a key is pressed on either one of the receivers or on one of the senders, the data transfer will start.

udp-receiver -f outfile

Starts the receiver program. It automatically locates (via broadcast) any sender on the local net, and connects to it.

Advanced options

A complete description of all options is here

The UDP cast boot disks

Making the UDP cast disks

Floppy edition

udpcast comes with two disk images. The first is the boot disk used to boot the kernel and the menu system (linuxrc). The second one will be loaded by the menu system, and contains the actual UDPcast programs. In order to make the disks, just download the images, and cat them to /dev/fd0. Do not uncompress them first.

cat bootdisk >/dev/fd0
change disks
cat udpcast.img.gz >/dev/fd0

You may also store the second UDP cast disk image on an FTP or NFS server, rather than writing it to floppy disk. In that case, the image file must be called udpcast.img.gz (i.e. do not rename it).

CD-Rom edition

Make CD as follows:

cdrecord speed=4 dev=0,0,0 udpcd.img

Substitute speed and device id as appropriate.

If may seem silly to waste an entire CD just to store a small 2.88MB boot image. However, in this day and age, some unscrupulous PC vendors unfortunately assume that nobody uses floppy any longer, and try to save a couple of bucks by building non functional units into their boxen. In such a case, you'll be glad to have a bootable Udpcast CD ready.

A later version of the udp-linuxrc package will support storing the disk image itself on the CD, rather than loading it over the network.

Pre-requisites

If you use UDP-Cast to duplicate OS installations, all participating machines should be rigorously identical hardware-wise. Most notably:

Their hard disks should have the same size and layout (sectors, heads, cylinders). Number of sectors and heads of source system should stay identical, whereas number of cylinders of target system only be greater or identical. N.B. If you are only transferring Linux partitions (no other system), there is no need for matching sectors and heads. Only the overall size of the target system must be equal or bigger than that of the source system.
Any peripherals (network cards, keyboards, mice) should be identical.
If the template machine (sender) has more peripherals (such as for instance a CD-Rom drive) than the client machines, their drivers should be uninstalled before the multicast.
See also the notes on special notes about copying Windows machines

Note about copying to hard disks with different geometry:

If the new disk (or partition) is bigger, the copied system won't actually make any use of the additional space. In order to do that, you need to resize the partition using a partition resizing tool such as Partition Magic or PartEd
When copying a linux system to a partition or disk with differing geometry, be sure to use the linear option in /etc/lilo.conf. If for some reason you can't use this, be sure to rerun lilo on the target system after the copy (boot from a rescue disk).

Using the boot disks

Insert the first floppy disk (or the CD-Rom), and boot your computer from it.
After the system is booted up (a little more than a minute), you get dropped into the menu system
Choose language (german, french or english), screen color (color or monochrome), your keyboard layout
Chose network card module and enter its parameters
Chose network configuration. You may either use automatic configuration via bootp (if a DHCP/Bootp server is running on your network), or you may manually enter the IP address, netmask and router IP.
Chose "Start installation / system"
Chose one of the four modes of operation:

Sender
This machine will act as the sender (source, orginal) of the data. Data will be sent uncompressed
Receiver
This machine will act as a receiver (copy) of the data. Data that is received will be considered as uncompressed data.
Sender (compressed)
This machine will act as the sender (source, orginal) of the data. Data will be sent compressed.
Receiver (compressed)
This machine will act as a receiver (copy) of the data. Data that is received will be considered as compressed data, and the receiver will first uncompress it before writing it to disk
N.B. Compressed operation cuts down on the data to be transferred, but is unfortunately rather computation-intensive:
- Use compressed operation if you have data that is very compressible (i.e. a hard disk that is mostly empty), fast processors (minimum 200 Mhz Pentium) and a (relatively) slow network (i.e. 10Mbps Ethernet).
- Use uncompressed operation if you have data that is not very compressible (a hard disk that is almost full and contains mostly executables), slow processors (slower than 166 Mhz Pentium) or a fast network (100 Mbps Ethernet).
Choose a Port for the UDP Cast (default is 9000).
Chose the device where the image will be copied to/copied from. Default is the entire disk (/dev/hda). You can obtain a speedier copy by only copying the partition you need (Example: /dev/hda1).
N.B. Copying just one partition (rather than the whole disk) may not be enough in the following situations:
- If you installed Lilo in the master boot record (possible inconsistencies between MBR (not copied) and info in the partition (copied))
- If the target machine has never been installed (empty partition table)
Caution: Make sure you chose the same device on senders and receivers.
Enter any additional udp-sender or udp-receiver parameters, if needed (such as, for example --broadcast if you have an Ethernet card not supporting multicast). Basic options such as file, pipe, portbase, etc. need not be entered, these have already been covered by the dialogs before. Yes, this means that in the vast majority of cases, you can leave the "additional parameters" box blank.
If you use the CD edition, at this point you're ready for receiving transmitting/receiving. If you use the floppy edition, the instructions below tell you how to load the second image.
Chose the "installation" source (i.e. where the second image is located): NFS, FTP, or floppy disk. N.B. If the second image is located on Ftp or NFS, it must bear the name udpcast.img.gz. Indeed this name is hardwired into the system, however, you may chose the directory it is located in.
Chose an installation source (NFS, Ftp, or Floppy disk).
Depending on the installation source, enter additional parameters:
- For FTP, enter Server IP, username, password and directory where udpcast.img.gz is located
- For NFS, enter server IP and directory where udpcast.img.gz is located
- No additional parameters are needed if the second image is loaded from floppy disk
The system asks you whether you want to save the current configuration. If you answer yes here, the answers to all the preceding questions are saved to the floppy (make sure that the first floppy is still in the drive). That way, you don't need to answer those questions again when you use this boot disk again. For more details, see below.
Now that all questions are answered, the second disk image is loaded (if chose to you load it from floppy, make sure the second disk is in the drive now)
When the system is ready, it displays UDP Receiver at IP Address
When all participating systems (the sender and the receivers) are ready, press Enter on any one of them to start the transfer.

Making a customized version of the udpcast boot disk

It is possible to set up the UDP boot disk in such a way that it skips all questions, and directly starts the receiver or the sender. In order to do make a customized disk, boot the first disk normally, answer all questions normally, and answer yes to the last question ("Save configuration"). The next time you boot this disk, the system skips all the questions and reuses the answers from this session. This way, you can prepare a floppy disk for each type of machines that you need to install.

N.B. As the action (send or receive) is now "hard coded" on the disk (along with the rest of the configuration), you need to make two different kinds of disks: sender and receivers.

Copying customized udpcast disks

The easy way:

Insert source disk
cat /dev/fd0 >image
Insert target disk
cat image >/dev/fd0

The fast way:

Insert source disk
mcopy a:syslinux.cfg a:info .
Insert an unconfigured udpcast boot disk
mcopy syslinux.cfg info a:

Use the first method if you do not yet have other udpcast boot disk ready. Use the second method if you do already have unconfigured udpcast disks, or differently configured udpcast disks that you wish to update with the new configuration.

Tuning hints

Chose the proper duplex mode: if your network has a Hub, use --half-duplex, if it has a switch, use --full-duplex. If the network is mixed (central switch, hubs around), use --half-duplex
If you have a switch:
- enable "IGMP snooping".
- If hubs are connected to your switch, disable flow control on the ports to which the hubs are connected. Disable flow control is also necessary (on all ports where udp-senders site) if IGMP snooping is not available or not working correctly.
- Disable Broadcast Storm Control (or else the switch might consider the udpcast session as a broadcast storm, and slow it down...)
- If too many retransmissions happen in full-duplex mode, consider lowering the slice size using the --slice-size option (default is 112).

Adding extra modules

If your computers have a network card which is not supported by the SuSE boot disk, you can add it as follows:

Compile the needed modules from source (for a 2.2.18 kernel), or grab the module elsewhere.
Insert the UDP boot disk, and copy its initrd image using the following command (for instance):
```
mcopy a:initrd initdisk.gz
```
Make a backup copy:
```
cp initdisk.gz initdisk.gz.bak
```
Uncompress it:
```
gunzip initdisk.gz
```
Mount it (you need loopback support):
```
mount -o loop -t minix initdisk /mnt
```
Make space on it by removing net modules that you don't need (by default the SuSE disk is flush full...):
```
rm /mnt/modules/3c59x.o
```
Copy the needed network module from your kernel tree:
```
cp /lib/modules/2.2.18/net/3c509.o /mnt/modules
```
Fill up the remaining space with a file full of zeroes, to make sure to clean out all unused blocks. This is needed to make the disk as compressible as possible:
```
cat /dev/zero >/mnt/x
sync
rm /mnt/x
```
Don't worry about the Disk full error after the first command.
Unmount it:
```
umount /mnt
```
Zip it:
```
gzip -9 initdisk
```
N.B. The -9 option is necessary for maximal space efficiency. Remember: the disk is flush full!
Copy initdisk.gz back to the UDP boot disk:
```
mcopy initdisk.gz a:initrd
```
It's here where you notice whether it is too full or not.

Compiling udpcast from source

There are two tar files: udpcast-20011231.tar.bz2 and udp-linuxrc-20011231.tar.bz2. The former is the udpcaster itself, and the second is the modified SuSE linuxrc that we use as our menu system on the first disk.

Compiling udpcast

Untar the file:
```
tar xfIv udpcast-20011231.tar.bz2
```
Cd into the source directory:
```
cd udpcast
```
Compile:
```
make
```
As root, install:
```
make install
```

Compiling udp-linuxrc and making boot disks

Make sure you have loop-back device support enabled in your kernel. (Check whether /proc/devices contains the entry 7 loop)
Untar the file:
```
tar xfzv udp-linuxrc-20011231.tar.bz2
```
Cd into the source directory:
```
cd udp-linuxrc
```
Compile:
```
make
```
Become root, and
```
make install
```
As prompted, insert the two first SuSE CD's (preferably 7.1)
When done, bootdisk.img and udpcast.img.gz can be found in /udpcast. The CD floppy image can be found in udp-big.disk in current directory.
Write bootdisk.img to a floppy using the following command:
```
cat /udpcast/bootdisk.img >/dev/fd0
```
Use makecd script to make bootable UDPCast CD (depending on your hardware config and media, you may need to edit the script)
Alternatively, if you have an extra-density floppy drive (2.88MB), you may use the udp-big.disk image with floppy:
```
cat udp-big.disk >/dev/fd0
```

Alain Knaff

Last modified: Sat Apr 27 10:16:47 CEST 2002