Network Booting

Robert G. Brown rgb at
Thu Jun 19 13:40:47 EDT 2003

On Thu, 19 Jun 2003, torsten wrote:

> Good Morning:
> I've  been struggling  for days  to make  a single  computer be  able to
> network  boot.  I've  read many  documents, and  I'm a  little confused.
> Here  are some  questions, and  I am  very grateful  for assistance,  in
> advance.

Have you read in particular the HOWTOs on  The DHCP and the
Diskless Linux howtos:

are good, and the following aren't quite HOWTOs for PXE but they cross
reference a lot of documents that can be used to help you out.

> 1. I've seen  references to bootp/dhcp,  bootp or dhcp, bootp  and dhcp,
> dhcp over bootp - Which one do I need.  Do I need both bootp and dhcp?

bootp is a very old network protocol supporting diskless booting.  You
send a bootp broadcast/request out on a network, and it returns your
network identity (which it works out based on e.g. your MAC address) and
a tftp (trivial file transfer protocol) address where you can find what
you devoutly hope is the kernel you are supposed to boot.

TFTP is a sucking wound from the point of view of security and hence
must be used with caution -- in the old days it was pretty easy to set
up tftp by accident so that anybody in the netverse could do a tftp
connection to the server and ask nicely for /etc/passwd (see earlier
comment on parallel crack:-) and it would just say uh, duh, sure...

dhcp is a modern improvement (specified by various RFC's as is, for that
matter, bootp itself) on bootp.  In addition to giving the connecting
host an IP number either from a pool or based on its explicit MAC
address, it can send all kinds of other information back to the client.
For example, mask information, nameserver(s), lease time, NIS
information, NTP information, routing information -- enough to totally
configure the interface.  In addition, it can (like bootp) specify both
server addresses and file paths on the servers.

A lot of your conflict likely arises because for a while dhcp couldn't
directly support pxe booting, so a pxe client was developed that could.
We've used it here for a fairly long time and it works fine, and you can
even find details of setting up pxelinux including sample configuration
files on the brahma resources page (resources for beowulf builders).

However, in recent linuces dhpc can directly handle PXE (bootp)
requests.  So all you SHOULD need is tftp and dhcp.  Alas, the TLDP DHPC
HOWTO seems not to have caught on to this yet, although the PXE HOWTO
has gone away -- "interesting"... a TLDP gap.

Still, as Kegel's lovely article makes clear, with just DHPC and TFTP
you can give a system all kinds of things to boot:  grub, pxelinux,
etherboot, a network install image, a diskless boot image, or "whatever
evil program you desire".  The latter means that you need to look out
for people running dhcp servers in your LAN, as in at least the old days
the fastest server (when there is more than one) wins and gets to tell
the booting host what to boot.

[Ah, I remember well the days when SGI's came with bootparamd on by
default and were always the fastest thing on any LAN and were never
configured to support diskless boots of e.g. Suns and in fact were
somewhat broken for that purpose.  You could always tell when a new one
was installed because all the diskless systems stopped booting until you
went over and drilled a stake through the SGI's little daemonic heart.]

Nowadays it's merely a security problem.  Use whatever means necessary
to ensure that no bootp/dhcp servers exist within the broadcast boundary
of your LAN (up to the router/switch that filters broadcasts, if you've
got one).  Try not to kill anyone.  Permanently that is.  Shooting them
in the kneecaps is generally held to be ok.

> 2. My  Lab is  on  the  University subnet.   Will  my bootp/dhcp  server
> conflict with the  university system?  Do I need to  isolate my lab with
> IP masq first?

See previous comments.  I personally am fond of my kneecaps and imagine
that you kinda like your own. Therefore, I would be very certain that
your LAN is isolated from the University sytem if they indeed run a WAN
dhpc server before setting up one of your own.  Even if they aren't
running a dhcp server on the WAN, you'll still want this isolation.  Do
YOU want to be giving out IP numbers and so forth to all the booting
systems on campus?  Of course not.  You might do a really good job and
find yourself saddled with it permamently.  I think I'd rather get shot
in the kneecaps than that.

> 3. On which broadcast/subnet does PXE send bootp requests.  Does it make
> a difference, with PXE, whether I use bootp or DHCP?

The broadcasts are ETHERNET broadcasts.  That is, they aren't IP
broadcasts at all as IP hasn't yet been configured on the host
containing the device issuing the request (if that makes sense).  The
NIC shouts at the top of its metaphorical lungs IS THERE ANYBODY OUT
Server says "why certainly, you'd be, your netmask is, you can route through, your server is, and if you ask it nicely it will send you an image at
/tftpboot/i386/kernel_RH9 so you can boot..."

The security problem is obvious, as if my slightly faster server answers
first, I can tell it to boot /tftpboot/i386/kernel_RH9_evil from my very
own system, the one that behaves just like RH9's kernel except for the
backdoor on port 47319 that I can get into if I enter the right 1337

Other reasons for limiting the range of the broadcast are similarly
obvious.  An ethernet broadcast is moderately expensive as it goes to
every port on a switch and requires active processing by every host when
it gets there.  It actually eats a bit of CPU and net bandwidth from
every host and branch of the LAN.  This is why network segmentation was
invented, in a way -- some old networks were notorious for generating
truly incredible amounts of broadcast traffic, kind of like a
perpetually running DoS attack that was perfectly "normal".

It is pretty easy these days to create a boundary to a broadcast domain.
Lots of (managed) switches permit you to turn off broadcast forwarding.
Of course, a router or dual headed linux box used as a router accomplish
the same thing even better.

> 4. What are people using to setup/install partitions?  I've seen bpbatch
> and ka-tools.  I'm interested in GPL software, primarily.

I'm not sure what you mean here.  Are you talking about e.g. fdisk/mkfs?
Higher order front ends of same?  Or something for setting up diskless
systems specifically?

We actually mostly use PXE/DHCP to fuel kickstart, which creates
partitions and installs them flawlessly and even moderately
intelligently.  There are lots of ways to accomplish the same thing,
though, including scripts that use fdisk and mkfs.  Most of them are
probably GPL -- I didn't realize that anybody was selling proprietary
linux tools in this space (there's an oxymoron for you -- "proprietary
linux tools";-).

> 5. Are there any known problems with Realtek PXE ROMS?  My network cards
> are system integrated Realtek's.

I will try to stifle any response, since I honestly don't know how their
PXE cards behave.  However, overall I loathe RTL NICs.  I wouldn't be
surprised if they behave badly.  Then again, in the fullness of time
even RealTek may get the humble NIC right...

> 6. Should I just download and setup the MOSIX stuff?

Depends on what you want to do.  Generally speaking I'd answer "no" with
a puzzled expression on my face, although for certain task mixes it
isn't horribly unreasonable.  One major problem is that it lags the
major distributions pretty significantly, which can produce both
security and user satisfaction gaps.  However, it does work nicely as a
user-transparent, moderately "expensive" job migration facility, when it

> 7. Does the  Linux Terminal Server  Project (LTSP) have  anything useful
> that I could/should be using?

See answer to 6.  Conceivably, but it depends on what you're trying to
do.  So what are you trying to do?

(Hope some of the above helps:-)


Robert G. Brown	             
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list