Filesystem question (sort of newbie)
becker at scyld.com
Mon Oct 6 13:29:09 EDT 2003
On Fri, 3 Oct 2003, Mark Hahn wrote:
> > 8 or but I haven't tested it), but it boots into a small Ram
> > Disk (about 70 megs depending upon what you need on the
For the Scyld Beowulf system we developed a more sophisticated "diskless
administrative" approach that has better scaling and more predictable
performance. We cache executable objects, libraries and executables,
using an method that works unchanged with either Ramdisk (==tmpfs) or
local disk cache.
Keep in mind that this is just one element of making a cluster system
scalable and easy to manage. Using a workstation-oriented distribution
as the software base for compute nodes means generating many different
kinds of configuration files, and dealing with the scheduling impact of
the various daemons.
> alternately, it's almost trivial to PXE boot nodes, mount a simple
> root FS from a server/master, and use the local disk, if any, for
> swap and/or tmp. one nice thing about this is that you can do it
> with any distribution you like - mine's RH8, for instance.
The obvious problems are configuration, scaling and update consistency
> personally, I prefer the nfs-root approach, probably because once
> you boot, you won't be wasting any ram with boot-only files.
They are trivial to get rid of either by explicitly erasing or switching
to a new ramdisk (e.g. our old stage 3) when initialization completes.
> for a cluster of 48 nodes, there seems to be no drawback;
> for a much larger cluster, I expect all the boot-time traffic
> would be crippling, and you might want to use some kind of
> multicast to distribute a ramdisk image just once...
Multicast bulk data transfer was a good idea back when we had Ethernet
repeaters. Today it should only be used for service discovery and
low-rate status updates.
Donald Becker becker at scyld.com
Scyld Computing Corporation http://www.scyld.com
914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system
Annapolis MD 21403 410-990-9993
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf