Filesystem question (sort of newbie)

[Donald Becker]

On Fri, 3 Oct 2003, Mark Hahn wrote:

> > 8 or but I haven't tested it), but it boots into a small Ram
> > Disk (about 70 megs depending upon what you need on the

For the Scyld Beowulf system we developed a more sophisticated "diskless
administrative" approach that has better scaling and more predictable
performance.  We cache executable objects, libraries and executables,
using an method that works unchanged with either Ramdisk (==tmpfs) or
local disk cache.

Keep in mind that this is just one element of making a cluster system
scalable and easy to manage.  Using a workstation-oriented distribution
as the software base for compute nodes means generating many different
kinds of configuration files, and dealing with the scheduling impact of
the various daemons.

> alternately, it's almost trivial to PXE boot nodes, mount a simple
> root FS from a server/master, and use the local disk, if any, for 
> swap and/or tmp.  one nice thing about this is that you can do it
> with any distribution you like - mine's RH8, for instance.

The obvious problems are configuration, scaling and update consistency
issues.

> personally, I prefer the nfs-root approach, probably because once
> you boot, you won't be wasting any ram with boot-only files.

They are trivial to get rid of either by explicitly erasing or switching
to a new ramdisk (e.g. our old stage 3) when initialization completes.

> for a cluster of 48 nodes, there seems to be no drawback;
> for a much larger cluster, I expect all the boot-time traffic 
> would be crippling, and you might want to use some kind of 
> multicast to distribute a ramdisk image just once...

Multicast bulk data transfer was a good idea back when we had Ethernet
repeaters.  Today it should only be used for service discovery and
low-rate status updates.

-- 
Donald Becker				becker at scyld.com
Scyld Computing Corporation		http://www.scyld.com
914 Bay Ridge Road, Suite 220		Scyld Beowulf cluster system
Annapolis MD 21403			410-990-9993

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf