When are diskless compute nodes inappropriate?

Joseph Landman landman at scalableinformatics.com
Tue Jul 15 12:53:45 EDT 2003

When you do lots of disk IO to large blocks, sequential reads/writes. 
Remote disk will bottleneck you either at the network port of the
compute node (~10 MB/s for 100 Base T, or ~80 MB/s for gigabit), or at
the network port(s) of the file server (even if you multihome it, N
clients distributed over M ports all heavily utilizing the file system
will slow down the whole system if the requested bandwidth exceeds what
the server is able to provide out its port(s)).  Or even at the disk of
the server.  

Local IO to a single spindle IDE disk can get you 30(50) MB/s
write(read) performance.  RaidO (using Linux MD device) can get you
60(80) MB/s write(read) performance.  Sure, this is less than a 200 MB/s
fibre channel, but it is also not shared like the 200 MB/s fibre channel
(which becomes effectively (200/M) MB/s fibre channel for M requestors
using lots of bandwidth).

The aggregate IO when you get many writers/readers utilizing lots of
bandwidth is a win for local disk over shared disk.  From a cost
perspective this is far better bang per US$ than shared disk for the
heavy IO applications.  At about $60 for a 40 GB IDE (ATA 100, 7200
RPM), the price isn't significant compared to the cost of an individual
compute node.  That is, unless you go SCSI for compute nodes.

If you go diskless on the OS, just have a local scratch disk space for
your heavy IO jobs.

On Tue, 2003-07-15 at 11:48, Tod Hagan wrote:
> Okay, I'm convinced by the arguments in favor of diskless compute
> nodes, including cost savings applicable elsewhere, reduced power
> consumption, and increased reliability through the elimination of
> moving parts.
> With all the arguments against disks, what are the arguments in favor
> of diskful compute nodes? In particular, what are the situations or
> types of jobs for which a cluster with a high percentage of diskless
> nodes is contraindicated?
> I look forward to learning from the list's collective wisdom.
> Thanks.

Joseph Landman, Ph.D
Scalable Informatics LLC
email: landman at scalableinformatics.com
  web: http://scalableinformatics.com
phone: +1 734 612 4615

Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list