[Beowulf] Should I go for diskless or not?

Ashley Pittman ashley at pittman.co.uk
Fri May 15 13:12:28 EDT 2009

On Fri, 2009-05-15 at 06:43 -0400, Lawrence Stewart wrote:
> I'll echo the remarks about swapping, there is a large patch set for 
> swapping over IP, and we don't run that.  In fact right now we run 
> without swap space, and vm_overcommit_ratio set to "90".  This is 
> generous enough that we're not having problems, even on large
> systems, 
> with running out of memory.  Everyone seems to agree that having some 
> swap space is good for stability, so we do plan to add swap at some 
> point.  We've got a new network block device that can swap over the 
> interconnect (without any allocations) at about 2 GB/s which is
> probably 
> good enough to make DSM interesting.  If you have local disks, using 
> them for swap will work fine.

Another problem which nobodys mentioned yet is where are you going to
swap too? Sure each node might have 2GB/s network bandwidth to play with
but no frontend is going to cope with more than a handful of nodes
swapping at once.  It might be viable for a network of diskless
workstations but for a cluster forget it.

The only way that network swapping can make sense in a cluster is if you
know the application doesn't fit in memory and can allocate some extra
nodes to host the swapped memory, preferably swapping over the network
to RAM on a remote machine.  This doubles the nodes required to run your
job however and makes scheduling it with normal jobs impossible.

Ashley Pittman,

Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

More information about the Beowulf mailing list