[Beowulf] Q: IB message rate & large core counts (per node) ?

Mark Hahn hahn at mcmaster.ca
Fri Feb 26 00:10:41 EST 2010


>> Regardless of how tight the seastar per-hop latency is,
>> IB has 2.5x the per-hop fanout (2or 3 outgoing 9.6 GB links
>> versus 18 outgoing 4 GB/s links). higher radix means an
>> advantage that increases with size.
>
> Doesn't this assume worst case all-to-all type communication
> patterns.

I'm assuming random point-to-point communication, actually.

> If you are just trading ghost cell data with your neighbors
> and you have placed your job smartly on the torus the fan out
> advantage mentioned is irrelevant. No?

if your comms are nearest-neighbor, 
then yes, a nearest-neighbor fabric is your friend ;)

how often does that actually happen?  to work out so neatly would 
preclude, for instance, adaptive meshes, right?  it seems like mostly
I see jobs with no obvious regular structure to their communication.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



More information about the Beowulf mailing list