[Beowulf] How Would You Test Infiniband in New Cluster?

Bill Broadley bill at cse.ucdavis.edu
Tue Nov 17 22:18:58 EST 2009

Jon Forrest wrote:
> I had said "I believe these are with IB."
> Now I'm not so sure. I just did a

The performance numbers you showed from relay and mpi_nxnlatbw are definitely
much faster than GigE.  Unless it's multiple copies running on a single
machine (thus printing the hostname).

Assuming that it was actually using the interconnect (not multiple copies
running on a single machine)

>     "ifconfig ib0"

I suspect this is for TCPIP over ib, and doesn't show MPI traffic.  You didn't
mention which controllers do you have?  I suspect that there is a tool to show
the various counters on the HCA, let alone on the switch side.

> I'll have to figure out how to force IB when
> using OpenMPI.

Looks like IB to me, might want to do the reverse to see the real
differential, I find it very handy for cost justifying IB on future clusters
based on real application performance.

Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list