Infiniband: cost-effective switchless configurations

Mikhail Kuzminsky kus at
Mon Jul 23 12:44:16 EDT 2012

  It's possible to build 3-nodes switchless Infiniband-connected
cluster w/following topology (I assume one 2-ports Mellanox HCA card
per node):

    node2 -------IB------Central node-----IB-----node1
     !                                             !
     !                                             !

It gives complete nodes connectivity and I assume to have
3 separate subnets w/own subnet manager for each. But I think that
in the case if MPI broadcasting must use hardware multicasting,
MPI broadcast will not work from nodes 1,2 (is it right ?).

OK. But may be it's possible also to build the following topology
(I assume 2 x 2-ports Mellanox HCAs per node, and it gives also
complete connectivity of nodes) ? :

  node 2----IB-------- C e n t r a l  n o d e -----IB------node1
       \              /                      \           /
         \          /                         \         /
           \       /                           \      /
             \--node3                         node4--

and I establish also additional IB links (2-1, 2-4, 3-1, 3-4, not
presenetd in the "picture") which gives me complete nodes connectivity.
Sorry, is it possible (I don't think about changes in device drivers)?
If yes, it's good way to build very small
and cost effective IB-based switchless clusters !

BTW, if I will use IPoIB service, is it possible to use netperf
and/or netpipe tools for measurements of TCP/IP performance ?
Mikhail Kuzminsky
Zelinsky Institute of Organic Chemistry

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list