Infiniband: cost-effective switchless configurations

Mikhail Kuzminsky kus at free.net
Mon Jul 23 12:44:16 EDT 2012


  It's possible to build 3-nodes switchless Infiniband-connected
cluster w/following topology (I assume one 2-ports Mellanox HCA card
per node):

    node2 -------IB------Central node-----IB-----node1
     !                                             !
     !                                             !
     ----------------------IB-----------------------

It gives complete nodes connectivity and I assume to have
3 separate subnets w/own subnet manager for each. But I think that
in the case if MPI broadcasting must use hardware multicasting,
MPI broadcast will not work from nodes 1,2 (is it right ?).

OK. But may be it's possible also to build the following topology
(I assume 2 x 2-ports Mellanox HCAs per node, and it gives also
complete connectivity of nodes) ? :


  node 2----IB-------- C e n t r a l  n o d e -----IB------node1
       \              /                      \           /
         \          /                         \         /
           \       /                           \      /
             \--node3                         node4--

and I establish also additional IB links (2-1, 2-4, 3-1, 3-4, not
presenetd in the "picture") which gives me complete nodes connectivity.
Sorry, is it possible (I don't think about changes in device drivers)?
If yes, it's good way to build very small
and cost effective IB-based switchless clusters !

BTW, if I will use IPoIB service, is it possible to use netperf
and/or netpipe tools for measurements of TCP/IP performance ?
       
Yours
Mikhail Kuzminsky
Zelinsky Institute of Organic Chemistry
Moscow



_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



More information about the Beowulf mailing list