Intel PRO/1000CT Gigabit ethernet with CSA

Daniel Pfenniger daniel.pfenniger at
Fri Jun 27 13:46:28 EDT 2003

Mikhail Kuzminsky wrote:
> According to Daniel Pfenniger
>>For a small experimental cluster (24 dual Xeon nodes)
>>we decided to use InfiniBand technology, which from specs is
>>4 times faster (8Gb/s), 1.5 lower latency (~5musec) than
>>Myrinet for approximately the same cost/port.
>   Could you pls compare them a bit more detailed ?
> Infiniband card costs (as I heard) about $1000-, (HCA-Net from
> FabricNetworks, former InfiniSwitch ?), what is close to Myrinet.
> But what is about switches (I heard about high prices) ?
> In particular, I'm interesting in very small switches;
> FabricNetworks produce 8-port 800-series switch, but I don't
> know about prices. May be there is 6 or 4 port switches ?

I don't know about such small switches.  FabricNetworks list
prices are ~10K$ for a 12 port switch, ~15K$ for a 24 port
switch.  But these 2 switches are not fully fat.
A fully fat tree 32 port switch list price is estimated by me
around 30-40K$, but it is not official because very recent
(such a switch has been shown at the Heidelberg HPC conference
these days).

The exact costs are presently not well fixed because several companies
enter the market.  The nice thing about IB is that it is an open
standard, the components from different companies are compatible,
which is good for pressing costs down.

> BTW, is it possible to connect pair of nodes by means of
> "cross-over" cable (as in Ethernet), i.e. w/o switch ?

Yes. The same copper cable as for the switch should do
(but I have not tested yet).
Each HCA has *two* ports, so you should be able to build a fully
connected no hop 3-node cluster without a switch.
With two HCA per nodes you might raise this number to 5 and
still have a reasonable cost/node with respect to the switched
solution.   The 3m cables cost something around 300$ though,
not completely negligible.

	Daniel Pfenniger

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list