[Beowulf] Re: torus versus (fat) tree topologies

Rick Friedman rick at ammasso.com
Mon Nov 15 15:32:15 EST 2004

> simply because the are not enough users of it.  As a consequence, I am
> looking to find the "best" interconnect solution which will allow a
> few people use of most or all of the CPUs for the jobs we run.

	there's always a danger of over-benchmarking, but you should
probably see if you can get access to an IB cluster.  for CFM, I'm a
little surprised you appear to care so much about latency, since I'd
expect your workload to have the usual volume/surface-area scaling, and
thus doing a lot of work in a single node, and needing only moderate
bursts of bandwidth for nontrivial problem sizes.
	from looking at list prices on the web, Myrinet, IB and
Dolphinics have similar per-port prices which are noticably lower than
Quadrics but also dramatically higher than gigabit.  I suspect most
people would agree that Quadics is a latency specialist, at least for
not purely nearest-neighbor applications.  OTOH, for cheap nodes, you
should probably consider whether spending 50% of the node price makes
sense for the performance boost.  (I see 242-based servers starting at
around $2k list, and your total gigabit cost would be less than

Just to add to this list of options, there are currently a number of
companies working on a newer Ethernet technology called RDMA over
TCP/IP. Adapter cards that support this technology delivers lower
latency and higher CPU utilization than standard GigE, but runs on
standard Ethernet infrastructure (switches, cables, etc). 

I admit to working for one of those companies (Ammasso), but wanted to
let you know about other alternatives.



Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list