[Beowulf] Re: TOE on Linux

Greg Lindahl lindahl at pbm.com
Tue May 20 14:46:12 EDT 2008


Joe Landman wrote:

> Contrary to the detractors of the technologies comments, the
> TOE/RDMA card *did* provide fairly significant performance delta for
> real apps running MPI over gigabit ethernet.

As a detractor of TOEs, I should point out that one data point does
not prove that it's common that apps get a benefit.

I'd be willing to bet that this app was doing extremely large
transfers, and maybe even managed to get more concurrency with the
TOE... which could easily be a flaw in the MPI implementation's TCP
driver, a pretty common thing to be wrong. For example, LAM was always
much better than MPICH over TCP, and I wouldn't be surprised if
OpenMPI continues this superiority over MPICH-2.

The most interesting thing, to me, is that the various people selling
TOEs in the HPC arena publish almost no benchmarks. What's the message
rate and N1/2? The only N1/2 I've ever seen published was 100 kbytes.

(Obviously I'm not including Myricom in this bucket: they do publish
microbenchmarks.)

-- greg



_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



More information about the Beowulf mailing list