IB vs Myrinet

Jeff Layton jeffrey.b.layton at lmco.com
Tue Nov 4 09:53:55 EST 2003

Nicholas Henke wrote:

> On Mon, 2003-11-03 at 22:50, Dean Johnson wrote:
> > On Mon, 2003-11-03 at 21:19, Joey Sims wrote:
> > > I believe IB is a much better interconnect technology than Myrinet
> > > period.  Plus, you don't have to deal with Myricom.
> > >
> >
> > Hmmm, all my dealings with Myricom have been excellent. We had a frame
> > failure right before a holiday and they happily cross-shipped a
> > replacement. We were back in business very quickly. All our 
> questions of
> > support have been answered quickly and accurately.
> Same here -- the support has been great. We did run into a problem where
> they were short on line cards to send as replacements, but they did keep
> us posted and let us know what was going on.
> BTW -- did no one notice that the statement was 'comparable prices per
> port' but that you had to _pay_ for the mpi implementation? I feel alot
> better knowing there is a free and open source implementation of MPI
> over gm from LAM and mpich.

   I want to interject a comment here. In the past (recent and a few
years back) we've had trouble with the open source MPI implementations
with our codes. When we contacted them about our problem we got
a luke warm (at best) response. When we contacted a commercial
MPI vendor, they fixed the problem in less than a day. Plus our
codes were about 30% faster than the open-source ones. However,
we continue to look at LAM, MPICH, and others.
   While I'm a big proponent of open-source for many reasons, at
least for MPI, we've found that a commercial vendor is worthwhile
for us. The one we've used provides a very good and fast product for
our systems. Also, their technical support is extremely good (I normally
reserve that phrase, but it truly applies in this case). More importantly,
we've found that most of our problems beyond the first few months
that a cluster is in production, are with MPI. Having a company to
help us diagnose and fix the problem quickly means a great deal to
us (we're in production 24/7 and down time is a true killer).
   So for us, when we look at per port costs, we include a commercial
MPI for whatever network we're looking at, well with one exception.
While there are differences in MPI costs based on the type of
interconnect, the difference is in the noise for price/performance for
   One of the hidden costs from my prospective, that allows us to
compare interconnects, is a product of the cost of diagnosing problems,
fixing problems, and how frequently the problems occur. We have
experience with one high-speed interconnect in this regard and that
number is very large. This has made us gun-shy about trying any
other high-speed interconnect on a production basis (although we
continue to test).
   Just my 2 cents this morning.



Dr. Jeff Layton
Aerodynamics and CFD
Lockheed-Martin Aeronautical Company - Marietta

Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list