IB vs Myrinet

Patrick Geoffray patrick at myri.com
Tue Nov 4 17:33:57 EST 2003

Hi Joey,

On Mon, 2003-11-03 at 22:19, Joey Sims wrote:
> I believe IB is a much better interconnect technology than Myrinet
> period.  

Free country.

> Plus, you don't have to deal with Myricom.

Would you share the horror story of you dealing with Myricom ? Did
Myricom did something bad to you or your company ? 
> IB is about to find major traction in this industry and Myricom will not
> have the guns to stop it.  As adoption rates increase the price will

Which industry ? That's the real question. If you say HPC, it's bad news
for IB, for several reasons. First, it's a tiny market. The VC don't
find it very appealing, it's not the billions dollar market they were
promised. Storage and data center are worth it, but IB did not succeed
to penetrate these markets. Look at the fate of the last IB company to
close its doors (Fabric Networks if I remember well). The press release
was saying that they were taking their money to go do something else,
because the current market was not worth it.

The second reason is that HPC has very special needs. You can get some
success by having a big pipe, but it's usually not enough. MPI is
important, application performance is important. That's not what the
storage and data center needed, and that's not what IB was designed for.

> decrease quite rapidly.  I've been working with Mellanox and Topspin
> both using Mellanox chips but, their product positioning is different.

There is something very bad in this sentence: "both using Mellanox
chips". Where are the dozens of silicon vendors that were supposed to
flood the market and drive the price down ? They died the last 2 years.
Today, it's not Infiniband, it's Mellanox and resellers. 
Not that different from Myricom and resellers...

> It depends on how you're looking at the cost of IB.  First of all, it's

It really depends on how your are looking at the cost of IB. Mellanox
has been, and still is I believe, burning VC cash, as they don't have
the sales volume to sustain their internal cost. Today's price for IB
products are not sustainable price, they are aggressive penetration
price, that means it's near cost or below cost. That's why so many
players died.

> comparative to Myrinet in "cost per port".  Not too long ago, Myrinet
> was higher in price than IB is today and they haven't came out with
> anything "new" in forever.  Well except a PCI-X version when PCI Express
> is around the corner.  Myricom has a lot of installations worldwide and

The current PCI-X NIC is 1X, the second one to ship by SC03 is 2X, the
next one is 4X. It's not that hard to add links and aggregate bandwidth,
the rest is more important (like being able to do bidirectional faster
than unidirectional...). Why do you want to have a PCI-Express product
when no customers ask for it because PCI-Express is not shipping in
volume yet ? Don't worry, when you can buy PCI-Express nodes in volume,
you will be able to buy a PCI-Express Myrinet NIC to put it in.

> they are highly credible without a doubt but, this industry moves very
> fast and new things are not a new thing.  At 3x the performance of
> Myrinet, "comparative" is still a better value.  IB has many different

New things are not always hardware. We will demo a completely new
software stack at SC. Same hardware, much better application
performance. As I said, adding links to aggregate bandwidth is easy, but
doing the right thing to run applications faster is another level of
difficulties. Now, when you have the right software design, just ramp up
the pipe performance to please the spec believers and you have what
customer wants.

> options such as bridging between IB, GbE, or FC so you could hang your
> storage boxes off the IB switch without much hassle.

You can bridge Myrinet and GigE. 
Not FC, the protocol stinks too much.

> Up to 10GB/sec is fairly fat today.  The roadmap for IB has this
> interconnect technology ratcheted up way higher than 10GB.

Myricom's roadmap goes up way higher than 10 Pb/s, if that makes you
feel more comfortable.


Patrick Geoffray
Myricom, Inc.

Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list