IB vs Myrinet
tony at mpi-softtech.com
Wed Nov 5 13:22:04 EST 2003
Roger, FYI, we have had only trivial issues supporting Opteron-based IB, for
instance, things worked in a matter of hours with Racksaver-based Opteron
dual 1U boxes with
Mellanox cards and VAPI drivers. This is with a SuSE/UNITED linux type
that Racksaver generally ships (nothing specialized).
In polling mode, we had about 4.5us latency from the Mellanox A1 cards with
out of the box from MPI/Pro (above 4-5 months ago).
After that, we ran ChaMPIon/Pro on some larger cluster configs with dual
all very straightforward.
----- Original Message -----
From: "Roger L. Smith" <roger at ERC.MsState.Edu>
To: "Bill Broadley" <bill at math.ucdavis.edu>
Cc: "Joey Sims" <jsims at csiopen.com>; <beowulf at beowulf.org>
Sent: Tuesday, November 04, 2003 8:34 AM
Subject: Re: IB vs Myrinet
> On Tue, 4 Nov 2003, Bill Broadley wrote:
> > On Mon, Nov 03, 2003 at 10:19:48PM -0500, Joey Sims wrote:
> > > IB is about to find major traction in this industry and Myricom will
> > I've heard this statement for 2 years running, not that it couldn't
> > become true.
> Just look at all of the recent press releases for IB clusters being built.
> The hardware is finally actually available, and a lot of HPC clusters are
> starting to be built with it.
> In the spirit of full disclosure, I have three engineers on-site today
> from an IB vendor working with me to install a 192 node diskless IB
> > > Up to 10GB/sec is fairly fat today. The roadmap for IB has this
> > > interconnect technology ratcheted up way higher than 10GB.
> > Roadmaps are great, easy, and cheap. I'm most interested in
> > what I can build a cluster with today.
> > Do you mean 10 GB/sec = 10 Gigabytes/sec or 10 Gigabits/sec? In what
> > conditions? Where can I download a linux compatible driver? Linux
> > compatible MPI implementation? Linux/AMD64 drivers/MPI implementations?
> It's 10 gigabits per second (theoretical). Linux drivers are available
> from all of the vendors. Certain vendors (including the one I purchased
> my IB from) provide open-source drivers.
> There are a few MPI implementations, there are commercial versions MPI/Pro
> and ChaMPIon from MPI Software Technology, Inc. MVAPICH is available from
> OSC, and I'm hearing that there may be a version of LAM in the near
> I'm not sure of the status of the AMD64 drivers, although I know of at
> least one AMD64 cluster currently being built with IB, so at least some
> level of support exists.
> | Roger L. Smith Phone: 662-325-3625
> | Sr. Systems Administrator FAX: 662-325-7692
> | roger at ERC.MsState.Edu http://WWW.ERC.MsState.Edu/~roger
> | Mississippi State University
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf