IB vs Myrinet

Anthony Skjellum tony at mpi-softtech.com
Wed Nov 5 13:22:04 EST 2003


Roger, FYI, we have had only trivial issues supporting Opteron-based IB, for
instance, things worked in a matter of hours with Racksaver-based Opteron
dual 1U boxes with
Mellanox cards and VAPI drivers.   This is with a SuSE/UNITED linux type
build
that Racksaver generally ships (nothing specialized).

In polling mode, we had about 4.5us latency from the Mellanox A1 cards with
VAPI
out of the box from MPI/Pro (above 4-5 months ago).

After that, we ran ChaMPIon/Pro on some larger cluster configs with dual
Opteron too,
all very straightforward.

-Tony

----- Original Message ----- 
From: "Roger L. Smith" <roger at ERC.MsState.Edu>
To: "Bill Broadley" <bill at math.ucdavis.edu>
Cc: "Joey Sims" <jsims at csiopen.com>; <beowulf at beowulf.org>
Sent: Tuesday, November 04, 2003 8:34 AM
Subject: Re: IB vs Myrinet


> On Tue, 4 Nov 2003, Bill Broadley wrote:
>
> > On Mon, Nov 03, 2003 at 10:19:48PM -0500, Joey Sims wrote:
> >
> > > IB is about to find major traction in this industry and Myricom will
> not
> >
> > I've heard this statement for 2 years running, not that it couldn't
> > become true.
>
> Just look at all of the recent press releases for IB clusters being built.
> The hardware is finally actually available, and a lot of HPC clusters are
> starting to be built with it.
>
> In the spirit of full disclosure, I have three engineers on-site today
> from an IB vendor working with me to install a 192 node diskless IB
> cluster.
>
> > > Up to 10GB/sec is fairly fat today.  The roadmap for IB has this
> > > interconnect technology ratcheted up way higher than 10GB.
> >
> > Roadmaps are great, easy, and cheap.  I'm most interested in
> > what I can build a cluster with today.
> >
> > Do you mean 10 GB/sec = 10 Gigabytes/sec or 10 Gigabits/sec?  In what
> > conditions?  Where can I download a linux compatible driver?  Linux
> > compatible MPI implementation?  Linux/AMD64 drivers/MPI implementations?
>
> It's 10 gigabits per second (theoretical).  Linux drivers are available
> from all of the vendors.  Certain vendors (including the one I purchased
> my IB from) provide open-source drivers.
>
> There are a few MPI implementations, there are commercial versions MPI/Pro
> and ChaMPIon from MPI Software Technology, Inc.  MVAPICH is available from
> OSC, and I'm hearing that there may be a version of LAM in the near
> future.
>
> I'm not sure of the status of the AMD64 drivers, although I know of at
> least one AMD64 cluster currently being built with IB, so at least some
> level of support exists.
>
>  _\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_\|/_
> | Roger L. Smith                        Phone: 662-325-3625
|
> | Sr. Systems Administrator             FAX:   662-325-7692
|
> | roger at ERC.MsState.Edu                 http://WWW.ERC.MsState.Edu/~roger
|
> |                       Mississippi State University
|
>
|____________________________________ERC__________________________________|
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
>

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



More information about the Beowulf mailing list