top500 list (was: opteron VS Itanium 2)

canon at canon at
Mon Nov 17 14:37:38 EST 2003

I think the Big Mac guys deserve snaps for pulling this system off.
The VA Tech guys accomplished a real feat and I suspect they
worked their collective butts off to do it.  Who would have
predicted a Mac based cluster to be in the top 5?  Not me.

I still suspect this is an anomaly.  I don't think we are
going to see a bunch of Mac based clusters breaking into the
list next year.  Which begs the question "Why not?"  My feeling
is when you build a system that large, you want to know you
can get real work done with it.  That's where committed vendors
and large user communities become important.  At this point,
Big Mac is a one-of-a-kind.  The Apple crowd has never even
looked at HPC before this (probably because its typically a money loser).
Meanwhile there seems to be a growing community of people
that want to use the Opteron for HPC.  That's why I expect we will see 
more Opteron clusters over time.  But hey, maybe Big Mac will
make people look at the Apple stuff more closely.  There still
seems to be a lot of missing pieces though (parallel debuggers, profiling
tools, libraries, etc).  The long term measure for the Big Mac
is to see how well they can use the system, especially for 
generic codes.

Regarding the top500, I see the point of the top500 as being a ranking 
of capability of various machines.  Unfortunately, its difficult to come up
with a benchmark that accurately measures capability that is
super portable and easy to run.  Personally I don't
think LinPACK should necessarily be that code, but
at least it forces people to run a consistent problem
across the entries.

I think adding costs would be interesting since any real
purchase has to take this into consideration, but it would
be more for comparison purposes and not ranking.  NERSC(#9) has
used (sustained performance)/$, where the sustained performance
is calculated from a collection of standard codes used by
the NERSC community.  I think this approach has served us
well, but it can be challenging to get apples to apples comparison
when you are talking about projecting the performance of codes
to large scales.  Each vendor does the projection their own way and its
tricky to know how much to believe, especially if its on
a non-existent hardware or at untested scales.

My true measure for the top500 would be the value of the
science (or work) accomplished with it, a difficult to 
impossible thing to determine.  NERSC's puts all the 
emphasis on the science.  This means considering: how usable the system
is; how hard is it to harness the full capability of the system;
what will the sustained performance be.  Then we try to squeeze
every cycle out of the system.  We've ran Seaborg (#9)
with +90% utilization for years now.  We've gotten tons of 
science done with it, just like we did the T3E before it.
It can be a little disappointing to watch your system slide
down the rankings, when you know its still being used to do great 
stuff and its still making a large impact.  But I guess that's
just the nature of Moore's law.

I think this years top 500 raises all sorts of interesting questions.
How will the X-1 evolve?  Will Opteron systems become a big player?
What about Itanium?  Will the Blue Gene based systems make an
impact?  Its certainly more interesting than a few years ago
where there were just a handful of vendors and no clear direction
where things were heading.


Disclaimer:  These statements represent my own opinions and not those of NERSC.

Shane Canon                             voice: 510-486-6981
PSDF Project Lead                       fax:   510-486-7520
National Energy Research Scientific
  Computing Center
1 Cyclotron Road Mailstop 943-256
Berkeley, CA 94720                      canon at

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list