[Beowulf] Re: [Rocks-Discuss]Intel compiler specifically tuned for SPEC2k (and other benchmarks?)

Lombard, David N david.n.lombard at intel.com
Wed Feb 11 16:58:48 EST 2004


From: Robert G. Brown; Wednesday, February 11, 2004 8:44 AM
[deletia]
> 
> Finally, there are application benchmarks.  These tend to be "atomic"
> but at a very high level (an application is generally very complex).
> These are also subject to the Evil of comparative benchmarks (in fact
> some of comparative benchmark suites, especially in the WinX world,
are
> a collection of application benchmarks).

True.  I cringe to think how many systems were bought for scientific and
technical computations based on UT2003 "benchmarks".

> They also have some evil of
> their own when the application in question is commercial and not open
> source -- you have effectively no control over how it was built and
> tuned for your architecture, for example, and may not even have
> meaningful version information.

Let's be fair here. An ISV application is not the definition of evil.

Clearly, "you have effectively no control over how an application was
built and tuned for your architecture" has no direct correspondence to
performance.

Having been on the ISV side of the fence, and spent a tremendous amount
of energy making sure that each port of the application performed as
well as it could, I'm quite confident in saying we generally succeeded
in maximizing performance.  Realize that we had day after day to spend
on performance, usually with the attention of one or more experts from
the platform vendor at our beck and call -- and those experts would
spend even more time on even more narrow aspects of performance.

Having said that, there are some notable ISV applications that simply do
not perform as well as they should.  This can occur for a host of
reasons, such as they, did not care, didn't know how, could/would not to
make the effort, didn't have the time, were ignored by the vendor, &etc
-- basically the very same reasons that some people who don't work for
ISVs fail to make their own applications perform as well as they could.

> However, they are also undeniably useful.  Especially when the
> application being benchmarked is YOUR application and under your
> complete control.

Regardless of ownership or control, they're especially useful when
you're looking at an application being used in the way you intend on
using it. Many industrial users buy systems to run a specific list of
ISV applications.  In this instance, the application benchmark can be
the most valid benchmark, as it can model the system in the way it will
be used -- and that's the most important issue.

I'm not disagreeing with your message.  I too try to make sure that
people use the right benchmarks for the right purpose; I've seen way too
many people jump to absurd conclusions based on a single data point or
completely unrelated information.  I'm just trying to sharpen your
message by pointing out some too broad brush strokes...

Well, maybe I don't put as much faith in micro benchmarks unless in the
hands of a skilled interpreter, such as yourself.  My preference is for
whatever benchmarks most closely describe your use of the system.

-- 
David N. Lombard
 
My comments represent my opinions, not those of Intel Corporation.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



More information about the Beowulf mailing list