New cluster benchmark proposal (Re: top500 list)

Douglas Eadline, Cluster World Magazine deadline at
Tue Nov 18 16:24:27 EST 2003

On Tue, 18 Nov 2003, Robert G. Brown wrote:

> On Tue, 18 Nov 2003, John Hearns wrote:
> > There's a page frpm Paralogic with a packaged set of
> > benchmarking tools
> > 
> > Maybe could be a start to your ideas?
> Doug (Eadline) and I have talked about this for years now, and he has
> put together a small package that are used for design purposes, I
> believe, at paralogic.  Maybe he'll write an article about this himself
> in his new mag...:-)

Well, yes and more. We are going to address the benchmark thing
in a bit more detail in the future. The BPS package is described
It will be getting an upgrade soon and there will be some
real codes added as well. Stay tuned. We will have an issue on 
featuring benchmarking as well. 

You will notice that it is call BPS (Beowulf Performance Suite)
and not BBF (Beowulf Benchmark Suite). The reason is that BPS was not 
supposed to be a benchmark per se. It was intended to generate
a baseline from which to measure the effect of changes to the cluster
 ( driver, new kernel, etc.) and to diagnose some problems. I
intentionally omitted HPL because I did not want the suite to become a
contest until it could provide good data on which good engineering
decisions could be made. 

> I don't think that they are quite "done", though, (at least the last
> time I checked) so yes, I'd call it a "start" to the idea.  Not really
> my idea, as you can see.  I think there are lots of folks who have
> thought on this, and lots more that have a de facto suite they use
> whether or not they are packaged.  lmbench, netpipe, netperf, bonnie,
> memtest86 -- lots of tools out there for doing bits of this, some of
> them very nice.
> cpu_rate (which is available on my own website under the Beowulf link)
> is another such tool.  I've no time to work on it just now, but I'm in
> midstream on a fairly major rewrite to really separate out the timing
> harness and test invocation process so that code snippets can be wrapped
> in a standard subroutine pro/epilogue and timed, with correct
> subtraction of the subtroutine overhead.  This isn't as easy as you
> might think (at least to get consistent results) but when it is finished
> cpu_rate should be a highly extensible way to wrap up anything from
> microbenchmarks to specific code fragments you want to test.
> One day we might even get a little group together at a meeting and kick
> around specs for a really nice, full GPL cluster exerciser toolset that
> can test, benchmark, and help debug problems with clusters large and
> small.
>    rgb
> Robert G. Brown	             
> Duke University Dept. of Physics, Box 90305
> Durham, N.C. 27708-0305
> Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at
> _______________________________________________
> Beowulf mailing list, Beowulf at
> To change your subscription (digest mode or unsubscribe) visit

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list