Economics of clusters was Re: top500 list (was: opteron VS Itanium 2)

Robert G. Brown rgb at
Tue Nov 18 08:40:33 EST 2003

On Mon, 17 Nov 2003, Jim Lux wrote:

> Also, for anything novel, there's always the "I'm not going first" problem.
> Like penguins wondering if there's a leopard seal in the water, someone's
> got to jump in and show that you won't die instantly.  Sometimes, those
> programs of perceived little value (and hence, little opprobrium if you
> fail) provide the mechanism to demonstrate a new technology.

Agreed.  However, the fundamental underlying issue is economics.  You
have an approximately fixed budget of X billions of dollars allocated to
publically funded research in the US.  This money is distributed among
many agencies for targeted disbursement.  The target selection process
(as you note) contains elements of national, state, local, and
scientific politics -- there is plenty of pork in it.  Some does indeed
get distributed as a sort of jobs program for starving corporations who
not completely coincidentally made large donations to selected
politicians (often on both sides of a race).  Other parts go to fund
some scientific director's pet project.

However, at the crux of each funding decision, politics or no, is the
issue of opportunity cost.  It was opportunity cost that ultimately
brought down the SSC.  It is never and "and" operator with funding, it
is inherently an "or" operator, given a fixed budget (and if the budget
is deliberately expanded to include somebody's pet project, the "or"
operator needs apply to the expanded but still fixed pool, even if that
decisioning is done at a very coarse granularity and one decision level
up, e.g. the US Senate).

So I totally agree with everything you say.  Sure, we need to climb
certain scientific mountains just because they are there, and trust that
new worlds lie on the other side of some of them.

HOWEVER, that does not release us from the obligation of making choices.
For every project that is funded, the pool of funds is diminished, and
alternative projects are rejected and not funded.  My personal research
colleague is an ARO grant officer, and I am fairly frequently treated to
a view of this from the other side -- so much he'd LIKE to fund, so
finite a pool of resources to fund from, so much politics that sends
huge chunks of money to specific venues outside of the normal review and
selection process.

It is difficult to raise oneself to a sufficiently elevated level to
even begin to judge a lot of this.  However, waste openly abounds.  I
know of quite a few places, for example, that have bought e.g. SP2's to
do HPC computations in years past.  These are (were), recall, quite
expensive boxes.  Naturally, they were publically funded from various
grants.  At the time they were purchased, the beowulf model was already
well known, and on a per-processor/per-cycle basis a competing beowulf
cost perhaps 1/5th as much.  The grantholders even KNEW about the
beowulf model, and were using the systems to run primarily
embarrassingly parallel applications that would have run efficiently on
a pile of PCs and sneakernet.

However, politics or open ignorance or "deals" cut by IBM, bewteen one
thing or another there they were with $500,000 computers whose actual
benefit to their owners could easily be matched by $100,000 beowulfs,
even on considerably finer grained code than they were running.  Then
there is the ADDITIONAL issue of whether the work being done was worth
the cost of the hardware, compared to all the other work that might have
been done with that money.

I'm sure that the money from sales like these floated IBM's boat through
tough times, and kept its sales and engineeering force from having to go
on welfare, and I'm not even sarcastic about it.  The same hand
ultimately feeds me, after all, and I have no wish to bite it.  However,
there >>is<< the opportunity cost issue of the extra $400K or so.  If
the work was really valuable, perhaps it could have been completed much
faster with a more intelligent cluster model.  If an intelligent cluster
model had been used at the lower rate, perhaps some other deserving
project could have been funded to keep ITS researchers and support
people out of homeless shelters.

Choice is essential.  Cost-benefit is at the heart of economic choice.
Where admittedly, the liver is politics...;-)


Robert G. Brown	             
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list