[Beowulf] re: power prediction and planning

Mark Hahn hahn at mcmaster.ca
Mon Oct 5 03:14:42 EDT 2009

in reference to recent mention of TDP and power planning,
what do you think the trend in the next few years will be?

5 years ago, people were talking about exponential increases 
in power and power density - afaikt with straight faces.
that clearly didn't happen.  in fact, dissipation has clearly
gone down (on a per-node basis - power per flop has gone through
the floor!).

I'm looking at current nodes and seeing substantial benefits to
improvements in cpu power, lower voltage/power ram, more sensible
chipsets, more efficient power supplies.  we have a lot of nodes
bought ~4 years ago - dual-socket HP DL145G2's.  cpu power was 
around 100W/socket, and I suspect PSUs were probably ~80% efficient.
now it would be hard to miss 92-93% efficient PSUs and CPUs
in the 60-80W range.  I'd be surprised if support chips haven't 
improved as well.  I know even disks are dissipating a lot less,
even though they never were a large component.

I've even seen some vendors brag about how their fans are more 
efficient (which resonates with me because when a DL145G2 gets warm,
the ten (10) fans ramp up dissipate noticably more power, 
which doesn't help the room temperature at all...)

I'm also not really talking about heat density - I generally assume
that we'll get an average of 2 nodes per 1U - ~80/rack.

Intel seems to be agressively pushing fab tech as well - .32 nm is 
due this year, and they claim significant speed/power advances.

SO: do you expect pretty much constant per-node dissipation? 
if higher, why?  if lower, savings due to what?

we've planned with 300W/node for a long time, but current nodes are
noticably cooler than that.  I imagine next-gen nodes will be lower 
power, not higher, unless something else changes (4 socket becomes 
common, or every node needs a Fermi or Larrabee card...)

also, does anyone have thoughts about machineroom features that 
would support multiple generations, preferably without a lot of reno?
for instance, we have mostly L6-30 power, which seems reasonably safe 
for commodity systems.  but how about rack-back water cooling systems? 
what are the chances of getting multiple generations out of that 
(at least the heat-rejection part, if not the pipes or server-side 
heat-exchangers.)  is anyone working on commodity-ish systems 
with water/coolant going to each node?  (that is, skip or minimize 
the use of air)

thanks, mark hahn.
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list