Q: Building a small machine room? Materials/costs/etc.

[Michael Stein]

Just some additions, good coverage..

 - - -

Access to both front and back of racks...

>    a) power;
> There have been some very extensive discussions on the list about power
> gotchas associated with driving a large number of non-power-factor
> corrected switching power supplies.  There are vendors out there who
> sell server-room sized main transformers that compensate for the 180 Hz
> harmonic line distortion that can occur.

The 1U dual Xeon machines we've seen have power factor correcting
power supplies so the harmonic problems might not exist.

At 250 W/machine and 42 in a rack you do need an impressive amount
of power, over 10 KW/rack.  Watch out, future machines may need
more...

Watch the head/air flow.  Typical rack mount machines want cool air in
the front and exhaust hot air out the back.   A rack full will heat a
rack size volume of air all at once (something like 75 F in 99 F out
all at once!).   Don't run this hot air into the front of the next
row of racks...

Watch out -- the power usages varies depending on the CPU loading.

A quick measurement of an idle machine will be very misleading:

1U Dual Xeon 2.66 Ghz 2 GB ram  (& Kill-A-Watt power meter)
	idle:	109 W
	1 CPU	174 W	(1 x burnP6)
	2 CPU	254 W	(4 x burnP6)

Make sure the electricians know the power you intend to use -- they
need to size the circuits *larger* than this by some factor (from
code?) -- you can't put a 20 A load on a 20 A breaker.

>    c) structural integrity; 
> 
> Loaded racks are >>heavy<< -- they can weigh 800 lbs or even more if
> they contain UPS batteries

30 lb 1U machine * 42 in rack -> 1260 lbs 

or were they 40 lb machines?

40 lb 1U machine * 42 in rack -> 1680 lbs 

This does NOT include the weight of the rack, cables and any other stuff
you manage to hang on the rack (rack rails?).

> Oh, and I'd strongly suggest getting a professional architect (one with
> experience doing computer machine rooms) to do your plans.  And avoid
> bozo electricians who try e.g. wiring three phase circuits with a single
> common neutral.

I'd strongly suggest that (in addition) you also run the numbers yourself:

- power
- heat flow (and power for it?)

just to make sure (these numbers are large for more than a small
single digit size cluster and get absolutely huge for 1000 nodes).

Discovering a miscalculation here early is no big deal, late it's
a disaster...
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf