Q: Building a small machine room? Materials/costs/etc.
mas at ucla.edu
Wed Sep 17 12:09:01 EDT 2003
Just some additions, good coverage..
- - -
Access to both front and back of racks...
> a) power;
> There have been some very extensive discussions on the list about power
> gotchas associated with driving a large number of non-power-factor
> corrected switching power supplies. There are vendors out there who
> sell server-room sized main transformers that compensate for the 180 Hz
> harmonic line distortion that can occur.
The 1U dual Xeon machines we've seen have power factor correcting
power supplies so the harmonic problems might not exist.
At 250 W/machine and 42 in a rack you do need an impressive amount
of power, over 10 KW/rack. Watch out, future machines may need
Watch the head/air flow. Typical rack mount machines want cool air in
the front and exhaust hot air out the back. A rack full will heat a
rack size volume of air all at once (something like 75 F in 99 F out
all at once!). Don't run this hot air into the front of the next
row of racks...
Watch out -- the power usages varies depending on the CPU loading.
A quick measurement of an idle machine will be very misleading:
1U Dual Xeon 2.66 Ghz 2 GB ram (& Kill-A-Watt power meter)
idle: 109 W
1 CPU 174 W (1 x burnP6)
2 CPU 254 W (4 x burnP6)
Make sure the electricians know the power you intend to use -- they
need to size the circuits *larger* than this by some factor (from
code?) -- you can't put a 20 A load on a 20 A breaker.
> c) structural integrity;
> Loaded racks are >>heavy<< -- they can weigh 800 lbs or even more if
> they contain UPS batteries
30 lb 1U machine * 42 in rack -> 1260 lbs
or were they 40 lb machines?
40 lb 1U machine * 42 in rack -> 1680 lbs
This does NOT include the weight of the rack, cables and any other stuff
you manage to hang on the rack (rack rails?).
> Oh, and I'd strongly suggest getting a professional architect (one with
> experience doing computer machine rooms) to do your plans. And avoid
> bozo electricians who try e.g. wiring three phase circuits with a single
> common neutral.
I'd strongly suggest that (in addition) you also run the numbers yourself:
- heat flow (and power for it?)
just to make sure (these numbers are large for more than a small
single digit size cluster and get absolutely huge for 1000 nodes).
Discovering a miscalculation here early is no big deal, late it's
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf