canon at nersc.gov
canon at nersc.gov
Thu Jul 24 10:36:53 EDT 2003
We have a similar setup and have seen a similar problem.
The vendor determined the fans weren't robust enough
and sent replacements.
With regards to adding gaps... We have considered
(but haven't implemented) adding a gap every 10ish nodes.
This would be primarily to reset the vertical temperature
gradient. You can run your hand up the exhaust and feel
the temperature difference between the top and the bottom.
I suspect hot air rises. :-) The gap would allow us
to "reset" the temperature gradient. This would only
lose us 2 or 3U which isn't too bad if it helps the
Shane Canon voice: 510-486-6981
PSDF Project Lead fax: 510-486-7520
National Energy Research Scientific
1 Cyclotron Road Mailstop 943-256
Berkeley, CA 94720 canon at nersc.gov
> At 09:04 AM 7/23/2003 -1000, Mitchel Kagawa wrote:
> >I run a small 64 node cluster each with dual AMD MP2200's in a 1U chassis.
> >I am having problems with some of the nodes overheating and shutting down.
> >We are using Dynatron 1U CPU fans which are supposed to spin at 5400 rpm but
> >I notice that a lot (25%) of the fans tend to freeze up or blow the bearings
> >and spin at only 1000 RPM, which causes the cpu to overheat. After careful
> >inspection I noticed that the heatsink and fan sit very close to the lid of
> >the case. I was wondering how much clearance is needed between the lid and
> >the fan that blown down onto the short copper heatsink?
> To a first order, the area of the inlet should be comparable to the area of
> the outlet. A 60 mm diameter fan has an area of around 2800 mm^2. If you
> draw from around the entire periphery (which would be around 180 mm), you'd
> need a gap of around 15 mm (probably 20 mm would be a better idea) That's
> a fairly significant fraction of the 45 mm or so for 1 rack U.
> > When I put the lid
> >on the case it is almost as if the fan is working in a vaccum because it
> >actually speeds up an aditional 600-700 rpm to over 6000 rpm... like there
> >is no air resistance. Could this be why the fans are crapping out? I was
> >thinking that a 60x60x10mm cpu fan that has air intakes on the side of the
> >fan might work better but I have not seen any... have you?
> >Also the vendor suggested that we sepetate the 1U cases because he belives
> >that there is heat transfer between the nodeswhen they are stacked right on
> >top of eachother. I thought that if one node is running at 50c and another
> >node is running at 50c it wont generate a combined heatload of more than 50c
> So, your vendor essentially claims that his 1U case will work just fine as
> long as there is a 1U air gap above and below?
> Let's look at the problem with some simple calculations:
> Assume no heat transfer up or down (tightly packed), and that no heat
> transfers through the sides by conduction, as well, so all the heat has to
> go into airflow.
> Assume that you've got to move about 200W out of the box, and you can
> tolerate a 10C rise in temperature of the air moving through the box. The
> question is how much air do you need to move. Air has a density of about
> 1.13 kg/m^3 and a specific heat of about 1 kJ/kgK.
> 200W is 0.2 kJ/sec, so you need to move 0.02 kg of air every second (you
> get a 10 deg rise) is about 0.018 cubic meters/second. To relate this to
> more common fan specs: about 40 CFM or 65 cubic meters/hr. (I did a quick
> check on some smallish 60mm fans, and they only flow around 10-20 CFM into
> NO backpressure... http://www.papst.de/pdf_dat_d/Seite_13.pdf
> for instance)
> How fast is the air going to be moving through the vents? What's the vent
> area... say it's 10 square inches (1 inch high and 10 inches wide...).. 40
> CFM through .07 square feet is 576 ft/min for the air flow (which is a
> reasonable speed.. 1000 ft/min is getting fast and noisy...)
> But here's the thing.. you've got 32 of these things in the rack... are you
> moving 1300 CFM through the rack, or are you blowing hot air from one
> chassis into the next.
> >Mitchel Kagawa
> >Systems Admin.
> >Beowulf mailing list, Beowulf at beowulf.org
> >To change your subscription (digest mode or unsubscribe) visit
> James Lux, P.E.
> Spacecraft Telecommunications Section
> Jet Propulsion Laboratory, Mail Stop 161-213
> 4800 Oak Grove Drive
> Pasadena CA 91109
> tel: (818)354-2075
> fax: (818)393-6875
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beo
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf