The Norbert Limulus Cluster: Difference between revisions
No edit summary |
No edit summary |
||
| (One intermediate revision by the same user not shown) | |||
| Line 19: | Line 19: | ||
===Hardware Manifest=== | ===Hardware Manifest=== | ||
[[File:HW_Manifest.png]] | |||
===Performance=== | ===Performance=== | ||
HPL 2007: 47.71 GFLOPS before RAM upgrade (1 GB per node) Full HPL Results | *HPL 2007: 47.71 GFLOPS before RAM upgrade (1 GB per node) [[Full HPL Results 2007]] | ||
HPL 2008: 53.44 GFLOPS after RAM upgrade (2 GB per node) Full HPL Results | *HPL 2008: 53.44 GFLOPS after RAM upgrade (2 GB per node) [[Full HPL Results 2008]] | ||
Price to Performance | |||
===Price to Performance=== | |||
September 2007: $48.25/GFLOP | September 2007: $48.25/GFLOP | ||
September 2008: $39.15/GFLOP | September 2008: $39.15/GFLOP | ||
===Power=== | ===Power=== | ||
| Line 38: | Line 40: | ||
The following table summarizes the idle and loaded power draw (in Watts) measured for each node and processor (E6550=Core2 Dual 2.33GHz, Q6600=Core2 Quad 2.4GHz). All cores were busy running the single copies of the NAS parallel benchmark. The highest Watts valued was used. | The following table summarizes the idle and loaded power draw (in Watts) measured for each node and processor (E6550=Core2 Dual 2.33GHz, Q6600=Core2 Quad 2.4GHz). All cores were busy running the single copies of the NAS parallel benchmark. The highest Watts valued was used. | ||
[[File:Power-Use.png]] | |||
Of course, if the nodes are not being used, they are manually powered down to stand-by mode (5Watts) and then restarted with WOL (wake on lan). I would like to integrate this into the scheduler. | |||
===Power Load Running HPL=== | ===Power Load Running HPL=== | ||
When it is fully loaded running the HPL benchmark Norbert (four E6550's) draws between 371-483 Watts. The number varied as the benchmark progressed which I assumed is an indication of how modern processors are managing power consumption based on load. Even with the maximum power draw, the power performance ratio is a comfortable 10-11 Watts per GFLOPS. | When it is fully loaded running the HPL benchmark Norbert (four E6550's) draws between 371-483 Watts. The number varied as the benchmark progressed which I assumed is an indication of how modern processors are managing power consumption based on load. Even with the maximum power draw, the power performance ratio is a comfortable 10-11 Watts per GFLOPS. | ||
Latest revision as of 11:36, 29 September 2026
Norbert Design
[edit]Norbert was designed to use totally off-the-shelf parts. There is no custom case work required. Of course the hardware could be placed in a smaller volume with a custom case design, but that would require extra work and our goal is to create an easy to assemble system.
Norbert basically consists of a head node and three diskless worker nodes. All nodes have an extra PCIe GigE card for compute traffic. The head node has a DVD, a hard drive, and an extra network card to connect to the LAN. Each node has a single socket Gigbyte GA-G33M-DS2R motherboard with a 2.33 GHz Core2 Duo (E6550) processor, and one GByte of RAM (upgraded to 2GB in 2008). There are a total of eight cores in Norbert. Norbert Pictures
Norbert measures 26 inches (66 cm) across, 16 (41 cm) inches deep, and 25 (64 cm) inches high. I placed the cases on a inexpensive wire shelf cart so it can be moved easily.
The following is a front view. The two GigE switches are on the top. Units can be removed without too much trouble, however, there is little need to take anything off the shelves. The cases have build in temperature sensors (blue lights on the front). The head node is located on the top left (it has a DVD drive).
The following a picture of the back. The two switches can be seen on the top. Of course there is a need for some cabling clean up (I plan on making custom length cables). There is a powerstrip on bottom middle of the shelf. The single powerstrip cord is the only power connection needed.
Hardware Manifest
[edit]
Performance
[edit]- HPL 2007: 47.71 GFLOPS before RAM upgrade (1 GB per node) Full HPL Results 2007
- HPL 2008: 53.44 GFLOPS after RAM upgrade (2 GB per node) Full HPL Results 2008
Price to Performance
[edit]September 2007: $48.25/GFLOP September 2008: $39.15/GFLOP
Power
[edit]Note: all measurements were made "at the wall" using a Kill-O-Watt meter.
Node Idle and Load Power
The following table summarizes the idle and loaded power draw (in Watts) measured for each node and processor (E6550=Core2 Dual 2.33GHz, Q6600=Core2 Quad 2.4GHz). All cores were busy running the single copies of the NAS parallel benchmark. The highest Watts valued was used.
Of course, if the nodes are not being used, they are manually powered down to stand-by mode (5Watts) and then restarted with WOL (wake on lan). I would like to integrate this into the scheduler.
Power Load Running HPL
[edit]When it is fully loaded running the HPL benchmark Norbert (four E6550's) draws between 371-483 Watts. The number varied as the benchmark progressed which I assumed is an indication of how modern processors are managing power consumption based on load. Even with the maximum power draw, the power performance ratio is a comfortable 10-11 Watts per GFLOPS.