Network RAM revisited

Greg Lindahl lindahl at
Thu May 29 03:12:26 EDT 2003

On Wed, May 28, 2003 at 07:29:52PM -0400, Robert G. Brown wrote:

>  Weather, cosmology, there are a few
> "enormous" problems where researchers ALWAYS want to work bigger than
> current physical limits and budgets permit and can still get useful
> results even with the penalties imposed by disk or other VM extensions.

Yes, but those are bad examples.  The more memory you use in a 72-hour
weather forecast (more memory means more input data, higher resolution
output), the more cpu time it takes to make the computation -- the cpu
time increases as memory**(4/3). In reality, weather forecasting is
actually limited by our ability to insert fudge factors for local
physics, i.e. things that aren't resolved on the grid. You need ~ 7
cells in 1 km to resolve a thunder head. Current production runs are
at 10-20km cells. Those pesky weather satellites produce plenty of
high-resolution input. So memory size is not a problem, and the data
is big enough that you can use 100-200 cpus with Myrinet. Not too

Now if you have a serial code, sure, you might be able to use network
RAM. But parallelizing a weather code is old hat.

By the way, Opteron mobos make your arguments about big memory
motherboards a bit obsolete... there are 2 cpu motherboards with 2x4 =
8 total DIMM slots. I'm not sure what 4 cpu motherboards will do, but
it might be 4x4 = 16. These 2 cpu motherboards are not that
expensive... the Khapri is gold-plated, but its competition is not.
Bigger DIMMs are expensive, but they are always falling.

I always thought it was kind of a shame that motherboards used to have
such low limits on memory. While I was at UVa I wanted to build a
small cluster with 1 Tbyte of memory for big-memory apps. The figure
of merit was that the cluster was going to have a total cost less than
2x the price of just the memory. If Digital had made a low end Alpha
motherboard that could address big memory, I could have done it, but
their low end chipset didn't carry out enough address pins. Grrr.

-- greg

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list