Optimal SMP Stucture for Opteron
Jeffrey B. Layton
laytonjb at comcast.net
Sun Nov 16 11:11:26 EST 2003
I think the first question to answer, what do you want to do
with the cluster? In other words, what are your applications?
Also, what do you mean 'without corporate resources'?
If you can start filling in the answers to these questions, it
becomes a little easier to give advise (although that has never
stopped me before) :)
Let me embellish a little. If you application doesn't require
much network bandwidth or if network latency is not important,
then you can consider a slower, cheaper network such as FastE
or GigE (GigE is pretty cheap for smaller systems right now
or you can use the smaller GigE switches in some kind of tree
Even if bandwidth and latency is important to some degree
you could also start looking at dual Opteron boxes instead of
4-way or 8-way boxes. This may get you the performance you
need or may have a better price/performance ratio.
One last comment. This next week is SC2003 so many of the
regular posters to this list won't be posting much. So don't
be surprised if you don't get many answers to your questions
right away. However, in the meantime, I think I can safely say
that if you start thinking about the answers to these first few
questions, the more likely you are to get more concrete answers.
P.S. And to all of you folks going to SC2003 while I sit at
work sucking on the glass teet, green with envy, the lot of you
are all bastards! Bastards I say! Where's the love? I need T-shirts!
> In order to build a 16 to 32 processor Opteron
>without corporate resources; a high performance and
>economic approach would seem to be a cluster of Quad
>motherboards interconnected by infiniBand host channel
>adapters( a la SBS Technologies) or ,possibly, a less
>expensive RemoteDMA data transfer PCI card.
> This approach stems from the little I know of the
>Opteron memory model; it seems that the Opteron leans
>towards NUMA memory management in a SMP system with
>more than 8 CPU's.Many have opined that Opteron's
>current Hyper Transport bus becomes saturated with 8
>CPU's on one board. The locality of each CPU's memory
>seems to fit a NUMA model best
>and, more so, as the number of CPU's rise.
> SGI's Altrix has an approach somewhat similar to
>this. One distinctive feature that SGI has added to
>the Linux kernel to empower their NUMA model is
>process affinity - linking a process to one (or a
>group) of CPU's.
> If someone with an intimate knowledge of NUMA could
>critique this general approach for a SMP Opteron
>system, I would appreciate it greatly.
>Do you Yahoo!?
>Protect your identity with Yahoo! Mail AddressGuard
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf