OctigaBay 12K

Robert G. Brown rgb at phy.duke.edu
Fri Nov 7 08:44:11 EST 2003

On Fri, 7 Nov 2003, Franz Marini wrote:

> Hello,
>   just discover this interesting, imho, company and its first product :
>   http://www.octigabay.com/
>   Their first product is a linux opteron-based cluster that they said 
> could scale up to 12K processors. The base system is a 3.5U shelf with 12 
> opterons, 1Tb/s aggregate switching capacity, 1 microsec interprocessor 
> latency and 77GB/s aggregate mem bandwidth.
>   Seems nice, I would like to know what rgb and some of the other people 
> in here think about it :)

Why, it looks simply lovely, as hardware I've never actually tried goes.
I mean, if the octigabay people want to send me one for free just so I
can write a review for it on this list and the brahma website, well,
from the look of it I wouldn't kick it out of my machine room for
chewing crackers... and I >>can<< be bought, folks, yes I can, just look
at the brahma vendors page and my brazen demand for t-shirts in exchange
for space:-) I'll even dig up something fine grained to run on it so
that I can pretend to really test it.

The bottom line is, well, the bottom line.  Pretty isn't enough.
Performance (even performance that is absolutely everything promised)
isn't enough.  It is PRICE performance that matters, or better yet
cost-benefit.  How does the cost compare to the benefits the design
delivers in your environment.

For my own personal code, for example, I don't NEED their fancy
interconnect, and I can rack up a bunch of opterons for the cost of the
basic hardware and a nice case to put them in.  They'd therefore have to
literally give it to me to make it a cost-benefit win (especially true
since I just spent the last of my money in this grant cycle buying hey,
whaddya know, a stack of 9 dual Opteron 242's for a hair over $20K).
However, there are people out there who run fine grained synchronous
parallel code that is bottlenecked at the network IPC level.  Even THERE
the computations have some intrinsic "value" in that there are finite
amounts of money people are willing to pay to get them done, and there
are choices.  So ultimately it will come down to whether there is a
match between the value of the computation (amount people are willing to
pay to get it done), the needs of the computation, and the marketplace.

It's one of these people that you need to ask about whether or not this
is a good deal or good arrangement.  My knee jerk reaction is that it is
lovely but a bit too far into the big iron side (SP3-ish) to be likely
to win a hard-nosed CB comparison relative to a DIY cluster with e.g.
myrinet or SCI for MANY clustervolken (the market gets smaller and
smaller the further up one travels to super-high-speed networks), but
corporate consumers and the larger government consumers shy away from
DIY, and even in the intermediate market it comes down to
price/performance, eh?  If they price it competitively with the other
high speed networks and it has clear benefits (as it looks like it
might) well then, who knows?


Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu

Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list