space requirements

Robert G. Brown rgb at
Mon Mar 31 04:20:15 EST 2003

On Sun, 30 Mar 2003, Judd Tracy wrote:

> > > Just to let you know Extreme Networks has their BlackDiamond 6818 that
> will
> > > support upto 1440 100BT ports with 640Bbps non-blocking backplane all in
> 35U
> >
> > it's pretty impressive; I imagine the pricetag is too.
> > I think the 640 Gbps is a bit wrong though (actually 256):
> >
> I was looking at this page
>       BlackDiamond 6816
>       .   640 Gbps non-blocking bandwidth
>       .   Route/filter/forward 192 million pps
> > I'd question whether such a cluster would make much sense, though.
> > do you know of a class of probles that would lead to O(1440-way)
> > parallel jobs that needed only 100bT and yet had no hotspots?
> > 31 47U racks full of 1U nodes, ugh!
> I don't know of any that would need full speed at that size.   I was just
> pointing out that theses sizes of swtiches do exists if you need them.
> > there *are* sites which have real cluster networks (quadrics,
> > perhaps myrinet) of that size.  iirc they're not quite full BB
> > (fat trees and whatnot).  I'd still prefer a 740-node Quadrics
> > net over 1440 100bT ;)
> Even thought they are not full BB I am sure that are still faster then a
> full 100BTx network.

Being as how I'm ignorant and too lazy to look it up myself unless I
have to, do you happen to know offhand what they cost (in total or per
port) at this scale?  

There is potentially a fairly significant and interesting break in the
cost/benefit spectrum here.  A rack full of cheap 100BT switches
interconnected with gigabit uplinks to a gigabit switch defines a
lowball solution at prices likely to be in the tens of dollars per port,
with very definite limitations in latency and bisection bandwidth.  If a
kiloport-scale solution with a "decent" (but more symmetric and lower
latency) bisection bandwidth were no more than 2-3x this cost, it might
make sense for at least some buyers even if they were only using the
switch for coarse grained or EP code.  Then there are the serious
networks for folks running fine grained code, but those typically cost
order of a kilodollar per port (within a factor of 2-3, mind you:-) that
have 10x the bandwidth and 1/10 the latency.  I'm curious as to just
where the very large 100BT ethernet switches lie in between...and who is
buying them there.


Robert G. Brown	             
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list