10 Pb/sec? Re: IB vs Myrinet

[Jim Lux]

>
> > Up to 10GB/sec is fairly fat today.  The roadmap for IB has this
> > interconnect technology ratcheted up way higher than 10GB.
>
>Myricom's roadmap goes up way higher than 10 Pb/s, if that makes you
>feel more comfortable.
>
>Patrick
>--


Wait a minute here... You might run into some fundamental physics problems, 
especially when getting "way higher" than 10 Pb/sec... I'd like to see what 
you've paved that road map with, and make sure it doesn't ruin my shoes 
when I walk on it <grin>.


Say you can run a wire/fiber/whathaveyou at 10 Gb/sec (working in powers of 
10, here, and 100 Gb/sec is just too hard to envision..)

10 Pb/sec would be a million times faster (Peta = 1000 Tera = 1000,000 
Giga, right?)

Where are you going to fit those million wires/fibers/connectors?  Let's 
say you're using optical fibers that are 10 micron in diameter (which is a 
fairly impressive feat). Assuming you space them by 5 micron, you can pack 
1000x1000 of them in 5 mm x 5 mm... There is a bit of a problem with 
interconnects, etc., but perhaps you can terminate it right on top of a 
die, and the circuitry for one channel is small enough to fit?  How 
tolerant is Myricom's hardware of skew and jitter between the parallel 
lines?  At least with a million lines, you can use statistical techniques 
to characterize it, and you'd almost have to use some form of forward error 
correction, so the extreme outliers wouldn't give you troubles.

You might be able to push the bit rate a bit higher.. We've got some 
components working at 94 GHz here, and there are some novel techniques with 
propagating the wave in the boundary outside a dielectric rod, so the loss 
is reasonable.  We haven't figured out how to turn a corner yet, but that 
wouldn't violate any laws of physics. The distance is short, so maybe 
waveguide can work (optical fiber is waveguide and fairly low loss)

Hmm.. now, about that X-ium or X-lon mobo that is going to 
send/accept/process the 10 Pb/s data stream....

What is the physical limit on memory speed?  The cells can only be so 
small, and you've got to propagate the charge across it.  I suppose, 
theoretically, one could use a charge as small as 1 electron, so that sort 
of provides a lower bound.  I've heard of CMOS processes with fT of 10 GHz 
in very small feature sizes (the wireless market really wants to do RF and 
digital on the same chip).  Say you get that 10 GHz memory... you'll need 
million way interleaving.

This starts to make the SIMD systolic arrays look more attractive doesn't it.

Maybe free space optical interconnects with monolithically fabricated 
optics over the chip might be a solution? HeNe lasers are about 474 THz, as 
I recall, so if you baseband encode your 10 Pb/sec bitstream, you're only 
looking at 30 nm extreme UV kinds of bandwidth.  Lends new meaning to the 
RF designer's term: DC to light...






James Lux, P.E.
Spacecraft Telecommunications Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf