[Beowulf] Ambition

[Patrick Geoffray]

Ah Mark, I could not resist. Actually I could, but the list has been a 
little boring lately, so... :-)

Mark Hahn wrote:
> (Myri seems not to be very ambitious, and 10G eth seems to be straying into
> a morass of tcp-offload and the like...)

Myri is very ambitious, but you can be carefully ambitious or 
marketingly ambitious. Nobody buys an interconnect looking only at the 
specs. People try, benchmark, run their code, rationalize and buy what 
they need at the right price. If you look at what people are doing, 
there is a lot of Ethernet (Fast and GigE) because thats good enough for 
many, many codes. Then there is a smaller market for more demanding 
needs, either in term of performance or scalability, where you want to 
find the sweet spot in the performance/price curve.

Does it make sense to have 10Gb now ? I don't think so, and for several 
reasons:
* PCI-Express is not here yet: It's coming, yes, but it's not available 
in volume. Today, PCI-X supports 1 GB bidirectional, which is 4 Gb link 
speed. It's clearly the bottleneck right now. HyperTransport looks 
attractive, but there is no connector defined yet and vendors should be 
able to see a potential for volume before to commit resources for a 
native HT interface.
* 10 Gb optics are still expensive: price is going down, but there is 
not enough volume yet to drive the price down faster. Copper ? I still 
have nightmares about copper. 10 GigE will drive the technology price 
down as the 10 GigE market blossoms.
* 10 GigE is not attractive enough yet because there is no clear 
improvement at the application level. Running a naive IP stack at 10 Gb 
requires a lot of resources on the host. RDMA is just a buzword, it's 
not The Solution. Storage may leverage RDMA, but not IP and certainly 
not MPI. That's why people are working to put processing on the data 
path, but it is far from obvious so it takes some time.


Gigabit is the clear winner today and 10 GigE will be the clear winner 
tomorrow, because Ethernet is the de facto Standard. Everybody else are 
parasites, either breading on niches or marketing poop...

Patrick

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf