[Beowulf] Gigabit Ethernet vs Myrinet
Douglas Eadline, Cluster World Magazine
deadline at linux-mag.com
Sat Dec 27 13:21:34 EST 2003
On Sat, 27 Dec 2003, Mahmoud Saleh wrote:
> Reading a couple of comparison tables regarding latency of different NIC
> protocols, I noticed that many solutions suggest to use Myrinet style
> NIC due to its low latency, namely around 8usec for I/O intensive
> cluster jobs. I was wondering if Gigabit Ethernet does the same.
> Suppose that Maximum packet size in GE is 1500 bytes and the minimum is
> aroud 100 bytes. This translates to an average of 800 bytes or 6400
> bits. In Gigabit Ethernet that would cause a delay of 6400/10^9 sec or
> 6.4usec for packet assembly, which is in the same order as Myrinet.
The best 1 byte latency for GigE I have measured has been 25 us.
This test was using netpipe/TCP. It is hard to provide a solid number
because Ethernet chip-sets/drivers vary as do motherboards that include
GigE. The best thing to do is test some hardware.
> Is this justification correct? If so, how wise is it to use Gigabit
> Ethernet for an I/O intensive cluster?
More tests are needed to answer that. With Myrinet, Quadrics, SCI, you
will get better performance -- and spend more money. Somethings you may
need to consider with this decision:
1. What API you will use MPI, PVM, sockets? (API can add overhead to
2. How many nodes do expect to use ?
3. Is there a single NFS server for the data or are you using something
like PVFS or GFS?
4. What are your I/O block sizes?
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf