[Beowulf] Gigabit Ethernet vs Myrinet

[Douglas Eadline, Cluster World Magazine]

On Sat, 27 Dec 2003, Mahmoud Saleh wrote:

> Folks,
> 
> Reading a couple of comparison tables regarding latency of different NIC 
> protocols, I noticed that many solutions suggest to use Myrinet style 
> NIC due to its low latency, namely around 8usec for I/O intensive 
> cluster jobs. I was wondering if Gigabit Ethernet does the same.
> 
> Suppose that Maximum packet size in GE is 1500 bytes and the minimum is 
> aroud 100 bytes. This translates to an average of 800 bytes or 6400 
> bits. In Gigabit Ethernet that would cause a delay of 6400/10^9 sec  or 
> 6.4usec for packet assembly, which is in the same order as Myrinet. 

The best 1 byte latency for GigE I have measured has been 25 us.
This test was using netpipe/TCP. It is hard to provide a solid number
because Ethernet chip-sets/drivers vary as do motherboards that include
GigE. The best thing to do is test some hardware.

> 
> Is this justification correct? If so, how wise is it to use Gigabit 
> Ethernet for an I/O intensive cluster?

More tests are needed to answer that. With Myrinet, Quadrics, SCI, you 
will get better performance -- and spend more money. Somethings you may 
need to consider with this decision:

1. What  API you will use MPI, PVM, sockets? (API can add overhead to 
   latency numbers)
2. How many nodes do expect to use ?
3. Is there a single NFS server for the data or are you using something 
   like PVFS or GFS?
4. What are your I/O block sizes?

Doug
> 
>  
> 
> Regards,
> --
> Mahmoud
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf