Gigabit Switch

Donald Becker becker at
Wed Nov 12 10:58:38 EST 2003

On Tue, 11 Nov 2003, Mark Hahn wrote:

> indeed.  I usually take an Occam's apprach to features, too.
> but I was chatting with a big-gbe switch vendor last week,
> and thought of a couple of features which could be useful for HPC:
> 1. suppose you could attach a QOS/TOS tag to small packets, and 
> have the switch give them preferential treatment.  for instance,
> if there's a congested port with a backlog, let small packets 
> "cut" the queue.

QOS/TOS tags already exist, but consider very carefully before you wish
for a LAN switch that observes them.  Using QOS/TOS is a good idea on
multitraffic, multipath WANs, where bulk transfer would otherwise block
telnet-like traffic.  But ACKs bypassing data packets on a LAN will
likely lead to congestion, with higher overall latency and dropped
packets.  This is compounded with now-common flow control, which only
works within the LAN.

> 2. the vendor claims that multicast is reliable.

"Our equipment is perfect, and is not limited by fundamental principles".

>  I've often pondered whether multicast would be worth using in
> clusters, since it's going to be faster than even a tree-based
> multicast (as MPICH/LAM do, I think).

You can construct a set of hardware+protocol+tuning that will work for a
demo.  But we now use Ethernet switches, not repeaters. 
Consider that modern switched Ethernet has many of the same constraints
as Myrinet, SCI, and IB, where multicast must be emulated.

Again, multicast is good for service discovery and low-rate
communication.  Just don't rely on it for bulk data delivery.

> 3. it would be neat to be able to query performance/load/queueing stats
> from the switch on a per-port basis.

You can some of the info from the connected machines.  But you likely
want the time-averaged FIFO length and high-water-mark in packets and
bytes, right?

Donald Becker				becker at
Scyld Computing Corporation
914 Bay Ridge Road, Suite 220		Scyld Beowulf cluster system
Annapolis MD 21403			410-990-9993

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list