Linux 2.4.20 + bonding troubles

Martin Siegert siegert at
Thu Nov 20 13:01:07 EST 2003

Hi Scott,

we ran into the same problem here: the problem is not Linux (as you
mentioned you can set the channels to round-robin under Linux), but
the Cisco switch: you cannot set the Cisco to round-robin mode on
the etherchannel [partially to blame is the IEEE 802.3ad standard,
which does not specify round-robin mode; but that standard was
probably intended for Telco situations (serving many connections
at the same time) instead of HPC situations (aiming at high throughput)].
As a consequence the Cisco will always forward all packets to a single
leg of the outgoing trunk. If your receiving trunk is made out of
100Mbit/s connections this will limit you to 100Mbit/s.

There is not much you can do about this: If all machines that connect
to that network have two NICs, you can create two VLANs on the Cisco
and connect the first of two NICs of each box to VLAN 1 and the second
If you are not in that situation (and we aren't) the only thing that
you can do is to forklift the Cisco out of the way and buy a switch
that supports round-robin mode on etherchannels, e.g., Extreme's Black
Diamond switches.


Martin Siegert
Manager, Research Services
WestGrid Site Manager
Academic Computing Services                        phone: (604) 291-4691
Simon Fraser University                            fax:   (604) 291-4242
Burnaby, British Columbia                          email: siegert at
Canada  V5A 1S6

On Thu, Nov 20, 2003 at 11:05:29AM -0500, Scott Cameron wrote:
> Hi there,
> I'm not sure where to look for information regarding this.
> I've been trying to implement an etherchannel setup for one of my systems
> and have been seeing varied success.  I have the first etherchannel set up
> on 2 Intel Etherexpress 100 cards (e100 driver), it seems to work with
> little problem -- the only issue with this etherchannel I have seen is that
> the channel can not seem go above 100 megabits, while I certainly generate
> enough throughput to go beyond 100 megabits.
> The second etherchannel is on 2 Intel 1000 Mbit cards (e1000 driver).  I've
> had the most trouble with this channel -- when I bring it up, the interface
> begins showing CRC errors & collisions for Tx (not Rx).  However, I don't
> see the collisions on the switch -- just the CRC errors.  
> Both channels are running in load-balancing round-robin mode.  On the switch
> I have the port-channel configured to do source XOR destination IP
> load-balancing.
> The switch I'm connecting to is a Catalyst 6006 running the integrated IOS.
> I can't see any errors in the log on the switch, and not sure how to proceed
> to figure out where the CRC errors are coming from.
> If anyone could point me in the right direction that would be great.
> Scott
> Switch:  Cisco Catalyst 6006 (integrated IOS)
> Linux box:  P3-1.4 GHz, 2.4.20 kernel
> 2x Intel PRO/1000 (driver 5.2.20)
> 4x Intel PRO/100 (driver 2.1.24-k1)
> _______________________________________________
> Beowulf mailing list, Beowulf at
> To change your subscription (digest mode or unsubscribe) visit
Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list