Channel-bonding progress

Mike Weller weller at zyvex.com
Thu Aug 9 02:05:15 EDT 2001


I wrote:
>Hi,
>
>I think I managed to solve the problem.  I noticed that in 
>my "ifconfig" output, there's a lot of "overruns" packets reported.
>
>Also, someone mentioned the possibility of it not arp'ing through
>bond0.
>
>I did 2 things. I'm not sure what works yet, but here are my
>preliminary results.  I can achieve 180-200Mbps in bonding mode if I
>change the MTU packet size from 1500 to 1200.  I also removed the eth1
>and eth0 entries in the routing table to ensure that it uses bond0.
>I'm going to install a better bandwidth benchmark and possibly search
>for the optimal MTU size and post the results.  Here's the script that
>I used to get the 180-200Mbps with FTP:


Using a sniffer, I've determined that the actual packet sizes of eth0
and eth1 used are based on the MTU of bond0.  For instance, if I set
the MTU of eth0 and eth1 to 100, and bond0 to 1000, the packet sizes
sniffed on eth0 are close to 1000.  If I drop it to 500, the packets
on eth0 and eth1 drop to 500.

I also noticed that whenever I made a change to the routing table or
the MTU size of the interface, there is temporary instability, but
for the trials after that, I get consistency (99% confidence).
Can anyone explain the temporary instability?

I conclude that the MTU sizes of eth0 and eth1 are irrelevant, and
that having eth0 and eth1 in the routing table is irrelevant.

Using 2 crossover cables (no switch), along with the command: 
"netperf -H 10.0.0.2 -i 10,1", here's a table of results.  

1100 seems to be the optimal MTU size for bond0.  As the packet size
increases beyond 1100, a large number of packets are "overrun",
resulting in poor bandwidth.   I assume that the bandwidth drops
off below 1100 because more packets are required to be sent (thus,
more overhead).


MTU-bond0 MTU-eth0 MTU-eth1  Transfer Rate(Mbps) Confidence(%)  Overruns (%)
--------  -------- --------  ------------------  -------------  ----------
(keeping eth+eth0 in routing tables:)
  1000      1000   1000         159.53            99                0
  1000      1000   1000         159.66            99                0
(removing eth1+eth0 from routing table:)
  1000      1000   1000          57.79            low               ?
  1000      1000   1000         163.27            99                0
(conclusion: having eth0+eth1 in routing table doesnt hurt)
  1000       100    100         109.80            low               ?
  1000       100    100         163.74            99                0
  1000       100    100         164.98            99                0
  1000       500    500          67.93            ?                 0
  1000       500    500         163.61            99                0
(conclusion: the MTU sizes of eth0 and eth1 don't matter)
...     
  1500       750    750          15.84            40                2 
  1500       750    750          37.14            58                5
  1400       700    700          20.09            22                4
  1400       700    700          20.77            72                6
  1300       650    650          27.73            25                4
  1300       650    650          24.28            46                5
  1200       600    600          39.44            39                3
  1200       600    600          51.19            12                2
  1100       550    550          67.10           148                .07
  1100       550    550         168.75            99                .08
  1100       550    550         168.02            99                .08
  1000       500    500          43.93            29                .001
  1000       500    500         163.63            99                .001
   900       450    450          37.85            64                0!
   900       450    450         156.82            99                0
   800       400    400          25.91            15                0
   800       400    400         126.66            61                0
   800       400    400         140.08             8                0
   800       400    400         142.27            99                0
   700       350    350         101.49            96                0
   600       300    300         109.28            99                0
   400       200    200          76.37            99                0
  1100       550    550         169.03            99                .08
(disconnected crossovers, connected to switch instead)
  1100       550    550         165.42             8                .05
  1100       550    550         166.44             8                .06


-- 
Michael J. Weller, M.Sc.               office: (972) 235-7881 x.242
weller at zyvex.com                         cell: (214) 616-6340
Zyvex Corp., 1321 N Plano           facsimile: (972) 235-7882    
Richardson, TX 75081                      icq: 6180540

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



More information about the Beowulf mailing list