Cisco switches for lam mpi

[Gerry Creager N5JXS]

I should have commented earlier, but I didn't think I had time...

My experience with the Cisco 4006 was that as an aggregation switch it 
was OK for 10/100 or GBE.  It did fine for normal "enterprise switching. 
  The 4006's I've used had only older Supervisor Modules and ran CAT-OS, 
rather than IOS like the 4506 I'm testing now.

For higher performance, while CPU utilization stays low, the switch 
falls off at higher loads.

Caveat:  I did not test these devices in a cluster environment; the 
thought never crossed my mind.  I'd be using a 6509 if I had to use a 
Cisco, but I'd probably be shopping for HP ProCurves, Foundry's, 
Riverstones, or NEC Bluefires, based on what I've seen and done lately. 
  I tested the 4006 in normal enterprise mode, and loaded it for 
high-perf network modes.  If you ever need QoS do NOT use a 4006.  Or a 
4506.  They can't handle it too well.  But I digress.

I'm gonna try to get a couple of ProCurves in and test 'em against a LAN 
tester made by Anritsu (MD1230/1231) for small packet capability 
(RFC-2544).  That's been a killer for a lot of switches I've looked at.

gerry

Felix Rauch wrote:
> On Tue, 29 Jul 2003, Jack Douglas wrote:
> 
>>We have just installed a 32 Node Dual Xeon Cluster, with a Cisco Cataslyst
>>4003 Chassis with 48 1000Base-t ports.
>>
>>We are running LAM MPI over gigabit, but we seem to be experiencing
>>bottlenecks within the switch
>>
>>Typically, using the cisco, we only see CPU utilisation of around 30-40%
> 
> [...]
> 
> I'm not a Cisco expert, but...
> 
> We once got a Cisco switch from our networking people that we had to
> return immediately because it delivered such a bad performance. It was
> a Catalyst 2900XL with 24 Fast Ethernet ports, but it could only
> handle 12 ports at full speed. Above that, the performance brake down
> completely.
> 
> For some benchmark results see, e.g.:
> http://www.cs.inf.ethz.ch/~rauch/tmp/FE.Catalyst2900XL.Agg.pdf
> 
> As a comparison, the quite nice results of a CentreCom 742i:
> http://www.cs.inf.ethz.ch/~rauch/tmp/FE.CentreCom742i.Agg.pdf
> 
> Disclaimer: Maybe the Cisco you mentioned is better, or Ciscos improved
> anyway since spring 2001 when I did the above tests. Besides, the
> situation for Gigabit Ethernet could be different.
> 
> As we described on our workshop paper at CAC03 you can not trust the
> data sheets of switches anyway:
> http://www.cs.inf.ethz.ch/CoPs/publications/#cac03
> 
> Conclusion: If you need a very high performing switch, you have to
> evaluate/benchmark it yourself.
> 
> - Felix
> 

-- 
Gerry Creager -- gerry.creager at tamu.edu
Network Engineering -- AATLT, Texas A&M University	
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
Page: 979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf