[Beowulf] What are people seeing performance-wise for NFS over 10GbE

Joe Landman landman at scalableinformatics.com
Thu Feb 28 08:53:36 EST 2008


Hi folks:

   Have a few simple setups that I am trying to figure out if I have a 
problem, or if what I am seeing is normal.

   Two 10 GbE cards, connected with a CX4 cable.  Server is one of our 
JackRabbits, with 750+ MB/s direct IO and 500-650 MB/s buffered IO 
(read-write), for IO about 10x system ram (~100x RAID cache).

   Getting ok iperf numbers, about 7 Gb/s single thread.  Running NFS, 
and seeing ~200-300 MB/s best case between the systems.  Is this what 
others have seen?  Latest drivers from vendor, sent a note to them to 
see if we can figure this out.  Figured I would tap into the 'wulf 
collective memory to learn what other people see.

   2.6.23.14 kernel on both sides, jumbo frames enabled. No switch, just 
a CX4 cable.

   My expectations are that we would be able to see 80-90% of the 
JackRabbits speed, with the rest being eaten by stack issues.  This is 
the case over channel bonded GbE.

   Do you see 500 MB/s or more?  Less?

   Thanks.

Joe

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
        http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

!DSPAM:47c6bdc760241691722815!



More information about the Beowulf mailing list