A question of OS!!

Bill Broadley bill at math.ucdavis.edu
Thu Sep 25 20:45:34 EDT 2003

>   Yikes.. what kernels are used on these systems by default, and how large
> is the code?  I've been running SuSE 8.2 Pro on my nodes, and have gotten 

Factory default in both cases AFAIK.  I don't have access to the SLES
system at the moment, but the redhat box is:

Linux foo.math.ucdavis.edu 2.4.21-1.1931.2.349.2.2.entsmp #1 SMP Fri Jul 18
00:06:19 EDT 2003 x86_64 x86_64 x86_64 GNU/Linux

What relationship that has to the original 2.4.21 I know not.

> varying performance due to motherboard, BIOS level and kernel.  (SuSE 8.2 
> Pro comes a modified 2.4.19, but I've also run 2.6.0-test5)

>   Also, are the BIOS settings the same?  And how are the RAM slots 

I don't have access to the SLES bios.

> populated?  That made a difference, too!

I'm well aware of the RAM slot issues, and I've experimentally verified
that the full bandwidth is available.  Basically each cpu will see 2GB/sec
or so to main memory, and both see a total of 3GB/sec if both use
memory simultaneously.

>   (Oh, and I imagine they're both writing to a local disk, or minimal 
> amounts over NFS?  That could play a big part, too.. )

Yeah, both local disk, and not much.  I didn't notice any difference when
I commented out all output.

>   I should have some numbers at some point for how much things vary, but 
> at the moment we've been pretty busy on our systems.  Any more info on 
> this would be great, though, since I've been looking at the faster chips, 
> too!

ACK, I never considered that the opterons might be slower in some ways
at faster clock speeds.  My main suspicious is that MPICH was messaging
passing for local nodes in some strange way and triggering some corner
case under SLES.  I.e. writing an int at a time between CPUs who are
fighting over the same page.

None of my other MPI benchmarks for latency of bandwidth (at various
message sizes) have found any sign of problems.  Numerous recompiles
of MPICH haven't had any effect either.

Bill Broadley
UC Davis
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list