MPI & linux compilers
leventeh at hotmail.com
Sun Aug 3 02:14:23 EDT 2003
To whom it may concern,
We have 12 PCs set up for parallel computation. All are running linux
(Redhat 7.3) and MPI.
We would like to compute eigenvalues and eigenvectors for large matrices.
We have managed to do up to 10000x10000 matrix no problem. Our program uses
Scalapack and Blacs
routines. These routines require two matrix to be declared. On single
precision two 10000x10000
matrix occupies 800Mb of memory which is already exceeds the 512Mb local
each computer in our cluster. This memory were equally distributed over the
upon computation. So, we think that in theory we shouldn't have any problem
to large matrices; as our distributed memory is quite large 12*512Mb.
Now, if we try to run a larger size then the compiler mpif77 returns
a "large matrix" error. We have traced the compiler and found that mpif77 is
that calls up f77 and mpi libraries. Upon replacing the f77 with g77-3, we
there is no problem with the compilation up to a size of 15000x15000, then
compiler crashes. After tracing the compilation procedure, we found that
the linker "as" cannot link some of the .o and .s files in our /tmp
So, we used C rather than fortran. Statically, we cannot declare more than
a 1500x1500 matrix (that put in to a hello world program for MPI). We
it might be the problem with the static allocation of memory. So, we tried
to allocate this space dynamically without any success....
Our questions are: Are we doing something wrong here. Or are the compilers
gcc and g77-3
responsible for such an array limit. Or are we missing the ways to allocate
memory for large matrices....
This is not the end of our story. We tried "ifc" IBM fortran 90 compiler.
cannot link mpi libraries against this "ifc" compiler. It just doesn't see
them. We have
tried to compile ifc with the full path names of libraries using either
static and dynamics libraries.
In either case we had no success...
We would appreciate all of your comments and suggestions.
Thank you in advance....
ninemsn Extra Storage comes with McAfee Virus Scanning - to keep your
Hotmail account and PC safe. Click here http://join.msn.com/
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf