MPI & linux compilers

Javier Crespo javier.crespo at itp.es
Mon Aug 4 02:53:05 EDT 2003


Levente Horvath wrote:

> To whom it may concern,
>
> We have 12 PCs set up for parallel computation. All are running linux
> (Redhat 7.3) and MPI.
> We would like to compute eigenvalues and eigenvectors for large matrices.
>
> We have managed to do up to 10000x10000 matrix no problem. Our program uses
> Scalapack and Blacs
> routines. These routines require two matrix to be declared. On single
> precision two 10000x10000
> matrix occupies 800Mb of memory which is already exceeds the 512Mb local
> memory of
> each computer in our cluster. This memory were equally distributed over the
> 12 computers
> upon computation. So, we think that in theory we shouldn't have any problem
> going
> to large matrices; as our distributed memory is quite large 12*512Mb.
>
> Now, if we try to run a larger size then the compiler mpif77 returns
> a "large matrix" error. We have traced the compiler and found that mpif77 is
> a script
> that calls up f77 and mpi libraries. Upon replacing the f77 with g77-3, we
> found that
> there is no problem with the compilation up to a size of 15000x15000, then
> the
> compiler crashes. After tracing the compilation procedure, we found that
> the linker "as" cannot link some of the .o and .s files in our /tmp
> directory.
>
> So, we used C rather than fortran. Statically, we cannot declare more than
> a 1500x1500 matrix (that put in to a hello world program for MPI). We
> thought
> it might be the problem with the static allocation of memory. So, we tried
> to allocate this space dynamically without any success....
>
> Our questions are: Are we doing something wrong here. Or are the compilers
> gcc and g77-3
> responsible for such an array limit. Or are we missing the ways to allocate
> memory for large matrices....
>
> This is not the end of our story. We tried "ifc" IBM fortran 90 compiler.
> Unfortunately, we
> cannot link mpi libraries against this "ifc" compiler. It just doesn't see
> them. We have
> tried to compile ifc with the full path names of libraries using either
> static and dynamics libraries.
> In either case we had no success...
>
> We would appreciate all of your comments and suggestions.
> Thank you in advance....

If you want to link to mpi but compiling with "ifc" (is it really IBM? - I think it comes from intel), you first
at all should have to compile that libraries with the same compiler that you are going to use for the main
program, typically using the options "-fc=ifc","--f90=ifc" and "-f90linker=ifc" when configuring MPI and then
installing it in you path (in a different place than the MPI libraries compiled with f77).

Javier


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



More information about the Beowulf mailing list