[Beowulf] compilers vs mpi?

Mark Hahn hahn at mcmaster.ca
Tue Jul 20 14:25:32 EDT 2010

> we do exactly what you describe: compile OpenMPI with the gcc suite
> and then use it with gcc, intel and open64 compilers.


> This works out-of-the-box, almost.
> The problem is the f90 module mpi.mod. This is (usually) a binary
> file and specific to the f90 compiler that was used to compile
> OpenMPI. But there is a way to solve this problem:

ah, .mod files.  we have for the most part ignored them entirely.
(what are we losing by doing that?)

> 1. compile openmpi using the gcc compilers, i.e., gfortran as the Fortran
>   compiler and install it in /usr/local/openmpi

this is perhaps a tangent, but we install everything we support
under /opt/sharcnet/$packagebasename/$ver.  for openmpi, we've had 
to bodge the compiler flavor onto that (/opt/sharcnet/openmpi/1.4.2/intel).

> 2. move the Fortran module to the directory
>   /usr/local/openmpi/include/gfortran. In that directory
>   create softlinks to the files in /usr/local/openmpi/include.
> 3. compile openmpi using ifort and install the Fortran module (and only
>   the Fortran module!) in /usr/local/openmpi/include/intel. In that
>   directory create softlinks to the files in /usr/local/openmpi/include.

I guess I'm surprised this works - aren't you effectively assuming that 
the intel and gfortran interfaces are compatible here?  that is, the app
compiles with the compiler-specific module, which basically promises 
a particular type-safe interface (signature) for MPI functions, but 
then the linker just glues them together without any way to verify 
the signature compatibility...

am I misunderstanding?

> 4. in /usr/local/openmpi/bin create softlinks mpif90.ifort
>   and mpif90.gfortran pointing to opal_wrapper. Remove the
>   mpif90 softlink.
> 5. Move /usr/local/openmpi/share/openmpi/mpif90-wrapper-data.txt
>   to /usr/local/openmpi/share/openmpi/mpif90.ifort-wrapper-data.txt.
>   Change the line includedir=${includedir} to:
>   includedir=${includedir}/intel
>   Copy the file to
>   /usr/local/openmpi/share/openmpi/mpif90.gfortran-wrapper-data.txt
>   and change the line includedir=${includedir} to
>   includedir=${includedir}/gfortran
> 6. Create a wrapper script /usr/local/openmpi/bin/mpif90:
> #!/bin/bash
> OMPI_WRAPPER_FC=`basename $OMPI_FC 2> /dev/null`
> if [ "$OMPI_WRAPPER_FC" = 'gfortran' ]; then
>   exec $0.gfortran "$@"
> else
>   exec $0.ifort "$@"
> fi

this is a tangent, but perhaps interesting.  we don't use the wrappers 
from the MPI package, but rather our own single wrapper which has some
built-in intelligence (augmented by info from the compiler's (environment)

> The reason we use gfortran in step 1 is that otherwise you get those
> irritating error messages from the Intel libraries, cf.
> http://www.open-mpi.org/faq/?category=building#intel-compiler-wrapper-compiler-warnings

hmm.  we work around those by manipulating the link arguments in our wrapper.

Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

More information about the Beowulf mailing list