OpenMPI

From UF HPC Wiki

*NOTE* Executables built against OpenMPI can run over either IB or GigE. They prefer and will default to InfiniBand if the interfaces are detected and up. In the absence of an IB interconnect, OpenMPI apps will use GigE. When that happens you will see a message saying something like the following

--------------------------------------------------------------------------
[0,1,1]: OpenIB on host r6b-s10.ufhpc was unable to find any HCAs.
Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------

Unfortunately, recent OpenMPI releases have a bug that causes this fail over to not work.

To work around this, please make your MPIRUN command this:

mpirun --mca btl ^udapl,openib --mca btl_tcp_if_include eth0

This will cause OpenMPI to ignore IB, and use TCP over ethernet.

Versions Available

Version	ABI	Compiler	Location	Notes
1.2.7	64 bit	Intel v10	/usr/mpi/intel/openmpi-1.2.7-1	For Torque/Maui

Compiling Programs

• Use the OpenMPI compiler wrappers to build your codes. The compiler wrappers don't actually compile anything on their own; they invoke other compilers installed on the system. You can use the ompi_info command to see exactly what underlying compilers are used by the wrappers.

• Use the wrappers just as you would normal compilers; for example:

mpicc prog.c -o prog

• In your Makefiles, it is a good practice to *avoid* using full paths to the compiler installation areas wherever possible. It is easy to get into trouble this way; unless you are careful, you can end up with paths in your makefiles that are inconsistent with your MPI environment.

• Below is a table showing the name of the OpenMPI compiler wrapper that corresponds to popular programming languages:

Language	Compiler wrapper name
C	mpicc
C++	mpiCC
Fortran 77	mpif77
Fortran 90	mpif90

• For more information, please see the OpenMPI FAQ on Compiling MPI applications