OFED

From UF HPC Wiki

1 Building kernel modules from RPMs
2 Building kernel modules from source
3 Adding Support for New Devices to OFED
4 MVAPICH
5 MVAPICH2
6 OpenMPI
7 MPI Tests

Building kernel modules from RPMs

On production systems, we build the OFED kernel modules (kernel-ib package) from RPMs, via rpmbuild.

The OFED 1.2 build environment does not support building the amso1100 driver, and a patch is required to convince it to do so. Such a patch is available here.

To rebuild the OFED-supplied kernel-ib and kernel-ib-devel packages (from ofa_kernel) outside of the OFED build environment, with OFED-default flags, and with amso1100 support (the src.rpm below is already patched):

[root@test03 ~]# rpmbuild --rebuild --define '_prefix /usr' --define 'configure_options --with-cxgb3-mod --with-ipath_inf-mod --with-ipoib-mod --with-iser-mod  \
  --with-mthca-mod --with-sdp-mod --with-srp-mod --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-rds-mod  \
  --with-vnic-mod --with-amso1100-mod' --define 'KVERSION 2.6.9-55.ELsmp' --define 'KSRC /lib/modules/2.6.9-55.ELsmp/build' --define 'build_kernel_ib 1' \ 
  --define 'build_kernel_ib_devel 1' --define 'NETWORK_CONF_DIR /etc/sysconfig/network-scripts' --define 'modprobe_update 1' --define 'include_ipoib_conf 1' \
  /usr/src/redhat/SRPMS/ofa_kernel-1.2-0.1.src.rpm

Building kernel modules from source

When developing OFED kernel modules, it is most convenient to build from source, starting with the kernel-ib-devel packages that were built via rpmbuild as above.

The steps are similar to the "configure; make; make install" that is typical on UNIX/Linux systems. To configure the ofa_kernel source tree, do the following, where KVERSION is the version of the kernel you are building against (for your running kernel, the output of "uname -r"), KSRC points to your kernel configured kernel source (or header) tree (usually /lib/modules/${KVERSION}/build), and LIB_MOD_DIR is /lib/modules/${KVERSION}/updates/kernel/net:

[root@test03 ~]# cd /usr/src/ofa_kernel-1.2/
[root@test03 ofa_kernel-1.2]# ./configure --prefix=/usr --kernel-version ${KVERSION} --kernel-sources ${KSRC} --modules-dir ${LIB_MOD_DIR} \
  --with-cxgb3-mod --with-ipath_inf-mod --with-ipoib-mod --with-iser-mod --with-mthca-mod --with-sdp-mod --with-srp-mod --with-core-mod \
  --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-rds-mod --with-vnic-mod --with-amso1100-mod

Note that if you have not patched the ofa_kernel sources to support the amso1100, you can skip that configure option.

To build the modules, just do:

[root@test03 ofa_kernel-1.2]# make kernel

Similarly, to install the modules, do:

[root@test03 ofa_kernel-1.2]# make install_kernel

As final steps, we want to make sure the new modules will get loaded upon our next reboot. To do that, examine /etc/modprobe.conf, and make sure the following lines appear:

alias ib0 ib_ipoib
alias ib1 ib_ipoib
alias net-pf-27 ib_sdp

The we need to run depmod:

[root@test03 ofa_kernel-1.2]# depmod -r -ae ${KVERSION}

Finally, make sure the openibd service is chkconfig'd on. When we reboot, we will have our new OFED kernel modules.

Adding Support for New Devices to OFED

This is really jut a matter of adding configuration options and directories into ofa_kernel's configure script and Makefile. The support for amso1100 (used above) was added in by the patch below. Adding support for another source directory would be an identical process: 1) add support for new configuration and Kconfig options, and 2) act on those options in the makefile.

--- ofed_scripts/configure.orig 2007-07-27 19:54:59.000000000 -0400
+++ ofed_scripts/configure      2007-07-27 19:58:42.000000000 -0400
@@ -135,6 +135,12 @@
     --with-cxgb3_debug-mod    make CONFIG_INFINIBAND_CXGB3_DEBUG=y [no]
     --without-cxgb3_debug-mod    [yes]
 
+    --with-amso1100-mod    make CONFIG_INFINIBAND_AMSO1100=m [no]
+    --without-amso1100-mod    [yes]
+
+    --with-amso1100_debug-mod    make CONFIG_INFINIBAND_AMSO1100_DEBUG=y [no]
+    --without-amso1100_debug-mod    [yes]
+
     --help - print out options
 
 
@@ -654,6 +660,18 @@
                         --without-cxgb3_debug-mod)
                             CONFIG_INFINIBAND_CXGB3_DEBUG=
                         ;;
+                        --with-amso1100-mod)
+                            CONFIG_INFINIBAND_AMSO1100="m"
+                        ;;
+                        --without-amso1100-mod)
+                            CONFIG_INFINIBAND_AMSO1100=
+                        ;;
+                        --with-amso1100_debug-mod)
+                            CONFIG_INFINIBAND_AMSO1100_DEBUG="y"
+                        ;;
+                        --without-amso1100_debug-mod)
+                            CONFIG_INFINIBAND_AMSO1100_DEBUG=
+                        ;;
                         --with-modprobe|--without-modprobe)
                         ;;
                         -h | --help)
@@ -723,6 +741,7 @@
 CONFIG_INFINIBAND_VNIC=${CONFIG_INFINIBAND_VNIC:-''}
 CONFIG_INFINIBAND_CXGB3=${CONFIG_INFINIBAND_CXGB3:-''}
 CONFIG_CHELSIO_T3=${CONFIG_CHELSIO_T3:-''}
+CONFIG_INFINIBAND_AMSO1100=${CONFIG_INFINIBAND_AMSO1100:-''}
 
 CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=${CONFIG_INFINIBAND_IPOIB_DEBUG_DATA:-''}
 CONFIG_INFINIBAND_SDP_SEND_ZCOPY=${CONFIG_INFINIBAND_SDP_SEND_ZCOPY:-''}
@@ -734,6 +753,7 @@
 CONFIG_INFINIBAND_VNIC_DEBUG=${CONFIG_INFINIBAND_VNIC_DEBUG:-''}
 CONFIG_INFINIBAND_VNIC_STATS=${CONFIG_INFINIBAND_VNIC_STATS:-''}
 CONFIG_INFINIBAND_CXGB3_DEBUG=${CONFIG_INFINIBAND_CXGB3_DEBUG:-''}
+CONFIG_INFINIBAND_AMSO1100_DEBUG=${CONFIG_INFINIBAND_AMSO1100_DEBUG:-''}
 
         # Check for minimal supported kernel version
         if ! check_kerver ${KVERSION} ${MIN_KVERSION}; then
@@ -793,6 +813,7 @@
 CONFIG_INFINIBAND_VNIC=${CONFIG_INFINIBAND_VNIC}
 CONFIG_INFINIBAND_CXGB3=${CONFIG_INFINIBAND_CXGB3}
 CONFIG_CHELSIO_T3=${CONFIG_CHELSIO_T3}
+CONFIG_INFINIBAND_AMSO1100=${CONFIG_INFINIBAND_AMSO1100}
 
 CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=${CONFIG_INFINIBAND_IPOIB_DEBUG_DATA}
 CONFIG_INFINIBAND_SDP_SEND_ZCOPY=${CONFIG_INFINIBAND_SDP_SEND_ZCOPY}
@@ -804,6 +825,7 @@
 CONFIG_INFINIBAND_VNIC_DEBUG=${CONFIG_INFINIBAND_VNIC_DEBUG}
 CONFIG_INFINIBAND_VNIC_STATS=${CONFIG_INFINIBAND_VNIC_STATS}
 CONFIG_INFINIBAND_CXGB3_DEBUG=${CONFIG_INFINIBAND_CXGB3_DEBUG}
+CONFIG_INFINIBAND_AMSO1100_DEBUG=${CONFIG_INFINIBAND_AMSO1100_DEBUG}
 
 EOFCONFIG
         echo "Created ${CONFIG}:"
@@ -963,6 +985,16 @@
 else
         DEFINE_CHELSIO_T3="#undef CONFIG_CHELSIO_T3"
 fi
+if [ "X${CONFIG_INFINIBAND_AMSO1100}" == "Xm" ]; then
+        DEFINE_INFINIBAND_AMSO1100="#define CONFIG_INFINIBAND_AMSO1100 1"
+else
+        DEFINE_INFINIBAND_AMSO1100="#undef CONFIG_INFINIBAND_AMSO1100"
+fi
+if [ "X${CONFIG_INFINIBAND_AMSO1100_DEBUG}" == "Xy" ]; then
+        DEFINE_INFINIBAND_AMSO1100_DEBUG="#define CONFIG_INFINIBAND_AMSO1100_DEBUG 1"
+else
+        DEFINE_INFINIBAND_AMSO1100_DEBUG="#undef CONFIG_INFINIBAND_AMSO1100_DEBUG"
+fi
 cat >> ${AUTOCONF_H} << EOFAUTOCONF
 #undef CONFIG_INFINIBAND
 #undef CONFIG_INFINIBAND_IPOIB
@@ -990,6 +1022,8 @@
 #undef CONFIG_INFINIBAND_CXGB3
 #undef CONFIG_INFINIBAND_CXGB3_DEBUG
 #undef CONFIG_CHELSIO_T3
+#undef CONFIG_INFINIBAND_AMSO1100
+#undef CONFIG_INFINIBAND_AMSO1100_DEBUG
 
 #undef CONFIG_INFINIBAND_IPOIB_DEBUG_DATA
 #undef CONFIG_INFINIBAND_SDP_SEND_ZCOPY
@@ -1012,6 +1046,7 @@
 ${DEFINE_INFINIBAND_VNIC}
 ${DEFINE_INFINIBAND_CXGB3}
 ${DEFINE_CHELSIO_T3}
+${DEFINE_INFINIBAND_AMSO1100}
 
 ${DEFINE_INFINIBAND_IPOIB_DEBUG}
 ${DEFINE_INFINIBAND_ISER}
@@ -1025,6 +1060,7 @@
 ${DEFINE_INFINIBAND_VNIC_DEBUG}
 ${DEFINE_INFINIBAND_VNIC_STATS}
 ${DEFINE_INFINIBAND_CXGB3_DEBUG}
+${DEFINE_INFINIBAND_AMSO1100_DEBUG}
 
 ${DEFINE_INFINIBAND_IPOIB_DEBUG_DATA}
 ${DEFINE_INFINIBAND_SDP_SEND_ZCOPY}
--- ofed_scripts/makefile.orig  2007-07-27 20:08:56.000000000 -0400
+++ ofed_scripts/makefile       2007-07-27 20:13:32.000000000 -0400
@@ -101,6 +101,8 @@
                CONFIG_CHELSIO_T3=$(CONFIG_CHELSIO_T3) \
                CONFIG_INFINIBAND_CXGB3=$(CONFIG_INFINIBAND_CXGB3) \
                CONFIG_INFINIBAND_CXGB3_DEBUG=$(CONFIG_INFINIBAND_CXGB3_DEBUG) \
+               CONFIG_INFINIBAND_AMSO1100=$(CONFIG_INFINIBAND_AMSO1100) \
+               CONFIG_INFINIBAND_AMSO1100_DEBUG=$(CONFIG_INFINIBAND_AMSO1100_DEBUG) \
                LINUXINCLUDE=' \
                $(BACKPORT_INCLUDES) \
                -I$(CWD)/include \

MVAPICH

Building OFED-supplied MVAPICH with a variety of compilers:

By default, MVAPICH will use rsh or remsh asn the RSHCOMMAND - we need it to use ssh. We force that by patching the mvapich spec file like so:

[root@submit SPECS]# diff -u mvapich.spec.orig mvapich.spec
--- mvapich.spec.orig   2008-01-15 10:32:49.000000000 -0500
+++ mvapich.spec        2008-01-15 10:33:27.000000000 -0500
@@ -276,7 +276,7 @@
 export F90FLAGS="$F90FLAGS $EXTRA_CFLAG"
 export CONFIG_FLAGS
 export MPIRUN_CFLAGS="$MPIRUN_CFLAGS -DPARAM_GLOBAL=\\\"%{_prefix}/etc/$conffile\\\" -DLD_LIBRARY_PATH_MPI=\\\"%{_prefix}/lib/shared\\\" $DEF_BUILDID"
-./configure --enable-sharedlib --with-device=%{mpi_device} --with-arch=LINUX --prefix=%{build_root}%{_prefix} $CONFIG_ENABLE_F77 $CONFIG_ENABLE_F90 $COMPILER_CONFIG -lib="-L$IB_LIB -libverbs -libumad -libcommon -lpthread $EXTRA_CFLAG" $MPE_FLAGS $CONFIG_FLAGS 
+./configure --enable-sharedlib --with-device=%{mpi_device} --with-arch=LINUX --prefix=%{build_root}%{_prefix} $CONFIG_ENABLE_F77 $CONFIG_ENABLE_F90 $COMPILER_CONFIG -lib="-L$IB_LIB -libverbs -libumad -libcommon -lpthread $EXTRA_CFLAG" $MPE_FLAGS $CONFIG_FLAGS -rsh=ssh
 %{__make}
 
 #############################################################################
@@ -352,6 +352,8 @@
 #
 #############################################################################
 %changelog
+* Tue Jan 15 2008 Craig Prescott <prescott@hpc.ufl.edu>
+- Add -rsh=ssh configure option to force use of ssh.
 * Wed Jun  6 2007 Pavel Shamis <pasha@mellanox.co.il>
 - Fixed PGI build bug. PGI doesn't support -Wall flag.
 * Sun Mar 25 2007 Pavel Shamis <pasha@mellanox.co.il>

The OFED build script is just fine for creating MVAPICH packages with the Intel compilers. The GNU packages, however, get a bit of tweaking more for sanity's sake than anything else.

Intel compilers

For reference, here is how to rebuild OFED-supplied MVAPICH packages with Intel compilers outside of the OFED build environment:

rpmbuild --rebuild --define '_name mvapich_intel' --define 'ofed 1' --define 'compiler intel' --define 'openib_prefix /usr' --define 'mpi_selector /usr/bin/mpi-selector' --define '_prefix /usr/mpi/intel/mvapich-0.9.9' /dist/hpc/ofed/OFED-1.2/SRPMS/mvapich-0.9.9-1326.src.rpm

There is only one minor nit with the Intel MVAPICH package - the '-i-dynamic' flag is missing from the linker flag in the wrappers (mpiCC, mpicc, mpif77, mpif90). This is known to cause problems with feupdateenv. The easy thing to do to solve this is to set LDFLAGSBASE="-i-dynamic" in each wrapper, either by hand or by hacking the mvapich.spec file and setting LDFLAGS="-i-dynamic" in the Intel compiler option block. There is no way to set this from the rpmbuild command line.

GNU compilers (gcc, g++, g77)

If gfortran is available on the build system, the MVAPICH spec file will use 'gfortran' from GCC version 4 with 'gcc' and 'g++' from GCC version 3 when creating MVAPICH packages. This is a strange thing to do, IMHO. We want to use g77 with GCC version 3 gcc and g++. Fortran90 bindings for this combo will be unavailable. Unfortunately, there is no good way to do that other than to 'hide' the gfortran program while building the package:

mv /usr/bin/gfortran /usr/bin/gfortran.hide rpmbuild --rebuild --define '_name mvapich_gcc' --define 'ofed 1' --define 'compiler gcc' --define 'openib_prefix /usr' --define 'mpi_selector /usr/bin/mpi-selector' --define '_prefix /usr/mpi/gcc/mvapich-0.9.9' /dist/hpc/ofed/OFED-1.2/SRPMS/mvapich-0.9.9-1326.src.rpm mv /usr/bin/gfortran.hide /usr/bin/gfortran

MVAPICH2

Building OFED-supplied MVAPICH2 with a variety of compilers:

By default, the OFED build script is just fine for creating MVAPICH2 packages with Intel compilers. The GNU packages, however, get a bit of tweaking more for sanity's sake than anything else.

Intel compilers

For reference, here is how to rebuild OFED-supplied MVAPICH2 packages with Intel compilers outside of the OFED build environment:

rpmbuild --rebuild --define '_prefix /usr/mpi/intel/mvapich2-0.9.8-12' --define '_name mvapich2_intel' --define 'impl ofa' --define 'multithread 1' --define 'romio 1' --define 'shared_libs 1' --define 'rdma_cm 1' --define 'ckpt 0' --define 'open_ib_home /usr' --define 'comp_env CC="icc -i-dynamic" CXX="icpc -i-dynamic" F77="ifort -i-dynamic" F90="ifort -i-dynamic"' --define 'auto_req 0' --define 'mpi_selector /usr/bin/mpi-selector' /dist/hpc/ofed/OFED-1.2/SRPMS/mvapich2-0.9.8-12.src.rpm

GNU compilers (gcc, g++, g77)

If gfortran is available on the build system, the MVAPICH2 spec file will use 'gfortran' from GCC version 4 with 'gcc' and 'g++' from GCC version 3 when creating MVAPICH2 packages. This is a strange thing to do, IMHO. We want to use g77 with GCC version 3 gcc and g++. Fortran90 bindings for this combo will be unavailable.

rpmbuild --rebuild --define '_prefix /usr/mpi/gcc/mvapich2-0.9.8-12' --define '_name mvapich2_gcc' --define 'impl ofa' --define 'multithread 1' --define 'romio 1' --define 'shared_libs 1' --define 'rdma_cm 1' --define 'ckpt 0' --define 'open_ib_home /usr' --define 'comp_env CC=gcc CXX=g++ F77=g77 F90=g77' --define 'auto_req 0' --define 'mpi_selector /usr/bin/mpi-selector' /dist/hpc/ofed/OFED-1.2/SRPMS/mvapich2-0.9.8-12.src.rpm

GNU compilers (gcc4, g++4, gfortran)

By default, the OFED build script will not create an OpenMPI package built with GCC version 4 compilers. But I thought it would be fun to have one to play with!

rpmbuild --rebuild --define '_prefix /usr/mpi/gcc4/mvapich2-0.9.8-12' --define '_name mvapich2_gcc4' --define 'impl ofa' --define 'multithread 1' --define 'romio 1' --define 'shared_libs 1' --define 'rdma_cm 1' --define 'ckpt 0' --define 'open_ib_home /usr' --define 'comp_env CC=gcc4 CXX=g++4 F77=gfortran F90=gfortran' --define 'auto_req 0' --define 'mpi_selector /usr/bin/mpi-selector' /dist/hpc/ofed/OFED-1.2/SRPMS/mvapich2-0.9.8-12.src.rpm

OpenMPI

Building OFED-supplied OpenMPI with a variety of compilers:

OpenMPI has a very handy Task Management (aka 'tm') interface for interfacing with batch systems. Unfortunately, the OpenMPI packages as build from the RPMs in the OFED 1.2 distribution do not enable this interface. For this reason, and to diddle the compiler flags, we need to build the OpenMPI packages ourselves.

Intel compilers

By default, the OFED build script will not build OpenMPI with a Task Management interface; the '--with-tm' option is not given at configure time. So we need to add it in ourselves. Also, the '-m64' will be added to CFLAGS, CXXFLAGS, FFLAGS, and FCFLAGS; this causes harmless but annoying warnings from the underlying Intel compilers saying that they don't understand the '-m64' flag. So we will override CFLAGS, CXXFLAGS, FFLAGS, and FCFLAGS to remove the '-m64' flag. Finally, we want to add the '-i-dynamic' flag to each compiler to preempt problems with feupdatenv.

rpmbuild --rebuild --define '_name openmpi_intel' --define '_prefix /usr/mpi/intel/openmpi-1.2.2-1' --define '_mandir %{_prefix}/share/man' --define '_defaultdocdir /usr/mpi/intel/openmpi-1.2.2-1' --define 'configure_options LDFLAGS=\"-Wl,-R/usr/lib64\" --with-openib=/usr --with-openib-libdir=/usr/lib64 CC=icc CXX=icpc F77=ifort FC=ifort CFLAGS=\"-DNDEBUG -O2 -g -pipe -finline-functions -fno-strict-aliasing -restrict -pthread -i-dynamic\" CXXFLAGS=\"-DNDEBUG -O2 -g -pipe -finline-functions -pthread -i-dynamic\" FFLAGS=\"-O2 -g -pipe -i-dynamic\" FCFLAGS=\"-O2 -g -pipe -i-dynamic\" --with-tm --with-wrapper-ldflags="-Wl,-rpath -Wl,/usr/mpi/intel/openmpi-1.2.2-1/lib64" --enable-mpirun-prefix-by-default' --define 'mflags -j 4' --define 'mpi_selector /usr/bin/mpi-selector' --define 'ofed 1' --define 'use_default_rpm_opt_flags 1 ' /dist/hpc/ofed/OFED-1.2/SRPMS/openmpi-1.2.2-1.src.rpm

GNU compilers (gcc, g++, g77)

By default, the OFED build script will not build OpenMPI with a Task Management interface; the '--with-tm' option is not given at configure time. So we need to add it in ourselves. Also, if you have GCC version 3 and version 4 compilers installed in the build host, configure will try to use 'gfortran' from GCC version 4 with 'gcc' and 'g++' from GCC version 3. We need to use g77 with GCC version 3, and disable Fortran90 bindings.

rpmbuild --rebuild --define '_name openmpi_gcc' --define '_prefix /usr/mpi/gcc/openmpi-1.2.2-1' --define '_mandir %{_prefix}/share/man' --define '_defaultdocdir /usr/mpi/gcc/openmpi-1.2.2-1' --define 'configure_options LDFLAGS=\"-Wl,-R/usr/lib64\" --with-openib=/usr --with-openib-libdir=/usr/lib64 CC=gcc CXX=g++ F77=g77 FC=g77 --with-tm --with-wrapper-ldflags="-Wl,-rpath -Wl,/usr/mpi/gcc/openmpi-1.2.2-1/lib64" --disable-mpi-f90 --enable-mpirun-prefix-by-default' --define 'mflags -j 4' --define 'mpi_selector /usr/bin/mpi-selector' --define 'ofed 1' --define 'use_default_rpm_opt_flags 1 ' /dist/hpc/ofed/OFED-1.2/SRPMS/openmpi-1.2.2-1.src.rpm

GNU compilers Version 4 (gcc, g++, gfortran)

By default, the OFED build script will not create an OpenMPI package built with GCC version 4 compilers. But I thought it would be fun to have one to play with!

rpmbuild --rebuild --define '_name openmpi_gcc4' --define '_prefix /usr/mpi/gcc4/openmpi-1.2.2-1' --define '_mandir %{_prefix}/share/man' --define '_defaultdocdir /usr/mpi/gcc4/openmpi-1.2.2-1' --define 'configure_options LDFLAGS=\"-Wl,-R/usr/lib64\" --with-openib=/usr --with-openib-libdir=/usr/lib64 CC=gcc4 CXX=g++4 F77=gfortran FC=gfortran --with-tm --with-wrapper-ldflags="-Wl,-rpath -Wl,/usr/mpi/gcc4/openmpi-1.2.2-1/lib64" --enable-mpirun-prefix-by-default' --define 'mflags -j 4' --define 'mpi_selector /usr/bin/mpi-selector' --define 'ofed 1' --define 'use_default_rpm_opt_flags 1 ' /dist/hpc/ofed/OFED-1.2/SRPMS/openmpi-1.2.2-1.src.rpm

MPI Tests

Once we've rebuilt and deployed all the MPI implementations with all our compilers, we can rebuild the 'mpitests' packages corresponding to each.

MVAPICH

rpmbuild --rebuild --define '_name mpitests_mvapich_intel' --define 'path_to_mpihome /usr/mpi/intel/mvapich-0.9.9' --define 'root_path /' --define 'stack_prefix /usr' /dist/hpc/ofed/OFED-1.2/SRPMS/mpitests-2.0-705.src.rpm rpmbuild --rebuild --define '_name mpitests_mvapich_gcc' --define 'path_to_mpihome /usr/mpi/gcc/mvapich-0.9.9' --define 'root_path /' --define 'stack_prefix /usr' /dist/hpc/ofed/OFED-1.2/SRPMS/mpitests-2.0-705.src.rpm

MVAPICH2

rpmbuild --rebuild --define '_name mpitests_mvapich2_intel' --define 'path_to_mpihome /usr/mpi/intel/mvapich2-0.9.8-12' --define 'root_path /' --define 'stack_prefix /usr' /dist/hpc/ofed/OFED-1.2/SRPMS/mpitests-2.0-705.src.rpm rpmbuild --rebuild --define '_name mpitests_mvapich2_gcc' --define 'path_to_mpihome /usr/mpi/gcc/mvapich2-0.9.8-12' --define 'root_path /' --define 'stack_prefix /usr' /dist/hpc/ofed/OFED-1.2/SRPMS/mpitests-2.0-705.src.rpm rpmbuild --rebuild --define '_name mpitests_mvapich2_gcc4' --define 'path_to_mpihome /usr/mpi/gcc4/mvapich2-0.9.8-12' --define 'root_path /' --define 'stack_prefix /usr' /dist/hpc/ofed/OFED-1.2/SRPMS/mpitests-2.0-705.src.rpm

OpenMPI

rpmbuild --rebuild --define '_name mpitests_openmpi_intel' --define 'path_to_mpihome /usr/mpi/intel/openmpi-1.2.2-1' --define 'root_path /' --define 'stack_prefix /usr' /dist/hpc/ofed/OFED-1.2/SRPMS/mpitests-2.0-705.src.rpm rpmbuild --rebuild --define '_name mpitests_openmpi_gcc' --define 'path_to_mpihome /usr/mpi/gcc/openmpi-1.2.2-1' --define 'root_path /' --define 'stack_prefix /usr' /dist/hpc/ofed/OFED-1.2/SRPMS/mpitests-2.0-705.src.rpm rpmbuild --rebuild --define '_name mpitests_openmpi_gcc4' --define 'path_to_mpihome /usr/mpi/gcc4/openmpi-1.2.2-1' --define 'root_path /' --define 'stack_prefix /usr' /dist/hpc/ofed/OFED-1.2/SRPMS/mpitests-2.0-705.src.rpm

Retrieved from "http://wiki.hpc.ufl.edu/index.php/OFED"

OFED

From UF HPC Wiki

Contents

Building kernel modules from RPMs

Building kernel modules from source

Adding Support for New Devices to OFED

MVAPICH

MVAPICH2

OpenMPI

MPI Tests

Views

Personal tools

Navigation

Search

Toolbox