Amber
From UF HPC Wiki
Contents |
Installations
The UF HPC Center has a number of different versions of Amber installed. Users can use any of the following versions:
| Version | Compiler | MPI Version | Queue/TM API | AMBERHOME |
|---|---|---|---|---|
| 10 | Intel | OpenMPI | Torque/OpenPBS | /apps/amber/10/intel/ompi-1.2.7 |
| 10 | Intel | MVAPich | Torque/OpenPBS | /apps/amber/10/intel/mvapich-0.9.9 |
| 9 | GNU | OpenMPI | Torque/OpenPBS | /apps/amber/9/gnu/ompi |
| 9 | Intel | OpenMPI | Torque/OpenPBS | /apps/amber/9/intel/ompi/1.2.2 |
| 8 | Intel | OpenMPI | Torque/OpenPBS | /apps/amber/8 |
Basic Execution Instructions
Let us say you want to use the latest version of Amber, in this case version 10. You would need to do the following:
- Login and run the following command:
mpi-selector --set mvapich_intel10-0.9.9
- Logout and log back in again
- Set your AMBERHOME environment variable to the proper location:
export AMBERHOME=/apps/amber/10/intel/mvapich-0.9.9
- Write a job script similar to the one on the Amber Scripts page. Note that you will need to change things like input files, email addresses, etc. in order for one of these scripts to work for you.
FAQ
How do I use the leap program?
There is a graphic interface in the AMBER package to refine the atomic structure, which is "xleap". There is also a text-based version, called "tleap". When logged into the HPC Center, we recommend the use of the "tleap" program rather than "xleap".
How do I set the AMBERHOME environment variable?
Answer: Say you are using the Intel + OMPI version of Amber 9. Your base AMBERHOME variable would be the following:
/apps/amber/9/intel/ompi
As such, you want to set your bash environment variable through the following command:
export AMBERHOME=/apps/amber/9/intel/ompi
Now, what I am going to do, since this is something that you would have to do every time you login, is add it to one of your start-up scripts so that it is done automatically (Note the last two lines):
[username@submit ~]$ cat /home/username/.bashrc
# .bashrc
# User specific aliases and functions
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# Set Amber home directory
export AMBERHOME=/apps/amber/9/intel/ompi
Version 9 (Intel + MKL)
AMBER version 9 has been built using the Intel Compiler and MKL. This version will work with the newer Torque (OpenPBS) queues. Serial and multi-threaded (OpenMPI) executables are available.
Build Date: 04/27/07 Compiler: Intel 9.1 Libraries: MKL 8.1 (/opt/intel/mkl/8.1/lib/em64t) Location: /apps/amber/9/intel/ompi/1.2.2
Sample Torque/OpenPBS submission script:
The sample PBS script can now be found on the Amber Scripts page.
Version 9 (Patches, Intel 10, MKL 10)
Build Date: 03/02/09 Compiler: Intel 10.1.015 Libraries: MKL 10.0.2.018 MPI: OpenMPI 1.2.7-1 Location: /apps/amber/9/intel/ompi/exe.patched
This executable fixes a performance problem found on Intel-based quad-core nodes (Phase III) when the MPI version of AMBER is linked against either libguide or libiomp5. When linked using these libraries and the MKL liblapack, you end up with multiple OMP threads for each MPI thread during execution. This severely degrades (by a factor of 2 or more) peformance on Intel quad-core systems.
To run this executable version of AMBER you should be sure that your environment is set up properly using mpi-selector. You should run
mpi-selector --set openmpi_intel-1.2.7
Be sure and log out and log back in so that the environment variable changes take effect.
AMBER config.h
The config.h file for the new executable is below.
#============================================================================== # AMBER Makefile configuration for compiler/architecture: ifort_x86_64 # Generated via command: ./configure -openmpi ifort_x86_64 # # Configuration script written mainly by Joe Krahn, Scott Brozell, and # Dave Case, with contributions from lots of people. #============================================================================== #------------------------------------------------------------------------------ # Main AMBER source root directory #------------------------------------------------------------------------------ AMBER_SRC=/home/taylor/Amber/amber9/src #------------------------------------------------------------------------------ # AMBERBUILDFLAGS provides a hook into the build process for installers; # for example, to build debug versions of the amber programs # make -e AMBERBUILDFLAGS="-DDEBUG -g" #------------------------------------------------------------------------------ AMBERBUILDFLAGS= #------------------------------------------------------------------------------ # LOCALFLAGS is intended for program specific modifications to the # Fortran build process and may be modified by the program's local makefile #------------------------------------------------------------------------------ LOCALFLAGS= #------------------------------------------------------------------------------ # Availability and method of delivery of math and optional libraries #------------------------------------------------------------------------------ USE_BLASLIB=$(VENDOR_SUPPLIED) USE_LAPACKLIB=$(VENDOR_SUPPLIED) USE_LMODLIB=$(LMOD_UNAVAILABLE) #------------------------------------------------------------------------------ # C compiler #------------------------------------------------------------------------------ CC= mpicc CPLUSPLUS=mpiCC CFLAGS= -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -O2 CPPFLAGS= $(AMBERBUILDFLAGS) #------------------------------------------------------------------------------ # Fortran preprocessing and compiler. # FPPFLAGS holds the main Fortran options, such as whether MPI is used. #------------------------------------------------------------------------------ FPPFLAGS = -I/usr/mpi/intel/openmpi-1.2.7-1/include -P -DMPI -DMKL $(AMBERBUILDFLAGS) FPP = cpp -traditional $(FPPFLAGS) FC = mpif90 FFLAGS = -w95 -mp1 -O0 $(LOCALFLAGS) $(AMBERBUILDFLAGS) FOPTFLAGS = -w95 -mp1 -ip -O3 -axW $(LOCALFLAGS) $(AMBERBUILDFLAGS) FREEFORMAT_FLAG = -free #------------------------------------------------------------------------------ # Loader: #------------------------------------------------------------------------------ LOAD = mpif90 -i_dynamic $(LOCALFLAGS) $(AMBERBUILDFLAGS) LOADCC = mpicc $(LOCALFLAGS) $(AMBERBUILDFLAGS) MKLDIR = -L/opt/intel/mkl/10.0.2.018/lib/em64t MKLLIBS = -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core LOADLIB = $(MKLDIR) $(MKLLIBS) LM= -lm LOADPTRAJ = mpif90 -nofor_main $(LOCALFLAGS) $(AMBERBUILDFLAGS) XHOME = /usr/X11R6 XLIBS = -L/usr/X11R6/lib64 #------------------------------------------------------------------------------ # Other stuff: #------------------------------------------------------------------------------ .SUFFIXES: .f90 EMPTY= AR=ar rv $(EMPTY) M4=m4 RANLIB=ranlib SFX= NETCDF= NETCDFLIB= MODULEDIR=-I MAKEDEPEND=$(AMBER_SRC)/../bin/amber_makedepend SLKO=skipDFTB # default rules for Fortran and C compilation: .f.o: $< $(FPP) $< > _$< $(FC) -c $(FFLAGS) -o $@ _$< .c.o: $(CC) -c $(CFLAGS) $(CPPFLAGS) -o $@ $<
Version 10 (Intel 10, MKL)
Two sets of AMBER 10 executables have been installed. One set is built using OpenMPI 1.2.7 while the other uses MVAPICH 0.9.9. The executable directory paths are /apps/amber/10/intel/ompi-1.2.7/exe and /apps/amber/10/intel/mvapich-0.9.9/exe. In both cases the Intel 10 compiler suite was used along with MKL. Note that whichever executable you use, you must set your MPI environment appropriatetely (see mpi-selector).
Build Date: 03/02/09 Compiler: Intel 10.1.015 Libraries: MKL 10.0.2.018 MPI: OpenMPI 1.2.7-1 Location: /apps/amber/10/intel/ompi-1.2.7/exe
Build Date: 03/02/09 Compiler: Intel 10.1.015 Libraries: MKL 10.0.2.018 MPI: OpenMPI MVAPICH 0.9.9 Location: /apps/amber/9/intel/mvapich-0.9.9/exe
config_amber.h
The slightly modified (relative to what is produced by configure_amber) configure_amber.h file is below.
#============================================================================== # AMBER Makefile configuration for compiler/architecture: ifort_x86_64 # Generated via command: ./configure_amber -openmpi ifort # # Configuration script written mainly by Joe Krahn, Scott Brozell, and # Dave Case, with contributions from lots of people. #============================================================================== #------------------------------------------------------------------------------ # AMBERBUILDFLAGS provides a hook into the build process for installers; # for example, to build debug versions of the amber programs # make -e AMBERBUILDFLAGS="-DDEBUG -g" #------------------------------------------------------------------------------ AMBERBUILDFLAGS= #------------------------------------------------------------------------------ # LOCALFLAGS is intended for program specific modifications to the # Fortran build process and may be modified by the program's local makefile #------------------------------------------------------------------------------ LOCALFLAGS= #------------------------------------------------------------------------------ # Availability and method of delivery of math and optional libraries #------------------------------------------------------------------------------ USE_BLASLIB=$(VENDOR_SUPPLIED) USE_LAPACKLIB=$(VENDOR_SUPPLIED) #------------------------------------------------------------------------------ # C compiler #------------------------------------------------------------------------------ CC= mpicc CPLUSPLUS=mpiCC CFLAGS= -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -m64 -O2 $(AMBERBUILDFLAGS) CPPFLAGS= -DBINTRAJ -I../netcdf/include $(AMBERBUILDFLAGS) #------------------------------------------------------------------------------ # Fortran preprocessing and compiler. # FPPFLAGS holds the main Fortran options, such as whether MPI is used. #------------------------------------------------------------------------------ FPPFLAGS= -I$(MPI_HOME)/include -P -DMKL -DBINTRAJ -DMPI $(AMBERBUILDFLAGS) FPP= cpp -traditional $(FPPFLAGS) FC= mpif90 FFLAGS= -w95 -vec_report0 -mp1 -O0 $(LOCALFLAGS) $(AMBERBUILDFLAGS) FOPTFLAGS= -w95 -vec_report0 -mp1 -ip -O3 -axWP $(LOCALFLAGS) $(AMBERBUILDFLAGS) FREEFORMAT_FLAG= -free #------------------------------------------------------------------------------ # Loader: #------------------------------------------------------------------------------ LOAD= mpif90 $(LOCALFLAGS) $(AMBERBUILDFLAGS) LOADCC= mpicc $(LOCALFLAGS) $(AMBERBUILDFLAGS) MKLDIR = /opt/intel/mkl/10.0.2.018/lib/em64t LAPACK = -lmkl_lapack MKLLIBS = -lmkl_intel_lp64 -lmkl_sequential -lmkl_core #------------------------------------------------------------------------------ # You can use the below lines to include the Goto BLAS but there is really no # reason to since there is almost no difference in performance relative to using # just MKL. In fact, goto was *slightly* slower on the test I ran. # #GOTODIR = /opt/lib/goto/1.26 #GOTOLIB = -lgoto_penrynp-r1.26 #LOADLIB= -L$(MKLDIR) $(LAPACK) -L$(GOTODIR) $(GOTOLIB) -L$(MKLDIR) $(MKLLIBS) -Wl,-rpath -Wl,$(GOTODIR) #------------------------------------------------------------------------------ LOADLIB= -L$(MKLDIR) $(LAPACK) $(MKLLIBS) LM= -lm XHOME= /usr/X11R6 XLIBS= -L/usr/X11R6/lib64 #------------------------------------------------------------------------------ # Other stuff: #------------------------------------------------------------------------------ .SUFFIXES: .f90 EMPTY= AR=ar rv M4=m4 RANLIB=ranlib SFX= NETCDF=netcdf.mod NETCDFLIB=../netcdf/lib/libnetcdf.a MODULEDIR=-I testsanderDIVCON=test.sander.DIVCON INCDIVCON=divcon LIBDIVCON=../dcqtp/src/qmmm/libdivcon.a # Location for executables, etc: BINDIR=/home/taylor/Amber/amber10/bin LIBDIR=/home/taylor/Amber/amber10/lib INCDIR=/home/taylor/Amber/amber10/include DATDIR=/home/taylor/Amber/amber10/dat # default rules for Fortran and C compilation: .f.o: $< $(FPP) $< > _$< $(FC) -c $(FFLAGS) -o $@ _$< .c.o: $(CC) -c $(CFLAGS) $(CPPFLAGS) -o $@ $<
