Operating Environment

From UF HPC Wiki

Jump to: navigation, search

The University of Florida Phase I and II cluster has an up-to-date operating environment to offer our users the flexibility needed for operation of their projects.

Contents

Operating System

Monitoring Software

System Software

  • GNU Compiler Suite (3.4.6-8)
    • gcc
    • g++
    • g77
  • GNU Compiler Suite (4.1.1-53)
    • gcc
    • g++
    • gfortran
  • Intel Compiler Suite (9.1.044)
    • icc
    • ifort
  • Intel Compiler Suite (10.1.015)
    • icc
    • ifort
  • OFED 1.2 IB Stack
    • Kernel IB
    • OpenMPI (1.2.2)
    • MVAPICH (0.9.9)
    • MVAPICH2 (0.9.8)

Hardware

Compute Nodes

Type#ProcessorCoresMemoryInfinibandTotal CoresTotal Memory
Compute202Two dual core 2.2GHz AMD Opteron 27544 GBYES202 x 4 = 808202 x 4 = 808
Compute120Two dual core 2.2GHz AMD Opteron 27548 GBYES120 x 4 = 480120 x 8 = 960
Compute80Two dual core 2.2GHz AMD Opteron 27548 GBNO80 x 4 = 32080 x 8 = 640
Compute2Four dual core 2.2GHz AMD Opteron 875816 GBYES2 x 8 = 162 x 16 = 32
Compute91Two quad core 2.8GHz Intel E5462832 GBYES91 x 8 = 72891 x 32 = 2912
Compute21Two quad core 2.8GHz Intel E5462864 GBYES21 x 8 = 16821 x 64 = 1344
Compute4One quad core 2.8GHz Intel E5462416 GBYES4 x 4 = 164 x 16 = 64
MDS2Two dual core 2.0GHz AMD Opteron 2212HE48 GBYES2 x 4 = 82 x 8 = 16
OSS8Two dual core 2.2GHz AMD Opteron 27544 GBYES8 x 4 = 328 x 4 = 32
OSS2Two quad core 2.0GHz AMD Opteron 2350416 GBYES2 x 4 = 82 x 16 = 32



Networking

New Physics Building

  • InfiniBand Interconnect
    • 50% Blocking Fat Tree Topology
    • 2 Cisco 7008 Core IB Switches (96 Ports 4x IB)
    • 14 Cisco 7000 Leaf IB Switches (24 x 4x SDR)
    • 8 Silverstorm 9024S IB Leaf Switches (24 x 4X SDR/DDR)
    • 1 Cisco 3012 EtherNet/Fibre Channel Gateway
      • 4 FC Gateway Modules
      • 6 EtherNet Gateway Modules
  • Gigabit Ethernet Interconnect (Management)
    • Cisco 6506 Gigabit Ethernet Switch
    • Cisco 4948 Gigabit Ethernet Switch
    • 8 3Com 2848 48-Port Gigabit Ethernet Switches
    • 4 Foundry FastIron Edge X448 Ethernet Switches

Larsen Hall

  • InfiniBand Interconnect
  • 50% Blocking Fat Tree Topology
  • Voltaire 2012
    • 2 Fabric Controllers (Spines)
    • 3 24 Port (4x DDR IB) Modules
    • 2 20210G IB/Ethernet Gateway Modules
      • 2 10 Gb Ethernet (XFP Slots)
      • 20 Ports (4x DDR IB)
  • Force-10 C150 Ethernet Switch
    • 1 RP Module
    • 1 8 Port 10 Gb Ethernet (XFP Slots) Module

Node Performance

Penguin Relion 1670

  • 64 GB RAM (See configuration details above)
  • OpenMPI 1.2.2 (--btl sm,self)
  • Goto BLAS (1.26, -lgoto_penrynp-r1.26)
mpirun --mca btl sm,self -hostfile mpihosts -np 8 ./xhpl
============================================================================
HPLinpack 1.0a  --  High-Performance Linpack benchmark  --   January 20, 2004
Written by A. Petitet and R. Clint Whaley,  Innovative Computing Labs.,  UTK
============================================================================

An explanation of the input/output parameters follows:
T/V    : Wall time / encoded variant.
N      : The order of the coefficient matrix A.
NB     : The partitioning blocking factor.
P      : The number of process rows.
Q      : The number of process columns.
Time   : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N      :   90800 
NB     :     184 
PMAP   : Row-major process mapping
P      :       2 
Q      :       4 
PFACT  :   Right 
NBMIN  :       4 
NDIV   :       2 
RFACT  :   Right 
BCAST  :  2ringM 
DEPTH  :       0 
SWAP   : Spread-roll (long)
L1     : transposed form
U      : transposed form
EQUIL  : yes
ALIGN  : 8 double precision words

----------------------------------------------------------------------------

- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
   1) ||Ax-b||_oo / ( eps * ||A||_1  * N        )
   2) ||Ax-b||_oo / ( eps * ||A||_1  * ||x||_1  )
   3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be          1.110223e-16
- Computational tests pass if scaled residuals are less than           16.0

============================================================================
T/V                N    NB     P     Q               Time             Gflops
----------------------------------------------------------------------------
WR03R2R4       90800   184     2     4            6185.92          8.068e+01
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1  * N        ) =        0.0030523 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1  * ||x||_1  ) =        0.0029648 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) =        0.0004746 ...... PASSED
============================================================================

Finished      1 tests with the following results:
              1 tests completed and passed residual checks,
              0 tests completed and failed residual checks,
              0 tests skipped because of illegal input values.
----------------------------------------------------------------------------

End of Tests.
============================================================================
Personal tools