This document provides instructions for using the Vampir toolchain at Indiana University (IU) on the Quarry and BigRed clusters. It describes how to run Vampir, VampirServer, and trace applications with VampirTrace on these systems, including details on software environments, launching jobs, and visualizing traces.

1. How To Use The
Vampir Toolchain At IU
Robert Henschel
rhensche@indiana.edu
May 2009

2. Contents
• Quarry
• BigRed
• Hands On
Robert Henschel

3. General
• Trace creation and visualization does not have to
happen on the same platform!
– You can look at traces created on Quarry (or your
own desktop) on BigRed
Robert Henschel

4. Quarry – SoftEnv Keys
• Vampir Version 5.2
+Vampir-5.2-32 or +Vampir-5.2-64
• VampirServer Version 1.10
+VampirServer-1.10-32
or
+VampirServer-1.10-64
• VampirTrace Version 5.6.3
@intel-64-v10.1
+openmpi-1.3.1-intel-64
+VampirTrace-5.6.3-intel-64
or
@intel-32-v10.1
+openmpi-1.3.1-intel-32
+VampirTrace-5.6.3-intel-32
Robert Henschel

5. Quarry – Running Vampir
• Vampir Version 5.2
– Interactive job in PBS
– On interactive nodes b005-b008
Robert Henschel

6. Quarry – Running Vampir
• Vampir Version 5.2
– Interactive job in PBS
qsub -I -X
soft add +Vampir-5.2-64
vampir
– On interactive nodes b005-b008
• You share the node with other people, this is
especially a problem for opening large trace files!
ssh -X b005
soft add +Vampir-5.2-64
vampir
Robert Henschel

7. Quarry – Running VampirServer
• Vampir Server Version 1.10
– Server in PBS, viewer on interactive node
– Server and viewer in interactive job in PBS
– Server and viewer on interactive nodes b005-b008
Robert Henschel

8. Quarry – Running VampirServer
• Vampir Server Version 1.10
– Server in PBS, viewer on interactive node
• Server in PBS
– Make sure your “.soft” file contains
+openmpi-1.3.1-intel-64
+VampirServer-1.10-64
– PBS script
#PBS -l nodes=2:ppn=8,walltime=5:00:00
#PBS -m ae
#PBS -N VampirServer
#PBS
mpirun -np 16 -machinefile $PBS_NODEFILE vngd -n 16 1>$HOME/vampir.out 2>&1
Start job
qsub vampir.pbs
– Wait for the job to start
Robert Henschel

9. Quarry – Running VampirServer
– Lookup host and port information
» It may take a minute or two after the job
started for this file to get updated!!!
cat $HOME/vampir.out
Running...
Server listens on: b052:30000
• Viewer on interactive nodes b005-b008
ssh -X b005
soft add +openmpi-1.3.1-intel-64
soft add +VampirServer-1.10-64
vng -a b052 -p 30000
Robert Henschel

10. Quarry – Running VampirServer
– Interactive job in PBS
• Only one node
qsub -I -X
soft add +openmpi-1.3.1-intel-64
soft add +VampirServer-1.10-64
mpirun -np 8 vngd -n 8 1>output 2>&1 &
cat output
Running...
Server listens on: b048:30000
vng -a localhost -p 30000
Robert Henschel

11. Quarry – Running VampirServer
– On interactive nodes b005-b008
• Only one node
• Runtime needs to be less than 24 hours
• You share the node with other people
ssh -X b005
soft add +openmpi-1.3.1-intel-64
soft add +VampirServer-1.10-64
mpirun -np 4 vngd -n 4 1>output 2>&1 &
cat output
Running...
Server listens on: b005:30000
vng -a localhost -p 30000
Robert Henschel

12. Quarry – Tracing with VampirTrace 5.6.3
• VampirTrace is configured to work with the Intel
compilers version 10.1 and OpenMPI 1.3.1
• Include the following keys in your .soft file
@intel-64-v10.1
+openmpi-1.3.1-intel-64
+VampirTrace-5.6.3-intel-64
• The VampirTrace compiler wrappers are now available
• It is possible to use them also with GCC, if you need to
do that, please send a mail to hpahelp@indiana.edu
• We can also provide a version for the PGI compilers.
Robert Henschel

13. BigRed – SoftEnv Keys
• Vampir Version 5.1
+Vampir-5.1-64
• VampirServer Version 1.10
+VampirServer-1.10-64
• VampirTrace Version 5.6.3
+openmpi-1.3.1-ibm-64
+VampirTrace-5.6.3-ibm-64
or
+openmpi-1.3.1-ibm-32
+VampirTrace-5.6.3-ibm-32
Robert Henschel

14. BigRed – Running Vampir
• Vampir Version 5.1
– Interactive job in LoadLeveler
Robert Henschel

15. BigRed – Running Vampir
• Vampir Version 5.1
– Interactive job in LoadLeveler
soft add +interactivejob
interactivejob -w 60
soft add +Vampir-5.1-64
vampir
• interactivejob can be customize for longer
interactive sessions
– run “interactivejob -?” for more
Robert Henschel

16. BigRed – Running VampirServer
• Vampir Server Version 1.10
– Server in LoadLeveler, viewer on login node
Robert Henschel

17. BigRed – Running VampirServer
• Vampir Server Version 1.10
– Server in LoadLeveler, viewer on login node
• Server in LoadLeveler
– Make sure your “.soft” file contains
+openmpi-1.3.1-ibm-64
+VampirServer-1.10-64
– Submit job to LoadLeveler
run_server.sh 4 DEBUG 15
or
run_server.sh 16 LONG 15
– Wait for the job to start
Robert Henschel

18. BigRed – Running VampirServer
– Lookup host and port information
» It may take a minute or two after the job
started for this file to get updated!!!
cat $HOME/paralleljob.x.0.out
Running...
Server listens on: s10c1b4:30000
• Viewer on login node
soft add +VampirServer-1.10-64
vng -a s10c1b4 -p 30000
Robert Henschel

19. BigRed – Tracing with VampirTrace 5.6.3
• VampirTrace is configured to work with the IBM compilers
and OpenMPI 1.3.1
• Include the following keys in your .soft file
+openmpi-1.3.1-ibm-64
+VampirTrace-5.6.3-ibm-64
• The VampirTrace compiler wrappers are now available
Robert Henschel