On Big Red II at IU, how do I use PCP to bundle multiple serial jobs to run them in parallel?
On this page:
- Adding PCP to your Big Red II user environment
- Bundling and running multiple serial jobs in parallel on Big Red II
- Getting help
Parallel Command Processor (PCP), developed by the Ohio Supercomputer Center (OSC) and the National Institute for Computational Sciences (NICS), is an application that lets you bundle multiple serial jobs and run them concurrently.
Using PCP to bundle and run your serial jobs in parallel lets you make efficient use of all the cores on a compute node. Conversely, running one serial job at a time can waste more than 90% of a node's computational power. PCP also lets you request multiple nodes for your jobs.
PCP is especially useful for running parametric studies and Monte Carlo simulations.
Adding PCP to your Big Red II user environment
At Indiana University, PCP is available on Big Red
II. To add PCP to your user environment on Big Red II, load the
pcp/2008 module; on the command line, enter:
module load pcp/2008
To make permanent changes to your environment, edit your
~/.modules file. For more, see In Modules, how do I save my environment with a .modules
For more about the Modules package, see the
module manual page
page. Additionally, see On Big Red II, Mason, Quarry, and Rockhopper at IU, how do I
use Modules to manage my software environment?
Running multiple serial jobs in parallel with PCP
For a parallel job with
N processors allocated, the
PCP manager process will read the first
N-1 commands in
the command stream and distribute them to the other
processors. As processes complete, the PCP manager reads the next
process in the stream and runs it on an idle core.
For example, to run 31 serial jobs in parallel on Big Red II:
- Create a text file (e.g.,
list.txt), listing each job, one per line:
./a.out > o1.txt ./a.out > o2.txt ./a.out > o3.txt ....... ./a.out > o30.txt ./a.out > o31.txt
- Create a TORQUE job script (e.g.,
pbs_script.sh); for details, see How do I run batch jobs on Big Red II at IU? In your script:
- Request the number of cores needed to run all the jobs
and the PCP manager process (for this example, you'd need
to request 32 cores to run the 31 jobs in
list.txtplus the PCP manager process).
- Launch PCP with
aprunto execute the jobs listed in your text file (e.g.,
#!/bin/bash #PBS -l nodes=1:ppn=32 #PBS -l walltime=00:10:00 #PBS -N my_job #PBS -q cpu #PBS -V aprun -n 32 pcp list.txt
- Request the number of cores needed to run all the jobs and the PCP manager process (for this example, you'd need to request 32 cores to run the 31 jobs in
- Submit the script (e.g.,
pbs_script.sh) from the command line using
As a result, the 31 serial jobs in
list.txt will run
in parallel on Big Red II. When the PCP manager runs out of commands
to run, it will wait for any remaining running processes to complete,
and then shut itself down.
Support for this system is provided by the UITS High Performance Systems (HPS) and Scientific Applications and Performance Tuning (SciAPT) groups. If you have system-specific questions, contact the HPS group. If you have questions about compilers, programming, scientific/numerical libraries, or debuggers on this system, contact the SciAPT group.
This is document bdka in the Knowledge Base.
Last modified on 2014-10-20.
- Fill out this form to submit your issue to the UITS Support Center.
- Please note that you must be affiliated with Indiana University to receive support.
- All fields are required.