Using MPI on the CS Linux Machines

You should be able to run MPI programs on any of our Linux machines. (Note that to use a machine as one of the MPI ``hosts'' it must be running Linux. You can check availability with the command ruptime; machines that show as ``up'' are available. Note however that output of this command includes both generally-available machines and some special-purpose machines such as faculty desktops and research machines to which you may not have access.)

Setting up your account

Enabling remote login without prompts

In order for MPI to start processes on other machines, your account must be set up so that MPI can remotely log into the other machines without prompting for a password. See my notes on passwordless login for notes on how to do this.

Other setup considerations

If your account is set up according to our default scheme, the commands needed to compile and execute MPI programs will not be in the search path. This is deliberate, at least in part because more than one version of MPI is installed:

To get access to the version of MPI from the standard repositories, use the command

module load openmpi-default

To get access to the version of MPI that uses the most recent version of the compilers, use the command

module load openmpi-latest

The command module avail may show you additional choices. See my notes on the ``Modules package'' for more about the module command.

Compiling MPI programs

Use the command mpicc to compile C programs using MPI functions. Simple usage:

mpicc -o foo foo.c

Refer to the man page for other options.

To compile C++ programs using MPI functions, use mpiCC, mpic++, or mpicxx. (I'm not sure how these commands differ, if at all.)

To compile Fortran programs using MPI functions, use mpif77 or mpif90.

Running MPI programs

Use the command mpirun to run an MPI program. Typical usage:

mpirun -np 10 -host host1,host2 foobar

mpirun -np 10 -hostfile list-of-hosts.txt foobar

Both commands start 10 processes, each running executable foobar. The first command starts them on machines host1 and host2; the second command reads machine names from the file list-of-hosts. Refer to the man page for other options.

Note: If the list of ``hosts'' includes anything other than the machine on which you execute mpirun, MPI may not be able to find your programs and its libraries. (The ``latest'' versions seem okay in this regard, but the ``default'' versions don't. Symptoms of failure vary from version to version, but all the error messages seem to say in some way that there are executables that aren't being found.)

To fix this, you can either add the appropriate module load command to your .bashrc file, as described in my notes on the ``Modules package'', or you can include the -prefix option, as in the following example:

mpirun -prefix $MPI_HOME -np 10 -host host1,host2 foobar

($MPI_HOME is an environment variable set by the MPI modules.)

Online documentation

Use the command man to display UNIX-style man pages for MPI-related commands and routines. Typical usage:

man mpicc

man MPI_Send

(Note that in order for this to work you must have loaded one of the MPI modules.)

Useful links

Berna Massingill