CSCI 3366 (Introduction to Parallel and Distributed Processing), Fall 2006:
Using MPI on the Linux Lab Machines

You should be able to run MPI programs on any of the CS department's Linux machines, including those in HAS 228 (Janusxx, where xx ranges from 00 through 21), those in HAS 340 (Xenaxx, where xx ranges from 00 through 21), those in HAS 329 (Atlasxx, where xx ranges from 00 through 09), and the parallel-programming cluster (Dione00, Dione01, and Dione2). Note that multiboot machines must be running Linux to be usable for this purpose.

Setting up your account

In order for MPI to start processes on other machines, your account must be set up so that MPI can remotely log into the other machines without prompting for a password.

Setup for ssh

Probably the preferred way to do this is to tell MPI to use ssh for the remote logins and set up your account to permit ssh logins without a password prompts.

To check that MPI will use ssh, type the command echo $LAMRSH. If the output is /usr/bin/ssh, all is well. If it's not, you need to execute the following command, which you should probably put in the file .bash_profile in your home directory so you don't have to type it in every time:

export LAMRSH=/usr/bin/ssh

Now you need to set up your account to allow passwordless logins. The easy way to do this is as follows:

• Create a public/private key pair thus:

  ssh-keygen -t dsa

  When asked for a passphrase, just hit return.

• Append the public key just generated (.ssh/id_dsa.pub) to .ssh/authorized_keys2. (If that file doesn't exist, create it by copying .ssh/id_dsa.pub to .ssh/authorized_keys2.)

• ssh is strict about permissions of some directories and files. A setup that appears to meet all its requirements is this:
  • The .ssh directory and all files it contains are accessible only by you. (chmod -R go= .ssh will accomplish this.)

  • Your home directory is writable only by you. (chmod go-w . will accomplish this.)

• Check that everything is working correctly by trying to execute a simple command, thus:

  ssh xena00 date

Setup for rsh

(Kept for now, for information purposes only. It appears that rsh has been disabled on the lab machines.) Another way to get the same effect (assuming rsh is enabled) is to tell MPI to use rsh for the remote logins and create a file .rhosts in your home directory containing a list of machine/username combinations from which such logins should be allowed without prompting for a password. (There are various security risks associated with all of this, but the consensus of the local experts is that they're not a major concern in our environment.)

To check that MPI will use rsh, type the command echo $LAMRSH. If the output is /usr/bin/rsh, all is well. If it's not, you need to execute the following command, which you should probably put in the file .bash_profile in your home directory so you don't have to type it in every time:

export LAMRSH=/usr/bin/rsh

The .rhosts file should look something like this with yourUserName replaced by your username.

Warning: To avoid a serious security hole, BE SURE that the permissions on this file do not allow others to modify it! You can be sure this is the case with the following command:

chmod go= .rhosts

Pathnames

The MPI-related commands for the LAM implementation are installed in one of the standard system directories (/usr/bin), so if you simply type, for example, mpicc to invoke the MPI C compiler, you will get the compiler for the LAM implementation. As with other compilers, it can be useful to compile with some non-default options (e.g., -Wall), and you can use a Makefile for this purpose. See the sample programs page for an example.

Compiling MPI programs (in C)

Use the command mpicc to compile C programs using MPI functions. Typical usage:

mpicc -o foo foo.c

See the sample programs page for a Makefile.

Running MPI programs

Starting the runtime environment

Before running an MPI program, use the command lamboot to start up the runtime environment. (You only need to do this once per login session. When you are finished running MPI programs, use the lamhalt command to shut things down again.) Typical usage:

lamboot -b -v MPIhosts

where MPIhosts specifies the hostnames of machines to be used; in its simplest form, it can just be a list of machine names, i.e., like this. All the machines in MPIhosts should be up (and running Linux) when this command is run, and the list of machine names should include the machine from which you run the lamboot command.

Running programs

Use the command mpirun to run an MPI program. Typical usage:

mpirun -np 10 foobar

(Starts 10 processes, each running executable foobar.)

Shutting down the runtime environment

After running MPI programs, use the command lamhalt to shut down the processes started by lamboot. Typical usage:

lamhalt

If this command hangs or appears not to work (perhaps because something has gone wrong earlier), you can use the following command to shut things down less gracefully:

wipe -b -v MPIhosts

The LAM implementers warn of unpredictable and undesirable consequences if you log out without issuing this command. A way to ensure that it is called when you log out is to put the following lines in a file .bash_logout in your home directory:

if [ ! -z "`ps aux | grep lamd | grep $USER | grep -v grep`" ]
then
lamhalt
fi

Online documentation

Use the command man to display UNIX-style man pages for MPI-related commands and routines. Typical usage:

man MPI_Send