CSCI 3366 (Introduction to Parallel and Distributed Processing):
Sample Programs

OpenMP

My notes Using OpenMP on the Linux Lab Machines.
Sample makefile Makefile for compiling C and C++ programs. (See the Note About Makefiles below.)
"Hello, world" programs: hello.c, hello++.cpp.

Utility functions for use with remaining examples: timer.h, cmdline.h.

Simple example of synchronization among threads:

Without synchronization: synch-1.c.
Using critical section for synchronization: synch-2.c.
Using lock for synchronization: synch-3.c.

Programs to compute pi by numerical integration:

Sequential version: num-int-seq.c.
Parallel version using OpenMP constructs: num-int-par.c.
Parallel versions using OpenMP constructs with SPMD approach (see comments in code for differences among versions): num-int-par-spmd-1.c, num-int-par-spmd-2.c.

Programs for heat diffusion example:

Sequential version: heat-eqn-seq.c.
Parallel version: heat-eqn-par.c.
SPMD-style parallel version: heat-eqn-par-spmd.c.
Revised versions that combine loops where feasible: heat-eqn-seq-v2.c, heat-eqn-par-v2.c, heat-eqn-par-spmd-v2.c.
Additional code for above programs (you will need these to compile): heat-eqn-functions.h.

Programs for Mandelbrot example:

Sequential version: mandelbrot-seq.c.
Parallel version with one-point-at-a-time critical section: mandelbrot-omp-by-points.c.
Parallel version with one-row-at-a-time critical section: mandelbrot-omp-by-rows.c.
Additional code for above programs (you will need these to compile): mandelbrot-gui.h.
Makefile: Makefile.mandelbrot

Programs for matrix multiplication example:

Simple sequential version: matrix-mult-seq-simple.c.
Block-based sequential version: matrix-mult-seq-block.c.
Block-based parallel version: matrix-mult-par-block.c.
Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h.

Generic master/worker programs:

Sequential version: master-worker-seq.c.
Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.
Additional code for above programs (you will need these to compile): master-worker-functions.h.

MPI

My notes Using MPI on the Linux Lab Machines.
Sample makefile Makefile for compiling C and C++ programs. (See the Note About Makefiles below.)
"Hello, world" programs: hello.c. hello++.cpp.
"Hello, world" program with command-line arguments and hostname: fancy-hello.c.
Utility functions for use with remaining examples: utility.h, timer.h, cmdline.h.
Trivial example of send/receive: send-recv.c.
Simple send/receive with timing: time-msg.c.
Programs to compute pi by numerical integration:
- Sequential version: num-int-seq.c.
- Parallel version: num-int-par.c.
Programs for heat diffusion example:
- Sequential version: heat-eqn-seq.c.
- Parallel version: heat-eqn-par.c.
- Parallel version with overlapping computation and communication: heat-eqn-par-overlap.c.
- Revised versions that combine loops where feasible: heat-eqn-seq-v2.c, heat-eqn-par-v2.c, heat-eqn-par-overlap-v2.c.
- Additional code for above programs (you will need these to compile): heat-eqn-functions.h, heat-eqn-mpi-functions.h.
Programs for Mandelbrot example:
- Sequential version: mandelbrot-seq.c.
- Parallel version assigning work by blocks: mandelbrot-mpi-block.c.
- Parallel version assigning work dynamically: mandelbrot-mpi-dynamic.c.
- Parallel version assigning work round-robin: mandelbrot-mpi-cyclic.c.
- Additional code for above programs (you will need these to compile): mandelbrot-gui.h.
- Makefile: Makefile.mandelbrot.
Programs for matrix multiplication example:
- Simple sequential version: matrix-mult-seq-simple.c.
- Block-based sequential version: matrix-mult-seq-block.c.
- Block-based parallel version: matrix-mult-par-block.c.
- Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h, matrix-mult-par-initfortest.h, matrix-mult-par-print.h, matrix-mult-par-utility.h.
Generic master/worker programs:
- Sequential version: master-worker-seq.c.
- Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
- Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.
- Additional code for above programs (you will need these to compile): master-worker-functions.h.

Java

My notes Java from the Command Line.
ZIP archive containing all examples csci3366sample.zip.
Utility classes: ThreadUtility.java.
"Hello, world" programs:
- Version using anonymous inner class: Hello1.java.
- Version using static inner class: Hello2.java.
- Versions using java.util.concurrent: Hello3.java, Hello4.java, Hello5.java.
Simple example of synchronization among threads:
- Without synchronization: HelloSynch1.java.
- With incorrect synchronization: HelloSynch2.java.
- With correct synchronization, two ways: HelloSynch3.java, HelloSynch4.java.
Programs to compute pi by numerical integration:
- Sequential version: NumIntSeq.java.
- Parallel version using explicit threads: NumIntPar1.java.
- Parallel version using java.util.concurrent: NumIntPar2.java.
"Bounded buffer" class and test program, showing use of wait/notify synchronization and interrupts: BoundedBuffer.java, TestBoundedBuffer.java.

Programs to test/time mergesort:

Sequential version: MergeSortSeq.java.
Parallel version: MergeSortPar.java.

Generic master/worker programs:

Supporting code: FakeTasks.java.
Sequential version: seq/MasterWorkerSeq.java.
Multi-threaded versions:
- threads/MasterWorkerParStatic.java.
- threads/MasterWorkerParDynamic.java.
Client-server versions using sockets:
- Supporting code: sockets/Master.java.
- Worker: sockets/Worker.java.
- Master using "parallel" approach (it knows how many workers): sockets/Master1.java.
- Master using "distributed" approach (it continues to accept workers until all tasks are done): sockets/Master2.java
To execute: Compile everything, and then run either sockets/Master1 or sockets/Master2 as the master, plus one or more workers. The start-workers script below may be useful.
Client-server version using RMI:
- Supporting code: rmi/MasterWorkerRMIMasterInterface.java, rmi/MasterWorkerRMIWorkerInterface.java.
- Master: rmi/Master.java.
- Worker: rmi/Worker.java.
To execute: Compile everything, start an instance of the registry with the command "rmiregistry" (in the directory containing the .class files), and then run one master and one or more workers. The start-workers script below may be useful.
Script start-workers to start "worker" processes on selected machines. (Somewhat preliminary version.) To use, copy into a directory in your search path (current directory will probably work) and make executable with chmod u+x start-workers. Executing with no arguments gives a usage message.

OpenCL

My notes Using OpenCL on the Linux Lab Machines.
Sample makefile Makefile for compiling C and C++ programs. (See the Note About Makefiles below.)
Utility functions for use with examples: timer.h, filesize.h. cmdline.h. utility.h.
First program (to show information about available devices): semi-hello.c.
Simple example (vector addition):
- Version shown in class: vector-add.c.
- Version using utility.h functions: vector-add-v2.c.
- Version using events, as an example of their use: vector-add-v3.c.
Example of totally contrived non-trivial computation: vector-compute.c, vector-compute-aux.h.
Programs to compute pi by numerical integration:
- Sequential version (simple version): num-int-seq-float1.c.
- Sequential version (homebrew "improved" version): num-int-seq-float2.c.
- Sequential version (improved version using Kahan algorithm): num-int-seq-float3.c.
- Parallel version using OpenCL kernel on GPU: num-int-par.c.
- Parallel version using OpenCL kernel, with command-line arguments that allow varying some parameters: num-int-par-with-args.c.
Programs for heat diffusion example:
- Sequential version (using floats rather than doubles): heat-eqn-seq-float.c.
- Parallel version: heat-eqn-par.c.
- Parallel version copying to local memory: heat-eqn-par-localmem.c.
- Revised versions that combine loops where feasible: heat-eqn-seq-float-v2.c, heat-eqn-par-v2.c, heat-eqn-par-localmem-v2.c.
- Additional code for above programs (you will need these to compile): heat-eqn-functions.h, heat-eqn-seq-functions.h, heat-eqn-par-functions.h.

POSIX Threads

Sample makefile Makefile for compiling C and C++ programs. (See the Note About Makefiles below.)
Utility functions for use with examples: timer.h, cmdline.h.
"Hello, world" program: hello.c,
Simple example of synchronization among threads:
- Without synchronization: synch-1.c.
- Using lock for synchronization: synch-2.c.
Programs to compute pi by numerical integration:
- Sequential version: num-int-seq.c.
- Parallel version (using lots of global variables): num-int-par1.c.
- Parallel version (not using global variables): num-int-par2.c.

C++11 Threads

Sample makefile Makefile for compiling C++ programs. (See the Note About Makefiles below.) Notice that you will need to use the more-recent of the two versions of gcc available. See my notes GNU compilers on the CS lab machines for more about that.
Utility functions for use with examples: timer.h, cmdline.h.
"Hello, world" program: hello.cpp,
Simple example of synchronization among threads:
- Without synchronization: synch-1.cpp.
- Using mutex lock for synchronization: synch-2.cpp.
Programs to compute pi by numerical integration:
- Sequential version: num-int-seq.cpp.
- Parallel version: num-int-par.cpp.

Note About Makefiles

To obtain a makefile, use your browser's download function; cutting and pasting will likely turn the tabs in the makefile into spaces and cause it to not work. Put the makefile in the directory in which you want to compile. All the makefiles linked from this page begin with brief documentation on their use (the "usage examples").

CSCI 3366 (Introduction to Parallel and Distributed Processing): Sample Programs