CSCI 3366 (Parallel and Distributed Processing):
Sample Programs

Disclaimer: Code linked from this page is intended as a cleaned-up and commented version of examples from class. Many programs could be further improved with regard to functionality or design, but then they would no longer be as close to what we did in class.

OpenMP

My notes Using OpenMP on the Linux Lab Machines.
Sample makefile Makefile. (See the Note About Makefiles below.)
ZIP archive containing all examples OpenMP.zip.
"Hello, world" programs: hello.c, hello++.cpp.
Simple example of synchronization among threads:
- Without synchronization: synch-1.c.
- Using critical section for synchronization: synch-2.c.
- Using lock for synchronization: synch-3.c.
Utility functions for use with remaining examples: timer.h,
Programs to compute pi by numerical integration:
- Sequential version: num-int-seq.c.
- Parallel version using OpenMP constructs: num-int-par.c.
- Parallel versions using OpenMP with SPMD approach (rather than OpenMP parallel for): num-int-par-spmd-1b.c, num-int-par-spmd-1a.c, num-int-par-spmd-2.c.
Programs for heat diffusion example:
- Sequential version: heat-eqn-seq.c.
- Parallel version: heat-eqn-par.c.
- SPMD-style parallel version: heat-eqn-par-spmd.c.
- Additional code for above programs (you will need these to compile): heat-eqn-functions.h.
Generic master/worker programs:
- Sequential version: master-worker-seq.c.
- Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
- Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.
- Additional code for above programs (you will need these to compile): master-worker-functions.h.
Programs for Mandelbrot example:
- Sequential version: mandelbrot-seq.c.
- Parallel version with one-point-at-a-time critical section: mandelbrot-omp-by-points.c.
- Parallel version with one-row-at-a-time critical section: mandelbrot-omp-by-rows.c.
- Additional code for above programs (you will need these to compile): mandelbrot-gui.h.
- Makefile: Makefile
Programs for matrix multiplication example:
- Simple sequential version: matrix-mult-seq-simple.c.
- Block-based sequential version: matrix-mult-seq-block.c.
- Simple parallel version: matrix-mult-par-simple.c.
- Block-based parallel version: matrix-mult-par-block.c.
- Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h. matrix-mult-par-utility.h.

MPI

My notes Using MPI on the Linux Lab Machines.
Sample makefile Makefile. (See the Note About Makefiles below.)
ZIP archive containing all examples MPI.zip.
Sample script to run program run-pgm. Uses list of machine names in Hosts-pandora.
"Hello, world" programs: hello.c. hello++.cpp.
"Hello, world" program with command-line arguments and hostname: fancy-hello.c.
Utility functions for use with remaining examples: utility.h, timer.h,
Trivial example of send/receive: send-recv.c.
Simple send/receive with timing: time-msg.c.
Programs to compute pi by numerical integration:
- Sequential version: num-int-seq.c.
- Parallel version: num-int-par.c.
Programs for heat diffusion example:
- Sequential version: heat-eqn-seq.c.
- Parallel version: heat-eqn-par.c.
- Parallel version with overlapping computation and communication: heat-eqn-par-overlap.c.
- Additional code for above programs (you will need these to compile): heat-eqn-functions.h, heat-eqn-mpi-functions.h.
Programs demonstrating various ways to communicate:
- Simple send/receive (can block if messages are large): exchange.c.
- Simple send/receive, but with half the processes sending then receiving and the other half receiving then sending: exchange-alt.c.
- Synchronous send/receive exchange-synch.c.
- Non-blocking send/receive: exchange-asynch.c.
- Send/receive with "persistent communication" (not shown in class): exchange-persistent.c.
Generic master/worker programs:
- Sequential version: master-worker-seq.c.
- Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
- Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.
- Additional code for above programs (you will need these to compile): master-worker-functions.h.
Programs for Mandelbrot example:
- Sequential version: mandelbrot-seq.c.
- Parallel version assigning work by blocks: mandelbrot-mpi-block.c.
- Parallel version assigning work dynamically: mandelbrot-mpi-dynamic.c.
- Parallel version assigning work round-robin: mandelbrot-mpi-cyclic.c.
- Additional code for above programs (you will need these to compile): mandelbrot-gui.h.
- Makefile: Makefile.
Programs for matrix multiplication example:
- Simple sequential version: matrix-mult-seq-simple.c.
- Block-based sequential version: matrix-mult-seq-block.c.
- Block-based parallel version: matrix-mult-par-block-1.c.
- Block-based parallel version, revised to use additionall commmunicators matrix-mult-par-block-2.c.
- Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h, matrix-mult-par-initfortest.h, matrix-mult-par-print.h, matrix-mult-par-utility.h.
Programs for matrix multiplication example:
- Simple sequential version: matrix-mult-seq-simple.c.
- Block-based sequential version: matrix-mult-seq-block.c.
- Block-based parallel version: matrix-mult-par-block.c.
- Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h.

Java

My notes Java from the Command Line.
ZIP archive containing all examples Java.zip.
Script to compile all Java files in directory: compile-all .
Utility classes: Utility.java.
"Hello, world" programs:
- Version using anonymous inner class: Hello1.java.
- Version using static inner class: Hello2.java.
- Versions using java.util.concurrent: Hello3.java, Hello4.java, Hello5.java.
Simple example of synchronization among threads:
- Without synchronization: HelloSynch1.java.
- With incorrect synchronization: HelloSynch2.java.
- With correct synchronization, two ways: HelloSynch3.java, HelloSynch4.java.
Programs to compute pi by numerical integration:
- Sequential version: NumIntSeq.java.
- Parallel version using explicit threads: NumIntPar1.java.
- Parallel version using java.util.concurrent: NumIntPar2.java.
Programs to test/time mergesort:
- Sequential version: MergeSortSeq.java.
- Parallel version: MergeSortPar.java.

OpenCL

My notes Using OpenCL on the Linux Lab Machines.
Sample makefile Makefile. (See the Note About Makefiles below.)
ZIP archive containing all examples OpenCL.zip.
Utility functions for use with examples: timer.h, filesize.h, utility.h.
First program (to show information about available devices): semi-hello.c.
Simple example (vector addition):
- Version shown in class: vector-add.c.
Programs to compute pi by numerical integration:
- Sequential version (simple version): num-int-seq-float.c.
- Sequential version (improved version using Kahan algorithm): num-int-seq-kahan.c.
- Parallel version using OpenCL kernel: num-int-par.c.
- Repackaged parallel version using OpenCL kernel, with kernel source code in another file (read in at runtime) , num-int-par-file.c, num-int-par-file-kernel.h.
Programs for heat diffusion example:
- Sequential version using float rather than double to be compatible with what OpenCL kernels can do: heat-eqn-seq-float.c.
- Parallel version: heat-eqn-par.c, with kernel source code in heat-eqn-par-kernel.h (read in at runtime).
- Additional code for above programs (you will need these to compile): heat-eqn-seq-functions.h, heat-eqn-par-functions.h.

POSIX Threads (P-Threads)

Sample makefile Makefile. (See the Note About Makefiles below.)
ZIP archive containing all examples P-Threads.zip.
"Hello, world" program: hello.c.
Simple example of synchronization among threads:
- Without synchronization: synch-1.c.
- Using lock for synchronization: synch-2.c.
Utility functions for use with remaining examples: timer.h,
Programs to compute pi by numerical integration:
- Sequential version: num-int-seq.c.
- Parallel versions, somewhat ugly version: num-int-par-1.c.
- Parallel version, revised to maybe be less ugly: num-int-par-2.c.
C++11 Threads
- Sample makefile Makefile. (See the Note About Makefiles below.)
- ZIP archive containing all examples C++Threads.zip.
- "Hello, world" program: hello.cpp.
- Simple example of synchronization among threads:
  - Without synchronization: synch-1.cpp.
  - Using lock for synchronization, one way: synch-2.cpp.
  - Using lock for synchronization, another way: synch-3.cpp.
- Utility functions for use with remaining examples: timer.h,
- Programs to compute pi by numerical integration:
  - Sequential version: num-int-seq.cpp.
  - Parallel version using shared variable and lock: num-int-par-1.cpp.
  - Parallel version using C++ "futures": num-int-par-2.cpp.

Note About Makefiles

To obtain a makefile, use your browser's download function; cutting and pasting will likely turn the tabs in the makefile into spaces and cause it to not work. Put the makefile in the directory in which you want to compile. All the makefiles linked from this page begin with brief documentation on their use (the "usage examples").

Other files

Script to mail files for homework submission mail-files. Instructions for usage here.

CSCI 3366 (Parallel and Distributed Processing): Sample Programs