Math 6395 -- Introduction to Parallel Scientific Computing

Spring 2009

Course Syllabus


[back to top]

Instructor:   Dr. Daniel Reynolds

Textbooks:

Highly Recommended:
  • Chapman, Jost & van der Pas, Using OpenMP, Portable Shared Memory Parallel Programming, MIT Press, 2007, ISBN 0262533022.
  • Gropp, Lusk & Skjellum, Using MPI -- 2nd Edition: Portable Parallel Programming with the Message Passing Interface, MIT Press, 1999, ISBN 0262571323.
Additional Reading (available in Math library or from instructor):
  • Metcalf, Reid & Cohen, Fortran 95/2003 Explained, Oxford University Press, 2004, ISBN 0198526938.
  • Shonkwiler & Lefton, Introduction to Parallel and Vector Scientific Computation, Cambridge University Press, 2006, ISBN 0521683378.
  • Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill Press, 2003, ISBN 0072822562.
Supporting Materials (online):

Course Description:

MATH 6395 -- Introduction to Parallel Scientific Computing [3 Credits]

Historically, parallel computing has addressed the demands of state-of-the-art engineering and scientific applications. Recently, however, other platforms including video game consoles, laptops and desktops have begun to transition toward chips with an increasing number of processor cores, requiring algorithms that may effectively utilize their parallel computing hardware.

The goal of MATH 6395 is to introduce students to the foundations of parallel computing, including the principles of parallel computer architectures and parallel algorithm design. We will investigate a variety of algorithms, studying the relative benefits and weaknesses among different approaches for their parallel implementation. Students will learn how to use OpenMP and MPI to write their own simple parallel programs for shared- and distributed-memory systems, and will execute these programs on clusters here at SMU and at national supercomputing centers including the Texas Advanced Computing Center (TACC). Instruction will consist of both classroom lectures and computer lab tutorials.

Topics include:

  • Introduction to the Unix/Linux operating system
  • Overview of scientific and engineering computing
  • Parallel computer architectures
  • Parallel programming models
  • Parallel programming with OpenMP and MPI
  • Advanced topics, including debugging parallel programs, performance analysis and optimization, parallel application packages, hybrid parallel programming and petascale computing, time permitting.

Students from all disciplines and computing background are welcome, as we will attempt to cover the necessary background from computer architecture, software, and numerical analysis. Although there are no specific prerequesites for the course, it is expected that students will have experience with the design and implementation of computer programs. While familiarity with the Unix operating system and programming experience in compiled languages (e.g. C, C++, Fortran77/90/95, etc.) will prove beneficial, we will assume only a minimum background in programming for interactive languages (e.g. Matlab, Python, Perl).

Prerequisites: MATH 5315/CSE 7365 or by instructor's permission.

Educational Outcomes:

Upon successful completion of this course, students will have learned:
  • To identify different types of parallel computing hardware.
  • How to write shared-memory parallel programs using OpenMP.
  • How to write distributed-memory parallel programs using MPI.
  • The relative benefits of static and dynamic load-balancing within a parallel program.
  • Some intuition regarding which scientific computing algorithms may best utilize OpenMP and MPI parallel computing.
  • About multiple open-source software libraries for parallel scientific computing.

Tentative Lecture Schedule:

  • Weeks 1-3: Introduction to UNIX and Scientific Computing (lab)
    • setting up your environment (lab and home)
    • the command line and filesystem
    • executables: running, suspending, killing, etc.
    • compiling code: command-line, makefiles, etc.
    • fortran 90/95 introduction: structure, keywords, subroutines, data types, loop and control structures
    • timing utilities, debuggers, profilers, etc.
  • Weeks 1-3: Machines and Computation (lecture)
    • basic von-Neumann computer architecture
    • modern computer architectures: hierarchical memory, cache, OS, etc.
    • architectural approaches to parallelism
    • software/algorithmic approaches to parallelism
    • parallel computing metrics
  • Weeks 4-7: Shared-Memory Parallel Programming (OpenMP)
    • origins of OpenMP
    • creating an OpenMP program
    • SMP programming styles
    • case studies: computing pi, dot-product, matrix-vector product
    • OpenMP feature set
    • OpenMP performance concerns
    • target: coarse-grained approaches
  • Weeks 8-12: Distributed-Memory Parallel Programming (MPI)
    • goals and introduction to the MPI programming model
    • case studies: computing pi, dot-product, matrix-vector product, poisson problem, n-body problem
    • MPI communicators
    • MPI timing and performance utilities
    • common MPI programming errors
    • target: overlapping communication and computation
    • public-domain parallel solver libraries: PETSc, Trilinos, FFTW, ScaLAPACK, HYPRE, SuperLU
  • Weeks 13-15: Hybrid OpenMP/MPI Programming
    • modern and upcoming supercomputer architectures, and the need for hybrid parallelism
    • fine-grained hybrid programming
    • coarse-grained hybrid programming
    • running hybrid programs

Computing:

In this course, homework and projects will make use of Unix and Linux workstations, clusters and supercomputers. Students who are unfamiliar with the Unix/Linux operating system should complete the following tutorial, to help them gain a modest understanding of Unix/Linux. Students who own Windows computers should follow the instructions here to set up the appropriate software on their computer to allow them to emulate and/or login to Unix/Linux servers. A good text editor for programming on Windows computers may be found here.

All students should request an account at the Texas Advanced Computing Center (TACC) Portal (follow the link in the "New TACC user?" box). We will be using their Ranger supercomputer (one of the largest computational resources in the world) in this class. Once you obtain your user account, email me with your username so that I can grant you access to our computing time.

Programming:

As described below, students will be expected to submit programming assignments for their evaluation in the course. These assignments may be written in any one of the programming languages {Fortran77, Fortran90, Fortran95, C, C++}, since each of these languages support the style of programs that we will encounter in this course. However, for any students without prior experience in one of these languages, I will require the use of Fortran90/95. Further information on the language is provided here, and I will be happy to discuss the relative merits of each language with students during office hours.

All programming assignments (homework and projects) must be submitted electronically to the instructor via email. For assignments and projects that involve more than one file, zip them together and send the resulting single file, e.g. if the files are in the directory Project1, go to the directory containing Project1 and type

tar -czf Project1.tar.gz Project1
Submit the resulting file, Project1.tar.gz.

Turning in Programs:

To turn in a programming assignment, copy the tarball containing your results to the folder /shared/dreynolds/username/ on blackbeard, where username is your user name.

Homework:

Homework assignments will be assigned regularly throughout the semester, and will consist of small programming exercises and written assignments. Homework problems must be completed on your own, though discussion of the assignments with other students and the instructor is encouraged. In addition, I encourage all students to visit my office when they get stuck on bugs in their codes.

Projects:

There will be a small number of programming projects throughout the semester. These may be individal or group projects, with group projects often including individual components. Group projects must be submitted together, and all members of the group should contribute equally to their solution. Group members will be required to write a brief summary of their individual contributions to each project.

These projects may involve a more substantial amount of programming or other work -- I highly recommend that you begin these early and do not procrastinate.

Grading:

Students' grades in this course will be determined based on homeworks and projects, with each accounting for about 50% of the total.

Honor Code:

Academic dishonesty is considered a serious offense, and is doubly inexcusable among graduate students. All individual assignments must represent your own individual effort. All group members must accurately describe the project portions to which they contributed.

SMU Regulations:

Disability Accommodations: Students needing academic accommodations for a disability must first contact Ms. Rebecca Marin, Coordinator, Services for Students with Disabilities (214-768-4557) to verify the disability and establish eligibility for accommodations. They should then schedule an appointment with the professor to make appropriate arrangements. (See University Policy No. 2.4.)

Religious Observance: Religiously observant students wishing to be absent on holidays that require missing class should notify their professors in writing at the beginning of the semester, and should discuss with them, in advance, acceptable ways of making up any work missed because of the absence. (See University Policy No. 1.9.)

Excused Absences for University Extracurricular Activities: Students participating in an officially sanctioned, scheduled University extracurricular activity should be given the opportunity to make up class assignments or other graded assignments missed as a result of their participation. It is the responsibility of the student to make arrangements with the instructor prior to any missed scheduled examination or other missed assignment for making up the work. (University Undergraduate Catalogue)