Introduction to Parallel Computing (184.710)

Plan

Lecture Monday 11.00 - 12.45, first 10.10.2011. All potential students should be present. Additional slot: Thursday, 12.00 - 14.00. Seminarroom 121.

Note: due to construction work, the lecture may be blocked toward the end of the semester. Cluster and shared-memory node system for the practical part are under installation.

Overview

Lectures, material

1. Monday 10.10: planning, schedule, overview
2. Thursday 13.10: Definitions, architecture and programming models, Speedup, work, cost, Amdahl's law, merging in parallel
3. Monday 17.10: Practical motivations, HPC computing, superlinear speedup, reductions and prefix sums
4. Monday 24.10: Shared-memory programming I: architectures, pthreads
5. Monday 31.10: Shared-memory programming II: pthreads (continued), OpenMP
6. Thursday 3.11: Shared-memory exercises: presentation, discussion
7. Monday, 7.11: Shared-memory programming: OpenMP (continued)
8. Thursday, 17.11: Distributed memory programming: Architectures, MPI
9. Monday 21.11, Thursday 24.11: No Lecture
10. Monday 28.11: Distributed memory programming: MPI (continued, point-to-point communication)
11. Monday 5.12: Distributed memory programming: MPI (one-sided, collective)
12. Monday 12.12: Distributed memory programming: MPI (benchmarking, tools)
13. Thursday 15.12: MPI exercises: presentation, discussion
14. Monday 19.12: No Lecture (installation of NEC cluster)
15. Merry christmas, happy new year...
16. Monday 9.1: Exercises, how to use the cluster
17. Thursday, 12.1: Exercises, discussion (as needed, I will leave after 10 minutes if nobody is there)
18. Monday 16.1: New architectures, other programming models
19. Thursday, 19.1: Exercises, discussion (as needed, I will leave after 10 minutes if nobody is there)
20. Monday 23.1: New architectures, other programming models
21. Thursday, 26.1: Exercises: hand-in (report/write up per email; everything on account)
22. Monday 30.1: Exercise presentation, set 1 (15-30 minutes per group)
23. Thursday 2.2: Exercise presentation, set 2 (15-30 minutes per group)

Resources

• OpenMP standard and documents
• MPI standard and documents
• Saturn and Jupiter
• Getting account on saturn: TU Wien 48-core AMD Opteron shared-memory node
• Login in to jupiter: 36-node InfiniBand-AMD distributed memory cluster
• Some themes for bachelor and master theses

Standards

• OpenMP 3.1 Specification
• MPI 2.2 Standard

Programs

• mpihello.c

  Compile with mpicc -o mpihello -O3 mpihello.c, execute with mpirun -np 4 ./mpihello

• openmphello.c

  Compile with gcc -Wall -fopenmp -o openmphello -O3 openmphello.c, execute with ./openmphello -t 10

• pthreadshello.c

  Compile with gcc -Wall -o pthreadshello -O3 Opthreadshello.c -pthread, execute with ./pthreadshello -t 25

Additional Literature

• Thomas Rauber, Gudula Rünger: Parallel Programming for Multicore and Cluster Systems. Springer 2010.

  Good overview, complements lecture well, also with topics/applications not in lecture.

• Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar: Introduction to Parallel Computing. Pearson Addison-Wesley 2003.

  Good selection of material, good coverage. Beware, many detail mistakes/typos.