You can become it in a week! Supercomputer Programmer
Repository (kaityo256/sevendayshpc)
HTML version
Bulk PDF version
Introduction
Day 1: Building an environment
Torie Azu prepares an environment where you can use MPI on your PC and try out simple MPI programming.
- What is MPI?
- Aside: Is MPI difficult?
- Installing MPI
- My first MPI
- Rank
- About standard output
- Debugging MPI programs with GDB
Day 2: How to use a supercomputer
Things to know when using a supercomputer. How to throw jobs, etc.
- Introduction
- What is a supercomputer?
- Aside: BlueGene/L memory error
- How to get a supercomputer account
- How job execution works
- How to write a job script
- Fair share
- Backfill
- Chain Job
- Staging
- Parallel File Systems
Day 3: Self-evident parallel
About the self-evident parallelism, commonly known as "Silly Parallel."
- What is a self-evident parallel, or even called stupid para?
- Example 1 of trivial parallelism: Circularity
- Self-evident parallel template
- Trivial Parallel Example 2: Handling Many Files
- Example 3 of trivial parallelism: Statistical processing
- Parallelization efficiency
- Difference between sample parallel and parameter parallelism
Day 4: Non-trivial parallelism by area division
As an example of non-trivial parallelism, we will divide the one-dimensional thermal conduction equations into regions.
- Non-trivial parallel
- One-dimensional diffusion equation (serial version)
- One-dimensional diffusion equation (parallel version)
- Aside: Eager and Rendezvous protocols
Day 5: Two-dimensional reaction diffusion equation
As an example of a full-scale MPI program, we will divide the two-dimensional reaction diffusion equation into a region.
- Serial version
- Parallelization step 1: Preparing for communication, etc.
- Parallelization Step 2: Save the data
- Parallelization Step 2: Communications of Norishiro
- Parallelization Step 3: Implementing Parallel Code
- Aside: MPI's hassle
Day 6: Hybrid parallel
Hybrid parallelization using process parallel and thread parallelism. Things to be especially careful about when paralleling threads.
- What is hybrid parallel?
- Virtual Memory and TLB
- Aside: About TLB Miss
- NUMA
- OpenMP Example
- Performance evaluation
- Aside: Rock's story
- Examples of hybrid parallelism
Day 7: SIMD
Regarding SIMD conversion.
- Introduction
- What is SIMD?
- Try touching the SIMD register
- Aside: Assembly language? Assembly language?
- Simple SIMD example
- Aside: Handling of floating point arithmetics in x86
- A more practical SIMD version
Conclusion
license
Copyright (C) 2018-present Hiroshi Watanabe
This text and illustration (including pptx files) is provided in the Creative Commons 4.0 Attribution (CC-BY 4.0).
This article and pictures are licensed under a Creative Commons Attribution 4.0 International License.
The programs included in this repository are provided under the MIT license.
The source codes in this repository are licensed under the MIT License.
When creating the HTML version, github-markdown-css is used as CSS.
