Dev Blog

Parallel and distributed computing ​

Modern programs run on multicore processors, graphics accelerators, computing clusters, and in the cloud. This course teaches you to use that computing power: you progress from threads on a single computer to distributed applications deployed in a cluster, and learn to choose a parallelism model for a specific problem.

This course covers parallel and distributed computing — from multithreaded programming for shared-memory systems in C# on .NET 10 (synchronization, TPL, PLINQ, SIMD vectorization) and the design of parallel numerical algorithms to high-performance computing in C++ using OpenMP, CUDA, and MPI on a Linux computing cluster managed by Slurm. In the final stage, you build distributed applications using gRPC, RabbitMQ, and Microsoft Orleans and deploy them in Docker containers and Kubernetes clusters, concluding with microservice architecture design. Each course topic combines a lecture with a lab assignment.

Course program ​

Parallel programming in C# ​

  1. Fundamentals of parallel and distributed computing: system architectures, Flynn’s taxonomy, Amdahl’s and Gustafson’s laws; .NET 10 and JetBrains Rider
  2. Operating system processes and threads: scheduling, priorities, processor affinity; threads and the thread pool in C#
  3. Race conditions, deadlocks, and starvation; thread synchronization tools in C#
  4. Thread-safe collections, channels, and the producer–consumer pattern; false sharing
  5. Task parallelism: the TPL library, cancellation, and exception handling; asynchronous programming with async/await
  6. Data parallelism: parallel loops, PLINQ, data partitioning; parallel sorting and reduction algorithms
  7. SIMD vectorization in .NET and parallel linear algebra algorithms
  8. Models and levels of parallelism, grid systems; designing parallel algorithms for vectors, matrices, and numerical methods

High-performance computing and clusters ​

  1. Multithreaded programming in C++ and parallel algorithms in the standard library
  2. OpenMP: parallel regions and loops, reductions, tasks; thread affinity and NUMA
  3. GPU computing: GPU architecture, CUDA C++, and the ILGPU library for C#
  4. Distributed computing with MPI message passing: point-to-point and collective operations, hybrid MPI + OpenMP programs
  5. Building a Linux computing cluster with the Slurm job scheduler; configuring high-performance computing

Distributed systems and cloud technologies ​

  1. Distributed computing models: network sockets and remote procedure calls (WCF, CoreWCF, gRPC)
  2. Messaging through the RabbitMQ broker: work queues, publish–subscribe, delivery guarantees
  3. The actor model and the Microsoft Orleans framework; the CAP theorem and fault tolerance in distributed systems
  4. Containerizing and orchestrating distributed applications: Docker, Kubernetes, .NET Aspire
  5. Microservice architecture: system decomposition, an API gateway, data consistency (Saga, Outbox), and observability

Required software ​

SoftwarePurposeTopic
.NET 10 SDKC# 14 programs1
JetBrains RiderC# development environment1
WSL 2 with Ubuntu 26.04 LTSLinux for module 21
GCC 15, CMake, Ninja, GDB and TBB (sudo apt install build-essential cmake ninja-build gdb libtbb-dev)C++ tools in Ubuntu9
JetBrains CLionC++ development environment (WSL toolchain)9
The NVIDIA driver and the CUDA Toolkit 13 in WSLGPU programming; needs an NVIDIA graphics card with CC 7.5 or higher11
Open MPI 5 (openmpi-bin libopenmpi-dev)MPI programs12
Hyper-V and Ubuntu Server 26.04 LTS, Slurm, MUNGEYour own training cluster13
grpcurlTesting gRPC services14
Docker DesktopContainers (RabbitMQ, Redis, PostgreSQL)15
kind, kubectl and HelmKubernetes cluster17
Aspire CLIOrchestration and the monitoring dashboard17

The dotnet-counters and dotnet-dump diagnostic tools (topics 2–3) are installed with dotnet tool install, and libraries (BenchmarkDotNet, ILGPU, gRPC, Orleans and others) are added to projects as NuGet packages.

How a course is organized ​

A course is divided into topics. Each topic corresponds to one lecture and one lab assignment and has four parts:

  1. Lecture. The theory is split into chapters in reading order, with code examples, diagrams and program output.
  2. Practice. Worked examples with complete code that show how to do the lab assignment.
  3. Tasks. 30 variants of the lab assignment, each at three levels of difficulty: 1 – initial, 2 – basic, 3 – advanced. Every task is self-contained, so you can start right away at the level you choose.
  4. Summary. The topic’s conclusions, self-check questions and useful links.

To prepare for assessment, every course also has Review questions, Review tasks, Useful links and Recommended reading.

How to study ​

  1. Open your course and install the development environment named on the course page. The first topic of each course explains how, except “Object-oriented programming in C# II”, which uses the environment from the “Object-oriented programming in C# I” course.
  2. Read the lecture chapters in order. Type in and run the examples yourself rather than just reading them.
  3. On the Practice page, first try to solve the problem yourself, then compare your solution with the worked one.
  4. On the Tasks page, find your variant (your teacher gives you its number) and choose a level of difficulty.
  5. Check yourself with the questions on the Summary page, and only then move on to the next topic.
  6. Before the exam, go over the Review questions and solve a few Review tasks.

Navigation

The topics and their chapters are listed in the menu on the left (on a phone, open it with the “Menu” button). The “Next page” button at the bottom of every page follows the course program, and the “On this page” list on the right takes you to a section.

Course materials ​