Tutorial

This hands-on CUDA workshop is designed to introduce developers, researchers, and engineers to GPU programming using NVIDIA CUDA. Through a combination of theory, practical examples, and guided exercises, participants will learn how to harness the computational power of GPUs to accelerate their applications.

Basic Topics:

Topics	Duration
GPU Overview	30 minutes
Exercise 1	20 minutes
GPU Execution Model	60 minutes
GPU Workflow	60 minutes
Exercise 2	20 minutes
Asynchronous CUDA Calls	60 minutes
Exercise 3	20 minutes
Shared Memory	30 minutes
CUDA Events	30 minutes
Exercise 4	20 minutes
Unified Memory	30 minutes
Exercise 5	20 minutes

• Graphics Processing Unit (GPU)
  • GPU Overview
  • CPU vs GPU
  • GPU Architecture
  • Streaming Multiprocessors (SMs)
  • CUDA Cores
  • L1 Cache (Level 1)
  • L2 Cache (Level 2)
  • Difference Between Shared Memory and DRAM
  • How They Work Together
• Exercise 1
• GPU Execution Model
  • CUDA Kernels vs Functions
    • CUDA Kernel
    • CUDA Device Function
    • Host Function
  • Execution Model
  • Thread and Block Indexing
    • Computing Global Thread Index
  • Why Global Thread Index Matters
• GPU Workflow
  • Kernel Definition
  • Memory Management
  • Common CUDA Error Codes
• Exercise 2
• Understanding Warps in CUDA
  • What is a Warp?
  • Why Use Warps?
  • Thread Grouping into Warps
  • Execution Behavior
  • Warp Divergence
  • Some additinal details
• Asynchronous CUDA Calls
  • CUDA Streams
  • Why streams result in more concurrency?
  • Main differences from the synchronous version
• Exercise 3
• Shared Memory in CUDA
  • Shared Memory Characteristics
• CUDA Events
  • CUDA Event Basics
  • Cross stream Synchronization
  • Timiming events across streams
• Exercise 4
• Unified Memory
  • Pinned Memory in Unified Memory
  • cudaMemPrefetchAsync
  • cudaMemAdvise
• Exercise 5

Advanced Topics:

Topics	Duration
Dynamic Parallelism	30 minutes
Exercise 6	20 minutes
Occupancy	20 minutes
Exercise 7	20 minutes
CUDA Graphs	60 minutes
Exercise 8	45 minutes
Memory pools	45 minutes
Exercise 9	30 minutes
cuBLAS	30 minutes
Exercise 10	30 minutes
Multi-GPU Workflow	20 minutes
Exercise 11	15 minutes
Profiling using Nsight Systems	15 minutes
Exercise 12	20 minutes
Debugging using cuda-gdb	15 minutes
Exercise 13	15 minutes

• Dynamic Parallelism in CUDA
  • Compilation Requirements
  • Code Example
• Exercise 6
• Finding Optimal GPU Occupancy
  • cudaOccupancyMaxPotentialBlockSize()
  • cudaOccupancyMaxActiveBlocksPerMultiprocessor
• Exercise 7
• CUDA Graphs
  • CUDA Graph Basics
  • Components of CUDA Graphs
  • Custom CUDA Graphs
  • Updating node parameters
• Exercise 8
• Memory Pool
  • Default Memory Pool
  • Custom Memory Pool
    • Create a memory pool
    • Set attributes (optional)
    • Use the memory pool
    • Pool trimming
• Exercise 9
• cuBLAS
  • Matrix Multiplication using cuBLAS
    • Column and Row Major Order
    • cuBLAS Handle
    • cuBLAS Operations
• Exercise 10
• Multi-GPU Workflow
  • cudaGetDeviceCount
  • cudaSetDevice
  • cudaMemcpyDeviceToDevice
• Exercise 11
• Profile using Nsight Systems
  • NVIDIA Nsight Systems (nsys)
• Exercise 12
• Debug using cuda-gdb
• Exercise 13