Introduction to CUDA Programming

This repository provides an introduction to CUDA programming using C. It covers the fundamentals of parallel programming with NVIDIA’s CUDA platform, including concepts such as GPU architecture, memory management, kernel functions, and performance optimization. The materials are designed for beginners and include step-by-step tutorials, practical examples, and exercises to help you get started with writing and running CUDA programs in C.

Note

This project is under active development.

Contents

• Prerequisite
• Learning Outcomes
• Tutorial
  • Graphics Processing Unit (GPU)
    • GPU Overview
    • CPU vs GPU
    • GPU Architecture
    • Streaming Multiprocessors (SMs)
    • CUDA Cores
    • L1 Cache (Level 1)
    • L2 Cache (Level 2)
    • Difference Between Shared Memory and DRAM
    • How They Work Together
  • Exercise 1
  • GPU Execution Model
    • CUDA Kernels vs Functions
      • CUDA Kernel
      • CUDA Device Function
      • Host Function
    • Execution Model
    • Thread and Block Indexing
      • Computing Global Thread Index
    • Why Global Thread Index Matters
  • GPU Workflow
    • Kernel Definition
    • Memory Management
    • Common CUDA Error Codes
  • Exercise 2
  • Understanding Warps in CUDA
    • What is a Warp?
    • Why Use Warps?
    • Thread Grouping into Warps
    • Execution Behavior
    • Warp Divergence
    • Some additinal details
  • Asynchronous CUDA Calls
    • CUDA Streams
    • Why streams result in more concurrency?
    • Main differences from the synchronous version
  • Exercise 3
  • Shared Memory in CUDA
    • Shared Memory Characteristics
  • CUDA Events
    • CUDA Event Basics
    • Cross stream Synchronization
    • Timiming events across streams
  • Exercise 4
  • Unified Memory
    • Pinned Memory in Unified Memory
    • cudaMemPrefetchAsync
    • cudaMemAdvise
  • Exercise 5
  • Dynamic Parallelism in CUDA
    • Compilation Requirements
    • Code Example
  • Exercise 6
  • Finding Optimal GPU Occupancy
    • cudaOccupancyMaxPotentialBlockSize()
    • cudaOccupancyMaxActiveBlocksPerMultiprocessor
  • Exercise 7
  • CUDA Graphs
    • CUDA Graph Basics
    • Components of CUDA Graphs
    • Custom CUDA Graphs
    • Updating node parameters
  • Exercise 8
  • Memory Pool
    • Default Memory Pool
    • Custom Memory Pool
      • Create a memory pool
      • Set attributes (optional)
      • Use the memory pool
      • Pool trimming
  • Exercise 9
  • cuBLAS
    • Matrix Multiplication using cuBLAS
      • Column and Row Major Order
      • cuBLAS Handle
      • cuBLAS Operations
  • Exercise 10
  • Multi-GPU Workflow
    • cudaGetDeviceCount
    • cudaSetDevice
    • cudaMemcpyDeviceToDevice
  • Exercise 11
  • Profile using Nsight Systems
    • NVIDIA Nsight Systems (nsys)
  • Exercise 12
  • Debug using cuda-gdb
  • Exercise 13
• Reference
• Contributers