cuda-gdb: Debug CUDA Kernels
cuda-gdb extends gdb to set breakpoints inside GPU kernels and inspect device threads, warps, and memory.
When a kernel misbehaves, cuda-gdb lets you step device code the way gdb steps host code. It needs device debug info, which changes how you build.
What it does
cuda-gdb is a CUDA-aware build of gdb. It can break inside kernels, switch focus between threads and warps with cuda thread/cuda block, and read device memory. Debugging device code requires compiling with -G to emit device debug information.
Common usage
# build with host (-g) and device (-G) debug info
nvcc -g -G kernel.cu -o kernel
cuda-gdb ./kernel
# inside the debugger
(cuda-gdb) break myKernel
(cuda-gdb) run
(cuda-gdb) cuda threadOptions
| Flag / command | What it does |
|---|---|
| nvcc -G | Emit device (kernel) debug info |
| nvcc -g | Emit host debug info |
| break <kernel> | Breakpoint inside a device kernel |
| cuda thread / cuda block | Switch focus to a specific GPU thread/block |
| CUDA_ENABLE_COREDUMP_ON_EXCEPTION=1 | Write a GPU core dump on a device fault |
In CI
For post-mortem debugging in headless CI, set CUDA_ENABLE_COREDUMP_ON_EXCEPTION=1 so a device exception writes a core file you can open later with cuda-gdb, instead of trying to attach interactively. Note -G disables device optimizations, so debug builds run much slower.
Common errors in CI
"No CUDA debugging information found" means the binary was built without -G. "Cannot enable CUDA debugging because another debugger is already attached" or a hang means an Nsight tool holds the device. On some setups cuda-gdb needs CUDA_VISIBLE_DEVICES or a driver that permits the debugger.