CUDA "no kernel image is available for execution on the device" in CI

The CUDA binaries loaded fine but contain no compiled kernel for this GPU's compute capability (SM architecture). The wheel or your own extension was built for a different set of architectures than the runner's card.

What this error means

A CUDA kernel launch fails with "CUDA error: no kernel image is available for execution on the device". The code runs on one GPU model but fails on the CI runner's GPU.

python

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the
stacktrace below might be incorrect.

Common causes

Binaries built for a different compute capability

The PyTorch/TensorFlow wheel or your custom extension was compiled for SM architectures that do not include the runner GPU's capability (for example a build targeting sm_70..sm_86 running on a newer sm_90 card, or an old wheel that predates the GPU).

A custom extension built with a narrow TORCH_CUDA_ARCH_LIST

A source build of a CUDA extension defaulted to the build host's architecture only, so it has no kernel for the GPU the tests actually run on.

How to fix it

Install a build that targets the runner GPU

Read the GPU's compute capability with nvidia-smi --query-gpu=compute_cap --format=csv.
Install a PyTorch/TF build whose CUDA version supports that architecture (newer GPUs need newer toolkits).
Re-run the kernel to confirm a matching image now exists.

Terminal

nvidia-smi --query-gpu=name,compute_cap --format=csv
pip install --index-url https://download.pytorch.org/whl/cu121 torch

Build extensions for the right architectures

When compiling a custom CUDA extension, set the architecture list to include the runner GPU before building.