TensorFlow "Could not load dynamic library 'libcudnn.so.8'" in CI

TensorFlow tried to load cuDNN (libcudnn.so.8) and the dynamic linker could not find it. cuDNN is a separate NVIDIA library from the CUDA toolkit, and TF needs the specific major version its build was compiled against.

What this error means

TensorFlow logs "Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file" and then runs on CPU or fails GPU ops.

Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open
shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda/lib64

Common causes

cuDNN is not installed on the runner

The CUDA toolkit alone does not include cuDNN. Without the separate cuDNN package, TF cannot load libcudnn.so.8.

cuDNN major version mismatch

TF compiled against cuDNN 8 cannot load a cuDNN 9 install (or vice versa); the soname .so.8 must be present.

How to fix it

Install the matching cuDNN

Install the cuDNN version TensorFlow expects, either via the NVIDIA pip wheel or the system package, and put it on the loader path.

pip install nvidia-cudnn-cu11
export LD_LIBRARY_PATH=$(python -c "import nvidia.cudnn, os; print(os.path.dirname(nvidia.cudnn.__file__))")/lib:$LD_LIBRARY_PATH

Use the tensorflow[and-cuda] extra

Installing the CUDA extra pulls a known-compatible CUDA and cuDNN set for that TF version.

pip install "tensorflow[and-cuda]"

How to prevent it

• Install cuDNN explicitly; the CUDA toolkit does not include it.
• Match the cuDNN major version to the TensorFlow build.
• Prefer tensorflow[and-cuda] so the CUDA/cuDNN set is consistent.

Related guides