Skip to content
Latchkey

TensorFlow "DNN library is not found" in CI

TensorFlow needs cuDNN for GPU neural-network primitives, and it could not find a usable cuDNN. Unlike a load-warning, this hard error stops the op because no DNN backend is available at all.

What this error means

A GPU op fails with "Node ... defined ... DNN library is not found." The job may have logged a libcudnn load warning earlier.

python
tensorflow.python.framework.errors_impl.UnimplementedError: DNN library is not found.
[[{{node sequential/conv2d/Conv2D}}]]

Common causes

cuDNN is absent or unloadable

No libcudnn is installed, or the one present cannot be loaded, so TF has no DNN backend for the GPU.

cuDNN not on the loader path

cuDNN is installed via pip but its lib directory is not on LD_LIBRARY_PATH, so TF cannot find it.

How to fix it

Install cuDNN and expose it to the loader

  1. Install the cuDNN wheel matching the TF CUDA version.
  2. Add its lib directory to LD_LIBRARY_PATH.
  3. Confirm with the TF build info that GPU support is active.
Terminal
pip install "tensorflow[and-cuda]"
python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

Verify cuDNN loads before training

Probe the library directly so a missing DNN backend fails fast with a clear message.

Terminal
python -c "import ctypes; ctypes.CDLL('libcudnn.so.8')"

How to prevent it

  • Install cuDNN alongside CUDA for every GPU TensorFlow job.
  • Add the cuDNN lib directory to LD_LIBRARY_PATH in CI.
  • List physical GPUs in a setup step to confirm the backend.

Related guides

Tired of flaky CI? Latchkey auto-heals failed jobs and retries them for you. Start free →