Kubernetes "Read-only file system" from readOnlyRootFilesystem in CI

With securityContext.readOnlyRootFilesystem: true, the container’s root filesystem is mounted read-only. Any write to a path not backed by a writable volume fails with Read-only file system, often crashing the app on startup.

What this error means

A container with a hardened security context crashes (CrashLoopBackOff) and its logs show Read-only file system (EROFS) when it tries to write a temp file, cache, or log under /tmp, /var, or the app dir.

OSError: [Errno 30] Read-only file system: '/app/cache/tmp.lock'
# or
nginx: [emerg] open() "/var/run/nginx.pid" failed (30: Read-only file system)

Common causes

The app writes to a read-only root path

Logs, caches, PID files, or scratch data written under the root filesystem fail when readOnlyRootFilesystem: true and no writable volume covers that path.

No emptyDir/tmpfs for scratch directories

Common writable paths like /tmp, /var/run, /var/cache need an explicit emptyDir (or other volume) mount; otherwise they inherit the read-only root.

How to fix it

Mount writable volumes for the paths that need writes

Keep the root read-only and add emptyDir volumes only where the app must write.

spec:
  containers:
    - name: app
      securityContext: { readOnlyRootFilesystem: true }
      volumeMounts:
        - { name: tmp, mountPath: /tmp }
        - { name: run, mountPath: /var/run }
  volumes:
    - { name: tmp, emptyDir: {} }
    - { name: run, emptyDir: {} }

Find every path the app writes

1. Reproduce the crash and read the EROFS path from the error.
2. Add an emptyDir mount for each writable directory (temp, cache, run, logs).
3. Configure the app to write logs to stdout/stderr instead of files where possible.

How to prevent it

• Enumerate writable paths and back each with a volume before enabling a read-only root.
• Prefer logging to stdout/stderr so no writable log path is needed.
• Test hardened security contexts in CI, not first in production.

Related guides