Python "UnicodeDecodeError" from runner locale in CI

The runner's locale is C/POSIX, so Python defaults the filesystem and text encoding to ASCII. Reading a UTF-8 file then fails on the first non-ASCII byte.

What this error means

Code that reads files or stdout fails with "UnicodeDecodeError: 'ascii' codec can't decode byte ..." in CI but works locally where the locale is UTF-8.

python

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 12:
ordinal not in range(128)

Common causes

A C/POSIX locale on the runner

Without a UTF-8 locale, Python's default text encoding is ASCII, which cannot decode UTF-8 content.

Code relying on the ambient encoding

open() without encoding= uses the locale encoding, so the same code behaves differently across environments.

How to fix it

Force UTF-8 in the runner environment

Set a UTF-8 locale and enable Python UTF-8 mode for the job.

.github/workflows/ci.yml

env:
  LANG: C.UTF-8
  LC_ALL: C.UTF-8
  PYTHONUTF8: '1'

Specify encoding explicitly in code

Pass encoding="utf-8" to open() so behavior does not depend on the locale.

reader.py

with open(path, encoding="utf-8") as f:
    data = f.read()

How to prevent it

Set PYTHONUTF8=1 and a UTF-8 locale in CI.
Always pass encoding="utf-8" when reading text files.
Avoid depending on the ambient locale encoding.