PowerShell Out-File default encoding (UTF-16) breaks tools in CI
By Daniel Zoghalchali·Latchkey
Windows PowerShell 5.1 writes UTF-16LE with a BOM by default from Out-File and > redirection. Tools expecting UTF-8 then choke on the BOM or the null bytes, failing in confusing ways.
What this error means
A file written in one step is rejected by a parser in the next step: a YAML/JSON loader reports an unexpected character at the start, or a tool reads garbled text. Deterministic for the default encoding.
powershell
# config.json written via > then parsed:
SyntaxError: Unexpected token in JSON at position 0
# the leading bytes are FF FE (UTF-16 BOM), not '{'
Diagnose it: which runtime is actually on PATH?
Runners ship multiple versions of most runtimes and select one through a version manager. When a setup action and a version file disagree, the resulting error is about your code rather than the version.
Terminal
which -a <runtime>
<runtime> --version
echo "PATH=$PATH" | tr ":" "\n" | head -20
# does a version file in the repo disagree with the workflow?
cat .tool-versions .nvmrc .ruby-version .python-version 2>/dev/null
Common causes
Windows PowerShell 5.1 defaults to UTF-16
Out-File and > in 5.1 default to Unicode (UTF-16LE) with a BOM. Tools written for UTF-8 cannot parse it.
Even UTF-8 output may include a BOM
Some encoders emit a UTF-8 BOM that strict parsers reject at position 0.
How to fix it
Specify UTF-8 explicitly
Force UTF-8 on every write so downstream tools get the bytes they expect.
Always pass -Encoding utf8 (or utf8NoBOM on PS7) when writing files that other tools read, or set $PSDefaultParameterValues to default Out-File to UTF-8.
Frequently asked questions
What causes PowerShell Out-File default encoding (UTF-16) breaks tools in CI?
There are 2 common causes: windows powershell 5.1 defaults to utf-16 and even utf-8 output may include a bom. Out-File and > in 5.1 default to Unicode (UTF-16LE) with a BOM.
How do I fix PowerShell Out-File default encoding (UTF-16) breaks tools in CI?
There are 2 fixes depending on which cause you have: specify utf-8 explicitly and write bytes without a bom. Work through them in order, since the first is the most common.
What does PowerShell Out-File default encoding (UTF-16) breaks tools in CI actually mean?
A file written in one step is rejected by a parser in the next step: a YAML/JSON loader reports an unexpected character at the start, or a tool reads garbled text.
How do I stop PowerShell Out-File default encoding (UTF-16) breaks tools in CI happening again?
Always pass -Encoding utf8 (or utf8NoBOM on PS7) when writing files that other tools read, or set $PSDefaultParameterValues to default Out-File to UTF-8.
