Hugging Face datasets "trust_remote_code" required in CI
load_dataset found a custom loading or processing script in the dataset repo. Newer datasets refuses to run repo-provided code unless you explicitly pass trust_remote_code=True, so the load aborts.
What this error means
load_dataset fails with "ValueError: The repository for X contains custom code which must be executed to correctly load the dataset. ... Please pass the argument trust_remote_code=True."
ValueError: The repository for squad contains custom code which must be executed to
correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/...
Please pass the argument `trust_remote_code=True` to allow custom code to be run.Common causes
The dataset ships a loading script
Script-based datasets execute repo code to build splits. Recent datasets versions block that by default for safety.
A datasets upgrade changed the default
Older code that loaded the dataset fine now fails because the library tightened the default to not trust remote code.
How to fix it
Opt in after reviewing the script
Inspect the dataset repo, then pass trust_remote_code=True if you accept running its code.
from datasets import load_dataset
ds = load_dataset("some/dataset", trust_remote_code=True)Prefer a no-script dataset format
Use a dataset stored as Parquet/CSV/JSON (no loading script) so no remote code execution is needed.
ds = load_dataset("parquet", data_files="data/*.parquet")How to prevent it
- Prefer script-free dataset formats in CI.
- Pin the dataset revision so its script does not change under you.
- Set
trust_remote_codeconsciously after reviewing the repo.