Hugging Face datasets "trust_remote_code=True" required in CI
The dataset ships a Python loading script, and recent datasets versions refuse to execute repository code unless you explicitly pass trust_remote_code=True. In CI this fails hard because there is no interactive prompt to accept.
What this error means
load_dataset raises "ValueError: The repository for <name> contains custom code which must be executed to correctly load the dataset. ... Please pass the argument trust_remote_code=True to allow custom code to be run."
ValueError: The repository for my_dataset contains custom code which must be executed to
correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/my_dataset.
Please pass the argument `trust_remote_code=True` to allow custom code to be run.Common causes
The dataset uses a loading script
Script-based datasets run repository code to build splits; datasets now blocks that by default for safety.
No interactive confirmation is possible in CI
The prompt to allow code cannot be answered on a runner, so the call fails unless the flag is set in code.
How to fix it
Pass trust_remote_code after reviewing the script
- Inspect the dataset repository script so you trust what it runs.
- Pass
trust_remote_code=Trueto load_dataset. - Pin the dataset revision so the code cannot change under you.
from datasets import load_dataset
load_dataset("my_dataset", trust_remote_code=True, revision="v1.0")Prefer a script-free dataset
Where possible use a Parquet or data-file dataset that needs no code execution, avoiding the flag entirely.
load_dataset("parquet", data_files="data/*.parquet")How to prevent it
- Review and pin the dataset revision before trusting its code.
- Prefer data-file datasets that require no
trust_remote_code. - Set the flag in code, not via an interactive prompt.