How to Load Hugging Face Datasets in GitHub Actions

Set HF_TOKEN from a secret and point HF_HOME at a cached directory so gated datasets load and repeated runs reuse the download.

Export HF_TOKEN from a repository secret for gated or private repos, set HF_HOME to a cacheable path, then load with the datasets library. Combine with actions/cache to avoid re-downloading.

Steps

• Add the Hugging Face token as a repository secret.
• Export HF_TOKEN and set HF_HOME to a cache directory.
• Load the dataset with datasets.load_dataset.

Workflow

jobs:
  data:
    runs-on: ubuntu-latest
    env:
      HF_TOKEN: ${{ secrets.HF_TOKEN }}
      HF_HOME: ${{ github.workspace }}/.hf
    steps:
      - uses: actions/checkout@v4
      - uses: actions/cache@v4
        with:
          path: ${{ github.workspace }}/.hf
          key: hf-home-${{ hashFiles('data.lock') }}
      - run: pip install datasets && python load_data.py

Gotchas

• Gated datasets fail without a token even when the repo is public to browsers.
• Set HF_HUB_OFFLINE=1 on later steps to prove the cache is populated.

Related guides