How to Install sentence-transformers in CI Without OOM

sentence-transformers is light Python on top of torch and transformers - the CI pain is the heavy backend, multi-GB model downloads, and OOM when an embedding model loads.

The sentence-transformers package wraps PyTorch and Hugging Face transformers to produce embeddings. It installs from a wheel, but pulls a deep-learning backend and downloads embedding-model weights at runtime - so CI breaks on the wrong backend, repeated downloads, and OOM.

Why it fails in CI

sentence-transformers needs PyTorch; installing the default CUDA torch on a CPU runner wastes gigabytes. Embedding models (e.g. all-MiniLM) download at first use and re-download every run if the HF cache is not persisted. Loading a large model on a small runner triggers an OOM kill.

ImportError / no backend - torch not installed before sentence-transformers.
Multi-GB model re-download each run because HF_HOME is not cached.
Exit 137 (OOM) loading a large embedding model on a small runner.

Install it reliably

Install CPU torch from the PyTorch index first, then sentence-transformers. Set HF_HOME to cache model downloads and use a small model (e.g. all-MiniLM-L6-v2) in tests.

Terminal

# CPU torch (not the multi-GB CUDA build) + sentence-transformers
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install sentence-transformers

# cache model downloads across runs; go offline once cached
export HF_HOME="$RUNNER_TEMP/hf-cache"
export HF_HUB_OFFLINE=1

Cache & speed

Cache both ~/.cache/pip and the Hugging Face model cache (HF_HOME) keyed on the model you load, so weights download once. Right-size the runner for model loads instead of retrying OOM kills.

.github/workflows/ci.yml

- uses: actions/cache@v4
  with:
    path: |
      ~/.cache/pip
      ${{ runner.temp }}/hf-cache
    key: st-${{ hashFiles('requirements*.txt') }}

Common errors

Error	Cause	Fix
No PyTorch backend	torch not installed	pip install torch (CPU index)
Exit 137 loading a model	OOM-killed	Bigger runner; use a small model
Re-downloads weights every run	HF cache not persisted	Set + cache HF_HOME
Network errors in offline tests	Hub access during tests	Set HF_HUB_OFFLINE=1 after caching

Key takeaways

Install CPU torch explicitly before sentence-transformers on non-GPU runners.
Set and cache HF_HOME; use a small embedding model in tests.
Exit 137 means OOM - right-size the runner.