How to Run puppeteer-cluster in CI Without Crashes

puppeteer-cluster runs many headless Chrome instances in parallel. In CI the parallelism is the problem: shared-memory limits, sandbox, and OOM on a small runner.

puppeteer-cluster pools many Puppeteer/Chrome instances to crawl or render concurrently. It inherits every headless-Chrome CI requirement, but the concurrency multiplies them: too many parallel browsers exhaust /dev/shm and memory, so the cluster crashes mid-run on a standard runner.

Why it fails in CI

Each browser needs the Chrome flags (--no-sandbox) and the shared-library set. Running N browsers at once exhausts the small default /dev/shm (causing tab crashes) and the runner’s memory (exit 137). Unbounded concurrency on a small runner is the core failure.

Target closed / tab crashes from a too-small /dev/shm.
Exit 137 (OOM) when concurrency outstrips the runner’s memory.
No usable sandbox! - Chrome in a container needs --no-sandbox.

Install & run it reliably

Pass the standard Chrome flags to the cluster’s launch options, cap maxConcurrency to what the runner can hold, and use --disable-dev-shm-usage (or a larger /dev/shm) so tabs do not crash.

crawler.js

// puppeteer-cluster launch options for CI
const cluster = await Cluster.launch({
  concurrency: Cluster.CONCURRENCY_CONTEXT,
  maxConcurrency: 2,                       // cap to the runner's capacity
  puppeteerOptions: {
    args: ['--no-sandbox', '--disable-dev-shm-usage'],
  },
})

Cache & speed

Cache the Puppeteer browser download (~/.cache/puppeteer) keyed on the Puppeteer version. The real speed lever is right-sizing concurrency to the runner: too high causes OOM retries that are slower than a sane cap.

.github/workflows/ci.yml

- uses: actions/cache@v4
  with:
    path: ~/.cache/puppeteer
    key: puppeteer-${{ hashFiles('**/package-lock.json') }}

Common errors

Target closed / tab crashes → add --disable-dev-shm-usage or enlarge /dev/shm.
Exit 137 (OOM) → lower maxConcurrency or use a bigger runner.
No usable sandbox! → add --no-sandbox to puppeteerOptions.args.
Missing libnss3.so/shared libs → install the Chromium system library set.

Key takeaways

puppeteer-cluster multiplies every headless-Chrome requirement by concurrency.
Add --no-sandbox and --disable-dev-shm-usage; cap maxConcurrency.
OOM (exit 137) means too much parallelism for the runner - lower it or size up.