Skip to content
Latchkey
Speed Up & Optimize CI

Make pipelines faster, cheaper, and more reliable.

Actionable playbooks to speed up CI/CD: caching, parallelization, test splitting, Docker layer caching, and reliability patterns. Plus step-by-step how-tos for the things teams set up most.

Speed up CI

Caching, parallelism, test splitting, slim images.

Stop re-downloading depsStop CI from re-downloading the same dependencies every run. Cache package stores, pin lockfiles, use offline… Avoid redundant buildsUse path filters and changed-file detection in GitHub Actions so jobs only run when relevant files change, sk… Cache Bazel remotelySet up a Bazel remote cache in CI so unchanged targets are downloaded instead of rebuilt. Configure the cache… Cache browser binariesStop re-downloading Chromium, Firefox, and WebKit every run. Cache Playwright and Cypress browser binaries in… Cache Bundler gemsCache Bundler gems in GitHub Actions so Ruby CI stops compiling native extensions every run. Use the setup-ru… Cache ComposerCache Composer dependencies in GitHub Actions so PHP installs stop re-downloading packages. Cache the Compose… Cache Docker layersSpeed up Docker builds in GitHub Actions with BuildKit registry cache - cache-from/cache-to, gha cache backen… Cache Go modulesCache Go modules and the build cache in GitHub Actions so go build and go test stop re-downloading and recomp… Cache Gradle/MavenCache Gradle and Maven dependencies in GitHub Actions to skip re-downloading JARs every run. Cache keys, the… Cache pip/poetryCache pip and Poetry dependencies in GitHub Actions to skip reinstalling Python packages every run. setup-pyt… Cache Rust/cargoCache Cargo registry, git deps, and the target directory in GitHub Actions so Rust CI stops recompiling the w… Cache Maven .m2Cache the Maven ~/.m2/repository in GitHub Actions so JVM builds stop re-downloading JARs every run. Use the… Cache the .next buildCache the Next.js .next build cache in CI so incremental builds reuse compiled output. Cache the right direct… Cache pnpm storeCache the pnpm content-addressable store in GitHub Actions so installs reuse already-downloaded packages. The… Cancel superseded runsUse concurrency groups in GitHub Actions to cancel in-progress runs when a newer commit lands, freeing runner… Compress test artifactsCompress test reports, coverage, traces, and screenshots before upload so CI artifacts are small and fast. Pi… Dockerize CI buildRun your CI build inside a Docker image so the toolchain is pinned and reproducible. Build a CI image, cache… Fail fastOrder CI to fail fast: run cheap checks first, enable matrix fail-fast, and stop a doomed pipeline early so y… Optimize monorepo CIMake monorepo CI fast: affected-project detection, sparse checkout, per-package caching, and a remote cache s… Parallelize a monorepoParallelize a monorepo build in CI by fanning out packages across jobs, respecting the dependency graph, and… Parallelize by timingSplit tests across parallel CI jobs by historical runtime, not file count, so every shard finishes at the sam… Prune Docker imagesFree CI disk by pruning Docker images, build cache, and dangling layers. Stop "no space left on device" failu… Reduce artifact sizeShrink CI artifacts so upload and download are fast and storage stays cheap. Upload only what you need, exclu… Reduce artifact upload timeCut artifact upload and download time in GitHub Actions: upload fewer files, compress, scope paths, and lower… Faster big-repo checkoutCut checkout time on a large repository in CI: shallow clone, sparse checkout, blob filtering, and partial fe… Reduce checkout timeSpeed up git checkout in CI with shallow clones, sparse checkout, and skipping submodules so large repos stop… Reduce cold startsCut CI cold-start time: warm dependency caches, prebuilt images, and warm-pool runners that skip the provisio… Warm runnersCut CI cold-start latency with warm runner pools: keep runners and caches pre-provisioned so jobs start in se… Reduce Docker image sizeShrink CI Docker images with multi-stage builds, slim base images, .dockerignore, and fewer layers so pushes… Reduce npm install timeCut npm install time in GitHub Actions: use npm ci, cache the npm cache or node_modules, prune dev deps, and… Right-size CPUPick the right CPU count for a CI build instead of guessing. Profile where the build is CPU-bound, match para… Run affected testsRun only the tests affected by a change in CI instead of the whole suite. Use git diff scoping, monorepo affe… Cache Nx/TurborepoEnable remote caching for Nx and Turborepo in CI so unchanged tasks are restored instead of rerun. Configure… Shallow submodulesSpeed up CI checkout when you use git submodules: shallow-fetch submodules, recurse only what you need, and a… Share caches across branchesUnderstand GitHub Actions cache scoping and share dependency caches across branches by building on main and u… Skip CI for docsStop running the full CI suite on docs-only changes. Use path filters and required-check shims so README edit… Speed up CypressMake Cypress tests fast in CI: parallelize across containers, cache the Cypress binary, group specs, and stop… Speed up Docker buildsMake Docker image builds in CI fast: BuildKit layer caching, registry cache, multi-stage builds, ordering lay… Speed up GitHub ActionsMake GitHub Actions faster: caching, parallelization, test splitting, Docker layer caching, slimmer images, a… Speed up JestMake Jest fast in CI: tune workers, shard the suite, enable the Jest cache, and run only tests affected by th… Speed up PlaywrightMake Playwright fast in CI: shard across workers, cache browsers, reuse a single web server, and trim trace/v… Speed up pytestMake pytest fast in CI: parallelize with pytest-xdist, split by timing, cache the test database, and run only… Split a slow test suiteBreak a long-running test suite into parallel CI shards: choose a split strategy, fan out with a matrix, and… Use a merge queueUse GitHub merge queue to batch and serialize merges, cutting redundant CI runs and eliminating the broken-ma… Multi-stage buildsUse multi-stage Docker builds to keep CI fast: build in a fat stage, ship a slim runtime stage, and cache the… Path-filtered matrixBuild a CI matrix that only includes the parts of the repo a change touched. Detect changed paths, generate a… Use a remote build cacheSet up a remote build cache for Bazel, Turborepo, Nx, or Gradle so CI and developer machines share compiled a… Smaller base imagePick a smaller base image to cut CI build and pull time. Compare slim, alpine, and distroless, and avoid the… Cache test resultsCache passing test results in CI so unchanged tests are not rerun. Hash test inputs, store the verdict, and r… Cache Gradle buildUse the Gradle build cache and configuration cache in CI so unchanged tasks are skipped and outputs reused. E… Warm dependency cachesPre-populate CI dependency caches from your default branch so pull-request runs always hit a warm cache inste…

Make CI reliable

Flaky tests, retries, and self-healing patterns.

Batch dependent jobsStructure dependent CI jobs so they batch efficiently: combine cheap steps, share setup across a job, and avo… Make CI reliableStop flaky GitHub Actions: isolate nondeterminism, quarantine flaky tests, add targeted retries, and use self… Reduce flaky-test rerunsCut flaky-test reruns in CI: quarantine known flakes, fix shared state and timing, retry only what is genuine… Retry a failed step/jobHow to retry failed GitHub Actions steps and jobs - manual re-runs, retry actions with backoff, and automatic… Schedule jobs off-peakMove heavy CI jobs off the critical path by scheduling them off-peak. Use cron triggers, nightly suites, and… Set sensible timeoutsSet job and step timeouts in GitHub Actions so a hung process fails fast instead of burning the full 6-hour l… Job-level concurrencyUse GitHub Actions concurrency groups to cancel superseded runs, serialize deploys, and stop wasted CI minute… Spot vs managed runnersCompare spot, self-hosted, and managed runners for CI cost and reliability. When spot interruptions hurt, and… Works locally, fails in CITests pass locally but fail in CI? The usual causes - environment differences, missing deps, case sensitivity…

How-to guides

OIDC, secrets, matrix, services, reusable workflows.

Batch matrix jobsStop matrix explosion in GitHub Actions: prune combinations with include/exclude, batch versions per job, and… Cache dependenciesCache dependencies in GitHub Actions with actions/cache - correct keys and paths for npm, yarn, pnpm, pip, Ma… Cut macOS minutesReduce expensive macOS GitHub Actions minutes: move non-Apple work to Linux, cache aggressively, and reserve… Debug a workflowDebug GitHub Actions: enable step debug logging, re-run with SSH/tmate, read the real error, and reproduce lo… Lower idle runner costStop paying for idle self-hosted runners: autoscale to zero, right-size instances, use spot capacity, and wei… Cut your CI billMeasure your GitHub Actions bill by runner OS and workflow, find where minutes go, and cut spend with caching… Reduce Windows CI costLower Windows GitHub Actions cost: move cross-platform jobs to Linux, cache and speed up slow Windows steps,… Right-size CI runnersPick the right CI runner size: read CPU, memory, and disk usage from real runs, then match the runner to the… Self-hosted runner setupSet up a self-hosted GitHub Actions runner - registration, labels, security, and why managed runners are usua… Matrix buildsUse GitHub Actions matrix builds to test across versions and OSes in parallel - strategy.matrix, include/excl… OIDC to AWSAuthenticate GitHub Actions to AWS with OIDC - no long-lived secrets. Set up the IAM role, trust policy, and… Use secrets safelyUse GitHub Actions secrets correctly - repo vs environment vs org secrets, why forks can not read them, and h… When to use bigger runnersDecide when a bigger CI runner actually pays off: CPU-bound builds and parallel tests benefit; I/O- and netwo…

Diagnose CI

Why is my CI slow, expensive, or flaky.

Stuck on "queued"GitHub Actions jobs stuck on "queued" and not starting? The causes - outages, concurrency, runner labels, bil… Debug a hanging jobFind why a CI job hangs until timeout: open handles, waiting on stdin, deadlocked processes, and unflushed se… Debug why CI is slowA systematic checklist to debug slow GitHub Actions: separate queue from run time, find the critical path, ch… Find a memory leakTrack down a memory leak that only shows up in CI: capture heap usage over the run, take snapshots, and isola… Find the slowest jobPinpoint the slowest job and step in a GitHub Actions workflow using the jobs API and step timings, so you op… Measure cache hit rateMeasure your CI cache hit rate so you know whether caching actually helps. Read cache-hit outputs, time resto… Measure CI durationMeasure GitHub Actions duration accurately: queue time vs run time, per-job timing from the API, and trends o… Profile a workflowProfile a GitHub Actions workflow end to end: per-step timing, the critical path of needs dependencies, cache… Profile a slow suiteFind what makes a test suite slow in CI: per-test timing, slow setup and teardown, serialized I/O, and the lo… Why is my CI slow?Diagnose slow GitHub Actions: no caching, serial tests, cold starts, oversized Docker context, and chatty tri…
Explore other topics
GitHub ActionsWorkflow, runner, and YAML errors - diagnosed and fixed. Node.js & npmnpm, yarn, and pnpm failures in CI - solved. DockerBuild, run, compose, and registry errors - explained. Pythonpip, poetry, venv, and pytest failures - fixed. Java & JVMMaven, Gradle, and JVM failures in CI - resolved. GoGo build, module, and test failures - diagnosed.