How to Build a Fan-In Fan-Out Workflow in CircleCI

The workflow requires key wires job dependencies, so you fan out from one job into many parallel jobs, then fan in to a job that waits for all of them.

List a job under requires: to make it wait for its dependencies. Multiple jobs that all require the same upstream job run in parallel (fan-out); a job that requires several jobs waits for every one (fan-in).

Steps

• Define one upstream job (for example build) with no requires.
• Give several jobs requires: [build] so they all start together after build.
• Give a final job requires: [lint, unit, e2e] so it runs only after all three finish.

Config

version: 2.1
workflows:
  pipeline:
    jobs:
      - build
      - lint:
          requires: [build]
      - unit:
          requires: [build]
      - e2e:
          requires: [build]
      - deploy:
          requires: [lint, unit, e2e]

Gotchas

• If the fan-in job needs files from the parallel jobs, pass them with persist_to_workspace and attach_workspace, not the filesystem.
• A circular requires chain fails config validation with a requires cycle error; keep the graph acyclic.

Related guides