Flaky Test Cost Calculator
Every flaky failure burns CI minutes you pay for and engineer time you cannot bill back. See the real number.
Flaky tests are not free. Each transient failure triggers a re-run that costs CI minutes, and it pulls an engineer off their work to babysit the retry. This calculator multiplies your flaky-failure rate by both costs to show the monthly and yearly total - and what a self-healing runner removes.
Flaky incidents/month = builds/day x %flaky x 30. Wasted CI minutes are billed at the Linux rate of $0.008/min. Human cost = lost minutes x ($/hour / 60). A managed runner that auto-retries transient failures makes these re-runs disappear.
The two costs of a flaky test
- Wasted CI minutes - the re-run you pay GitHub for, billed here at the $0.008/min Linux rate.
- Wasted engineer time - the human minutes lost noticing, re-running, and context-switching, priced at your hourly cost.
- Both scale with volume: flaky incidents/month = builds/day x %flaky x 30.
Self-healing removes most of this
Latchkey managed runners automatically retry transient and infrastructure-level failures, so the re-run never reaches your team. The CI minutes and the engineer interruptions in this calculator largely disappear.