How to Retry Only on Infrastructure Errors in CI

Retrying every failure hides real bugs, so scope retries to transient error types and let assertion failures fail immediately.

Most runners can limit retries to matching errors: pytest-rerunfailures --only-rerun, rspec-retry exceptions_to_retry, or a custom predicate. Match connection resets and timeouts, never AssertionError.

Steps

List the transient error types worth retrying (timeouts, resets).
Configure the runner to retry only those.
Ensure assertion failures are excluded so bugs surface fast.

Match transient errors only

Terminal

# pytest: retry only connection-style errors
pytest --reruns 2 --only-rerun 'ConnectionError|ReadTimeout'

RSpec exception allowlist

spec/spec_helper.rb

RSpec.configure do |c|
  c.default_retry_count = 2
  c.exceptions_to_retry = [Net::ReadTimeout, Errno::ECONNRESET]
end

Gotchas

Never retry AssertionError; a flaky assertion is a bug the retry will bury.
Keep the transient list short and specific so it does not silently grow to cover real failures.

Related guides

How to Choose Job-Level vs Test-Level Retry in CIChoose between job-level and test-level retry in CI, retrying a whole job for infrastructure flakiness and in… How to Avoid the Cost of Blind Retries in CIAvoid the cost of blind retries in CI, where retrying every failure masks real bugs and burns minutes, by tra… How to Retry a Failed RSpec Example in CIRetry a failing RSpec example in CI with the rspec-retry gem, configuring retry count globally and tagging in…