How to Retry a Failed JUnit Test in CI

On the JVM you retry either at the runner level with Surefire rerunFailingTestsCount or inside the test with a JUnit 4 RetryRule.

The simplest path is Maven Surefire rerunFailingTestsCount, which re-runs only failed tests. For fine control, implement a JUnit 4 TestRule that retries a statement a fixed number of times.

Steps

For a quick fix, set rerunFailingTestsCount in the Surefire plugin.
For per-test control, add a RetryRule implementing TestRule.
Report rerun tests so a flake is visible, not silently green.

Surefire rerun config

pom.xml

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <rerunFailingTestsCount>2</rerunFailingTestsCount>
  </configuration>
</plugin>

A JUnit 4 RetryRule

RetryRule.java

public class RetryRule implements TestRule {
  private final int retries;
  public RetryRule(int retries) { this.retries = retries; }
  public Statement apply(Statement base, Description d) {
    return new Statement() {
      public void evaluate() throws Throwable {
        Throwable last = null;
        for (int i = 0; i <= retries; i++) {
          try { base.evaluate(); return; }
          catch (Throwable t) { last = t; }
        }
        throw last;
      }
    };
  }
}

Gotchas

Surefire reruns are reported as "flakes", not passes; keep an eye on that section.
A RetryRule that swallows the first failure hides a real bug; log each attempt.