OutOfMemoryError: Java heap space in GitHub Actions

By Daniel Zoghalchali·Latchkey

OutOfMemoryError: Java heap space in GitHub Actions means one JVM in your build exhausted its heap, and on Gradle and Maven builds it is usually not the JVM your workflow configures. Raise the heap of the process the stack trace names, or set _JAVA_OPTIONS so every JVM in the job gets it.

What this error means

A compile, test or codegen step fails and the log carries java.lang.OutOfMemoryError: Java heap space with a stack trace at the allocation site. The build tool around it may report the failure in its own words first, as a failed task, a failed Surefire fork, or an expiring Gradle daemon, and the exception is often several screens above that summary. The same exception class appears with four other messages that are not the same problem: GC overhead limit exceeded means the heap is technically sufficient and the collector is thrashing, Metaspace means class metadata rather than objects, Direct buffer memory means off-heap NIO buffers, and unable to create new native thread means the process ran out of threads, not heap.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at ReportBuild.main(ReportBuild.java:9)

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at ReportBuild.main(ReportBuild.java:9)
[latchkey-bash-wrapper] BEGIN sidecar POST (boot_wait=30s max_time=320s url=http://localhost/diagnose socket=/run/latchkey-self-heal/sock)
[latchkey-bash-wrapper] END sidecar POST ok (attempts=1 http=200)
report built: 200 blocks, checksum 19900
Picked up _JAVA_OPTIONS: -Xmx4g -Xms512m
Picked up _JAVA_OPTIONS: -Xmx4g -Xms512m

Find out which JVM died

A Gradle build is at least three JVMs: the launcher, the long-lived daemon that compiles, and a forked worker per test executor. Maven is two: the Maven process and whatever Surefire forks. Each one takes its heap from a different place, so the first job is to name the process in the stack trace rather than to pick a number.

The thread name in the exception is the tell. Exception in thread "main" is the process your command started. Daemon worker or the notice "Expiring Daemon because JVM heap space is exhausted" is the Gradle daemon. Gradle Test Executor 1 or a Surefire ForkedBooter frame is a test fork, and a test fork does not read org.gradle.jvmargs at all.

Common causes

The ceiling is below what the build needs

The JVM defaults its maximum heap to a quarter of the machine's memory, which is a sensible desktop default and a low one on a shared runner. A build whose live set grows with the repository will cross it eventually, and annotation processors, code generators and large fixtures cross it sooner than application code does.

The heap you raised belongs to a different JVM

This is the most common reason a fix appears not to work. org.gradle.jvmargs sizes the daemon and has no effect on a forked test worker; MAVEN_OPTS sizes Maven and has no effect on a Surefire fork. The log names the process that died, and the setting has to match it.

The message is not about the heap

The same exception class carries Metaspace, Direct buffer memory and unable to create new native thread, and none of them are fixed by -Xmx. Reading the message rather than the class name saves the two or three wasted runs that usually follow.

Too many JVMs for one runner

Parallel Gradle workers, a matrix of forks, and maven.test.fork.count set from the core count all multiply the same ceiling. In our experience a build that fails only on CI and only sometimes is nearly always a parallelism setting reading a machine that is smaller than the developer's.

How to fix it

Raise the heap of the JVM that actually failed

1. Name the failing process from the thread name in the exception.
2. Set the ceiling where that process reads it: org.gradle.jvmargs for the daemon, test { maxHeapSize } for a test worker, MAVEN_OPTS for Maven, <argLine> for a Surefire fork.
3. Keep the sum of every concurrent JVM's ceiling under the runner's RAM.

# gradle.properties
org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=1g

# build.gradle
test {
    maxHeapSize = "2g"
    maxParallelForks = 2
}

Set it once for every JVM in the job

When the failing process is hard to reach, or the build shells out to tools you do not control, _JAVA_OPTIONS reaches all of them. The JVM applies it after the command line, so it also overrides an inline -Xmx that a wrapper script hardcoded. Every JVM it touches prints a Picked up _JAVA_OPTIONS line, so the change is visible in the log.

env:
  _JAVA_OPTIONS: -Xmx4g -Xms512m

Size the right pool for Metaspace and direct memory

If the message is Metaspace, raise -XX:MaxMetaspaceSize and stop reusing a daemon between jobs. If it is Direct buffer memory, raise -XX:MaxDirectMemorySize or find the buffers that are not being released.

env:
  _JAVA_OPTIONS: -XX:MaxMetaspaceSize=1g
  GRADLE_OPTS: -Dorg.gradle.daemon=false

Reduce the number of concurrent JVMs, or grow the runner

Pin maxParallelForks and --max-workers in CI rather than letting them read the core count, and if the build genuinely needs the memory, move it to a runner that has it. Two forks that finish beat four that die.

- run: ./gradlew test --max-workers=2 -Dorg.gradle.workers.max=2

Which setting reaches which JVM

Set the value in the place the failing process reads, not in the first place a search result suggests. On our reproduction the JVM printed Picked up _JAVA_OPTIONS: -Xmx4g -Xms512m and completed a workload that had just failed under an inline -Xmx64m, which is the behavior the table below describes in the last row.

Metaspace, GC overhead and native threads are different failures

Raising -Xmx is the right move for Java heap space and for GC overhead limit exceeded, where the collector is spending most of its time recovering a little memory. It does nothing for the other three.

Metaspace is class metadata, sized by -XX:MaxMetaspaceSize, and it fills up on builds that load thousands of classes or reuse a long-lived daemon across jobs. Direct buffer memory is off-heap and sized by -XX:MaxDirectMemorySize. unable to create new native thread is not a memory ceiling at all; it is the process or thread limit, and the fix is less parallelism rather than more heap.

Latchkey's detection pattern groups all five messages under one signature, JVM_HEAP_OOM, at confidence 0.94, and its recorded caveat is the same distinction: the bump via _JAVA_OPTIONS is the standard fix for heap space and GC overhead, while native-thread exhaustion "is more nuanced (ulimit)".

How much heap a runner can back

A standard Linux runner on a private repository has 8 GB of RAM for everything on the machine, and on a public repository 16 GB. A daemon at -Xmx4g plus two test forks at -Xmx2g is already past the smaller tier before the operating system takes its share. When the sum of the ceilings exceeds the machine, the JVM never gets to throw: the kernel kills the process first and the step ends at exit code 137 with no exception in the log.

How to prevent it

• Commit the heap settings to gradle.properties or the Maven config so CI and laptops agree.
• Disable the Gradle daemon in CI: a fresh JVM per job cannot accumulate class metadata.
• Pin fork and worker counts instead of deriving them from the runner's core count.
• Keep the sum of concurrent JVM ceilings under the runner's RAM, so a heap error stays an exception rather than becoming a kernel kill.

Frequently asked questions

Related guides

References

• Oracle: the java command and its -Xmx options
• Gradle: configuring the build environment
• GitHub-hosted runners: standard runner specifications
• gradle/gradle#8139: daemon heap exhaustion in CI
• Apache Maven guides
• Gradle user manual
• GitHub Actions documentation