Java "OutOfMemoryError: Direct buffer memory" in CI - Fix Off-Heap

The JVM ran out of direct (off-heap) buffer memory - the native region NIO direct ByteBuffers allocate from. It is bounded by MaxDirectMemorySize, separate from the Java heap, so raising -Xmx does nothing.

What this error means

A network/IO-heavy step (Netty, gRPC, NIO) fails with java.lang.OutOfMemoryError: Direct buffer memory. Heap looks fine; the exhaustion is in off-heap direct memory.

java.lang.OutOfMemoryError: Direct buffer memory
    at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
    at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)

Common causes

MaxDirectMemorySize too low

Direct memory defaults to roughly the max heap. IO-heavy code allocating many direct buffers can exceed that limit and fail off-heap.

Direct buffers not released (leak)

Direct ByteBuffers are freed when their owning objects are GC’d. If they are retained or pooled unbounded, off-heap memory fills even though the heap is healthy.

How to fix it

Raise MaxDirectMemorySize

Set an explicit off-heap limit sized for the workload.

java -XX:MaxDirectMemorySize=512m -jar app.jar
# Gradle forked test JVM:
# test { jvmArgs("-XX:MaxDirectMemorySize=512m") }

Bound or release direct buffers

1. Bound pooled direct buffers (e.g. Netty io.netty.maxDirectMemory/arena sizing).
2. Ensure buffers are released/closed deterministically rather than waiting on GC.
3. Add -XX:+HeapDumpOnOutOfMemoryError plus native memory tracking to find the retention.

How to prevent it

• Set MaxDirectMemorySize explicitly for IO-heavy services in CI.
• Release direct buffers deterministically; bound any buffer pools.
• Account for off-heap memory when sizing the container limit.

Frequently asked questions

Related guides