Python MemoryError in GitHub Actions

By Daniel Zoghalchali·Latchkey

Python MemoryError in GitHub Actions is the interpreter refusing one allocation it could not satisfy, not the kernel killing the process. You get a traceback and a nonzero exit; an out-of-memory kill gives you neither, and the two take different fixes.

What this error means

A data step or a test ends on a traceback whose last line is MemoryError, usually on a read, a concat, a merge or an array allocation. With NumPy in the stack the message is more useful: it names the size the allocation wanted, the shape and the dtype. The Python documentation describes it as "raised when an operation runs out of memory but the situation may still be rescued (by deleting some objects)", which is why you get a stack trace at all. The failure is nearly always CI-only.

Traceback (most recent call last):
  File "/home/runner/alloc_numpy.py", line 6, in <module>
    np.zeros((1_200_000_000,), dtype=np.float64)
numpy._core._exceptions._ArrayMemoryError: Unable to allocate 8.94 GiB for an array with shape (1200000000,) and data type float64

Traceback (most recent call last):
  File "/home/runner/alloc_python.py", line 12, in <module>
    load_rows()
  File "/home/runner/alloc_python.py", line 9, in load_rows
    rows.append(bytearray(16 * 1024 * 1024))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError

resident: 240 MB
resident: 256 MB
resident: 272 MB
resident: 288 MB
resident: 304 MB
resident: 320 MB
resident: 336 MB
resident: 352 MB
resident: 368 MB
resident: 384 MB
resident: 400 MB
resident: 416 MB
resident: 432 MB
resident: 448 MB
resident: 464 MB
resident: 480 MB
python allocation exit: 1
kernel oom-kill lines in the last 2 minutes: 0
Traceback (most recent call last):
  File "/home/runner/alloc_numpy.py", line 6, in <module>
    np.zeros((1_200_000_000,), dtype=np.float64)
numpy._core._exceptions._ArrayMemoryError: Unable to allocate 8.94 GiB for an array with shape (1200000000,) and data type float64
Traceback (most recent call last):
  File "/home/runner/alloc_python.py", line 12, in <module>
    load_rows()
  File "/home/runner/alloc_python.py", line 9, in load_rows
    rows.append(bytearray(16 * 1024 * 1024))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError
[latchkey-bash-wrapper] escalating failed Stage 2 heal to Stage 3

MemoryError or the OOM killer? The log tells you

Two different failures get the same words in most write-ups. Read the end of the log first.

A traceback ending in MemoryError is Python refusing a single allocation. The process was alive, asked for a block the allocator could not provide, and raised. Your code could have caught it.

No traceback, the word Killed on its own line, and exit 137 is the kernel out-of-memory killer. Nothing in Python ran and there is no file or line to look at. That failure is covered in exit code 137 in GitHub Actions, and its fixes are about the whole process, not one allocation.

Our recorded run checked this: after both allocations failed it counted the kernel out-of-memory lines from the last two minutes and found zero. The interpreter refused; the machine was never in trouble.

Common causes

The peak allocation is larger than the runner has

Reading a whole file into a frame, or copying one, needs contiguous memory the machine does not have. The number NumPy prints is the request, not the total, so a job can fail on an 8.94 GiB allocation while using little else.

One wide operation multiplies the data

A merge on a key with duplicates, a pivot, or a concat of many frames can produce something several times the size of any input. This is the version that surprises people, because the file on disk is small and the failure is not.

Every test worker holds its own copy

Parallel test runners fork one process per core and default their worker count to the machine. Each worker loads the fixtures again, so the memory needed is the per-process peak multiplied by the worker count, against a fixed pool.

A limit rather than the physical memory

An address-space limit, a container memory cap, or a 32-bit interpreter raises MemoryError long before the machine is full. Our own reproduction is this case on purpose, and it is worth ruling out before buying a bigger runner that changes nothing.

How to fix it

Process the data in chunks instead of one allocation

1. Replace the whole-file read with an iterator and aggregate as you go, so the full frame never exists.
2. Keep the accumulator small: append reduced results, not the chunks themselves.
3. Confirm the change by printing peak memory rather than by seeing the job go green once.

import pandas as pd

totals = {}
for chunk in pd.read_csv("big.csv", chunksize=100_000, usecols=["id", "value"]):
    for key, value in chunk.groupby("id")["value"].sum().items():
        totals[key] = totals.get(key, 0) + value

Narrow the dtypes and drop the columns you do not read

The cheapest change on the page and usually the largest. float64 to float32 halves a column, int64 to int32 quarters it, and a repeated string column to category can cut it by an order of magnitude. Pass usecols so columns you never touch are never allocated.

df = pd.read_csv(
    "big.csv",
    usecols=["id", "region", "value"],
    dtype={"id": "int32", "region": "category", "value": "float32"},
)

Cut the worker count for memory-heavy suites

Parallel workers trade memory for wall clock, and on a small runner that trade stops paying. Pin the number rather than letting it follow the core count, and split the heavy tests into their own job if the rest of the suite is fine.

- run: pytest -n 2 tests/unit          # not -n auto
- run: pytest -n 1 tests/data_pipeline  # the memory-heavy suite, alone

Move the job to a bigger machine, once the peak is as low as it goes

Some workloads genuinely need the memory, and the answer is a larger runner for that job alone. Runner size is chosen per job, so the heavy step moves without the lint job following it.

jobs:
  unit:
    runs-on: ubuntu-latest
  data-pipeline:
    runs-on: latchkey-large     # 8 vCPU, 32 GB, for the job that needs it

What the reproduction measured

The script sets a 512 MB address-space limit on itself with resource.setrlimit, then allocates past it twice: once through NumPy and once with plain bytearray blocks. The cap stands in for the runner's RAM ceiling, keeping the failure contained and repeatable, and it is the only thing the script changes. Both files are rewritten at the start of every attempt, so a second attempt runs the same two allocations.

The plain Python half printed the total it had requested after each block, labeled resident: in the log, and stopped at 480 MB, two 16 MB blocks short of the cap. That is the shape of the real failure: the allocation that fails is not the big one, it is the next ordinary one after the working set has grown.

Both halves exited nonzero on a Latchkey latchkey-small runner on 20 September 2026. The log ends on the runner wrapper escalating a failed Stage 2 heal to Stage 3: the diagnosis stage ran over this failure and did not repair it.

The size NumPy prints is the number to work with

When NumPy is in the stack you are handed the arithmetic. Our run asked for a one-dimensional float64 array of 1,200,000,000 values and NumPy reported "Unable to allocate 8.94 GiB". Eight bytes per value times 1.2 billion values is 9.6 billion bytes, which is 8.94 GiB, and that is the whole calculation.

Run it in the other direction and it becomes a budget. A standard GitHub-hosted Linux runner has 8 GB of RAM on a private repository and 16 GB on a public one, read from docs.github.com on 20 September 2026, so a float64 column of a billion values fits on neither. float32 halves it. int32 quarters it. Dropping the columns you never read removes them entirely.

Note the class name too. Our run printed numpy._core._exceptions._ArrayMemoryError, which is NumPy 2: the migration guide states that "the np.core namespace is now officially private and has been renamed to np._core". Older releases print the same failure under numpy.core, so a search that only matches one spelling will miss half the reports.

import pandas as pd

# read only what you use, and say what it is
df = pd.read_csv(
    "big.csv",
    usecols=["id", "ts", "value"],
    dtype={"id": "int32", "value": "float32"},
)

If it only fails in CI

That is the normal case rather than a clue. A developer machine has several times the memory of a standard runner, so the same script has room your CI job does not.

Parallelism widens the gap. pytest -n auto starts one worker per core, and every worker holds its own copy of whatever the fixtures loaded, so a per-process budget that is comfortable alone is not comfortable in a suite.

Before reaching for a bigger machine, check the peak is real. A read_csv of a file you immediately group down, a merge on a duplicated key, or a pivot will all spike far above the input on disk.

How to prevent it

• Use sampled fixtures in CI, not production-scale files.
• Declare dtypes and usecols at read time rather than downcasting later.
• Pin test worker counts on memory-heavy suites.
• Print peak memory in the job so a regression is visible before it fails.

Frequently asked questions

Related guides

References

• Python documentation: MemoryError
• NumPy 2.0 migration guide: numpy.core renamed to numpy._core
• GitHub-hosted runners: standard runner specifications
• Latchkey documentation: runner sizes and memory
• Python documentation
• pip documentation
• GitHub Actions documentation