# Python MemoryError in GitHub Actions

> Python MemoryError in GitHub Actions is the interpreter refusing one allocation, not the kernel OOM killer. Tell the two apart and cut the peak.

Source: https://latchkey.dev/learn/failures/python-memoryerror-in-ci  
Updated: 2026-09-20

Python MemoryError in GitHub Actions is the interpreter refusing one allocation it could not satisfy, not the kernel killing the process. You get a traceback and a nonzero exit; an out-of-memory kill gives you neither, and the two take different fixes.

## What this error means

A data step or a test ends on a traceback whose last line is `MemoryError`, usually on a read, a concat, a merge or an array allocation. With NumPy in the stack the message is more useful: it names the size the allocation wanted, the shape and the dtype. The Python documentation describes it as "raised when an operation runs out of memory but the situation may still be rescued (by deleting some objects)", which is why you get a stack trace at all. The failure is nearly always CI-only.

```Reproduction output, latchkey-small runner, 20 September 2026
Traceback (most recent call last):
  File "/home/runner/alloc_numpy.py", line 6, in <module>
    np.zeros((1_200_000_000,), dtype=np.float64)
numpy._core._exceptions._ArrayMemoryError: Unable to allocate 8.94 GiB for an array with shape (1200000000,) and data type float64

Traceback (most recent call last):
  File "/home/runner/alloc_python.py", line 12, in <module>
    load_rows()
  File "/home/runner/alloc_python.py", line 9, in load_rows
    rows.append(bytearray(16 * 1024 * 1024))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError
```

## Common causes

### The peak allocation is larger than the runner has

Reading a whole file into a frame, or copying one, needs contiguous memory the machine does not have. The number NumPy prints is the request, not the total, so a job can fail on an 8.94 GiB allocation while using little else.

### One wide operation multiplies the data

A merge on a key with duplicates, a pivot, or a concat of many frames can produce something several times the size of any input. This is the version that surprises people, because the file on disk is small and the failure is not.

### Every test worker holds its own copy

Parallel test runners fork one process per core and default their worker count to the machine. Each worker loads the fixtures again, so the memory needed is the per-process peak multiplied by the worker count, against a fixed pool.

### A limit rather than the physical memory

An address-space limit, a container memory cap, or a 32-bit interpreter raises `MemoryError` long before the machine is full. Our own reproduction is this case on purpose, and it is worth ruling out before buying a bigger runner that changes nothing.

## How to fix it

### Process the data in chunks instead of one allocation

1. Replace the whole-file read with an iterator and aggregate as you go, so the full frame never exists.
2. Keep the accumulator small: append reduced results, not the chunks themselves.
3. Confirm the change by printing peak memory rather than by seeing the job go green once.

```Python
import pandas as pd

totals = {}
for chunk in pd.read_csv("big.csv", chunksize=100_000, usecols=["id", "value"]):
    for key, value in chunk.groupby("id")["value"].sum().items():
        totals[key] = totals.get(key, 0) + value
```

### Narrow the dtypes and drop the columns you do not read

The cheapest change on the page and usually the largest. float64 to float32 halves a column, int64 to int32 quarters it, and a repeated string column to `category` can cut it by an order of magnitude. Pass `usecols` so columns you never touch are never allocated.

```Python
df = pd.read_csv(
    "big.csv",
    usecols=["id", "region", "value"],
    dtype={"id": "int32", "region": "category", "value": "float32"},
)
```

### Cut the worker count for memory-heavy suites

Parallel workers trade memory for wall clock, and on a small runner that trade stops paying. Pin the number rather than letting it follow the core count, and split the heavy tests into their own job if the rest of the suite is fine.

```.github/workflows/ci.yml
- run: pytest -n 2 tests/unit          # not -n auto
- run: pytest -n 1 tests/data_pipeline  # the memory-heavy suite, alone
```

### Move the job to a bigger machine, once the peak is as low as it goes

Some workloads genuinely need the memory, and the answer is a larger runner for that job alone. Runner size is chosen per job, so the heavy step moves without the lint job following it.

```.github/workflows/ci.yml
jobs:
  unit:
    runs-on: ubuntu-latest
  data-pipeline:
    runs-on: latchkey-large     # 8 vCPU, 32 GB, for the job that needs it
```

## How to prevent it

- Use sampled fixtures in CI, not production-scale files.
- Declare dtypes and `usecols` at read time rather than downcasting later.
- Pin test worker counts on memory-heavy suites.
- Print peak memory in the job so a regression is visible before it fails.

## MemoryError or the OOM killer? The log tells you

Two different failures get the same words in most write-ups. Read the end of the log first.

A traceback ending in `MemoryError` is Python refusing a single allocation. The process was alive, asked for a block the allocator could not provide, and raised. Your code could have caught it.

No traceback, the word Killed on its own line, and exit 137 is the kernel out-of-memory killer. Nothing in Python ran and there is no file or line to look at. That failure is covered in [exit code 137 in GitHub Actions](/learn/failures/exit-code-137-in-github-actions), and its fixes are about the whole process, not one allocation.

Our recorded run checked this: after both allocations failed it counted the kernel out-of-memory lines from the last two minutes and found zero. The interpreter refused; the machine was never in trouble.

## What the reproduction measured

The script sets a 512 MB address-space limit on itself with `resource.setrlimit`, then allocates past it twice: once through NumPy and once with plain `bytearray` blocks. The cap stands in for the runner's RAM ceiling, keeping the failure contained and repeatable, and it is the only thing the script changes. Both files are rewritten at the start of every attempt, so a second attempt runs the same two allocations.

The plain Python half printed the total it had requested after each block, labeled `resident:` in the log, and stopped at 480 MB, two 16 MB blocks short of the cap. That is the shape of the real failure: the allocation that fails is not the big one, it is the next ordinary one after the working set has grown.

Both halves exited nonzero on a Latchkey `latchkey-small` runner on 20 September 2026. The log ends on the runner wrapper escalating a failed Stage 2 heal to Stage 3: the diagnosis stage ran over this failure and did not repair it.

## The size NumPy prints is the number to work with

When NumPy is in the stack you are handed the arithmetic. Our run asked for a one-dimensional float64 array of 1,200,000,000 values and NumPy reported "Unable to allocate 8.94 GiB". Eight bytes per value times 1.2 billion values is 9.6 billion bytes, which is 8.94 GiB, and that is the whole calculation.

Run it in the other direction and it becomes a budget. A standard GitHub-hosted Linux runner has 8 GB of RAM on a private repository and 16 GB on a public one, read from docs.github.com on 20 September 2026, so a float64 column of a billion values fits on neither. `float32` halves it. `int32` quarters it. Dropping the columns you never read removes them entirely.

Note the class name too. Our run printed `numpy._core._exceptions._ArrayMemoryError`, which is NumPy 2: the migration guide states that "the `np.core` namespace is now officially private and has been renamed to `np._core`". Older releases print the same failure under `numpy.core`, so a search that only matches one spelling will miss half the reports.

```Python
import pandas as pd

# read only what you use, and say what it is
df = pd.read_csv(
    "big.csv",
    usecols=["id", "ts", "value"],
    dtype={"id": "int32", "value": "float32"},
)
```

## If it only fails in CI

That is the normal case rather than a clue. A developer machine has several times the memory of a standard runner, so the same script has room your CI job does not.

Parallelism widens the gap. `pytest -n auto` starts one worker per core, and every worker holds its own copy of whatever the fixtures loaded, so a per-process budget that is comfortable alone is not comfortable in a suite.

Before reaching for a bigger machine, check the peak is real. A `read_csv` of a file you immediately group down, a merge on a duplicated key, or a pivot will all spike far above the input on disk.

## FAQ

### How can I prevent an out of memory error in GitHub Actions?

Lower the peak before you raise the ceiling: read in chunks, declare narrow dtypes, drop unused columns, and pin the worker count so each one is not holding its own copy. A standard Linux runner has 8 GB on a private repository and 16 GB on a public one, so the budget is fixed and the working set is the part you control.

### What is the difference between MemoryError and being OOM-killed?

MemoryError is raised inside the process, so you get a traceback naming the file and line. An out-of-memory kill happens outside it: the kernel terminates the process with SIGKILL, the log shows Killed and exit 137, and there is no traceback because no Python code ran after it. Our reproduction confirmed the difference by checking the kernel log and finding no out-of-memory lines.

### How much memory does a GitHub Actions runner have?

The standard Linux runner is 2 vCPU with 8 GB of RAM and 14 GB of SSD on a private repository, and 4 vCPU with 16 GB on a public one, per docs.github.com read on 20 September 2026. Older answers still quote 7 GB. Latchkey sizes start at 2 vCPU with 8 GB on `latchkey-small` and reach 16 vCPU with 64 GB on `latchkey-xlarge`.

### Does a bigger runner fix a pandas MemoryError?

Only when the peak is already as small as the work allows. If a merge is multiplying rows or a whole file is being read to compute one aggregate, a bigger machine moves the failure later rather than removing it, and you pay for the memory every run. Fix the peak first, then size the machine to what is left.

## References

- [Python documentation: MemoryError](https://docs.python.org/3/library/exceptions.html#MemoryError)
- [NumPy 2.0 migration guide: numpy.core renamed to numpy._core](https://numpy.org/doc/stable/numpy_2_0_migration_guide.html)
- [GitHub-hosted runners: standard runner specifications](https://docs.github.com/en/actions/reference/runners/github-hosted-runners)
- [Latchkey documentation: runner sizes and memory](https://latchkey.dev/documentation/runners-overview)

---

Latchkey runs CI/CD that repairs its own failures. Agent entry points: https://latchkey.dev/agent.txt, https://latchkey.dev/openapi.json, https://latchkey.dev/llms.txt