pytest "Failed: DID NOT RAISE" in CI

By Daniel Zoghalchali·Latchkey

A with pytest.raises(...) block asserts that the enclosed code raises a given exception. The code ran to completion without raising, so pytest fails the assertion with "DID NOT RAISE".

What this error means

A test fails with "Failed: DID NOT RAISE <class 'ValueError'>" - the behavior changed and no longer raises, or the expected exception type is wrong.

def test_rejects_empty():
    with pytest.raises(ValueError):
        parse("")
E   Failed: DID NOT RAISE <class 'ValueError'>

Diagnose it: what did pytest actually collect?

Most pytest failures that only happen in CI are collection or import-path problems rather than test failures. The runner has a different working directory, a different sys.path, and usually no editable install, so a test module that imports your package locally may not resolve at all.

# what would run, without running it
pytest --collect-only -q | tail -20

# where pytest thinks the root is (drives conftest and import mode)
pytest --collect-only 2>&1 | grep -i rootdir

# is the package importable at all from the runner cwd?
python -c "import yourpackage, sys; print(yourpackage.__file__)"
python -c "import sys; print(sys.path)"

Common causes

The code no longer raises

A change made the function tolerate the input (or raise later), so the expected exception never occurs in the block.

The wrong exception type is expected

The code raises a different exception than the one in pytest.raises, so the expected type is not seen.

How to fix it

Confirm the real behavior and align the test

1. Run the code under test with the input to see what it actually does.
2. If it should raise, fix the code; if behavior changed intentionally, update the test.
3. Match the exception type in pytest.raises to what is raised.

with pytest.raises(ValueError, match="empty"):
    parse("")

Keep the asserted call inside the block

Ensure the call that should raise is inside the with block, not before it.

Make a zero-test run fail the build

# fail explicitly when nothing is collected
pytest --strict-markers -q
test "${PIPESTATUS[0]}" -ne 5 || { echo "pytest collected no tests"; exit 1; }

How to prevent it

• Update raise-expectations when behavior intentionally changes.
• Assert the exception type and message with match=.
• Keep only the raising call inside the pytest.raises block.

Frequently asked questions

Related guides

References

• Python documentation
• pip documentation
• GitHub Actions documentation