schemathesis "1 checks failed" status conformance in CI

schemathesis runs conformance checks against your schema. When a response returns a status the operation does not document, or a body that does not match the declared schema, the corresponding check fails and the run exits non-zero.

What this error means

schemathesis reports "1 checks failed" naming status_code_conformance or response_schema_conformance, showing the undocumented status or the schema mismatch it found.

1. Received a response with a status code, which is not defined
   in the schema: 422
   Declared status codes: 200, 404
Checks: 1 passed, 1 failed

Common causes

The API returns an undocumented status code

The handler responds with, for example, 422, but the OpenAPI operation only lists 200 and 404, so the conformance check flags the drift.

The response body does not match the schema

A field is missing, mistyped, or extra relative to the declared response schema, so response_schema_conformance fails.

How to fix it

Reconcile the schema with the real responses

1. Decide whether the response or the schema is wrong.
2. Add the missing status/field to the OpenAPI spec, or fix the handler to match it.
3. Re-run so the conformance checks pass.

schemathesis run http://localhost:8000/openapi.json \
  --checks status_code_conformance,response_schema_conformance

Scope the checks you gate on

Select exactly the checks you want to enforce so CI fails on real drift and not on checks you are not ready for yet.

schemathesis run ./openapi.yaml --base-url http://localhost:8000 \
  --checks all --exclude-checks ignored_auth

How to prevent it

• Keep the OpenAPI schema updated whenever responses change.
• Enforce conformance checks in CI so schema drift is caught in review.
• Document every status code an operation can return.

Related guides