schemathesis run: Property-Test an API from OpenAPI

schemathesis run derives test cases from an OpenAPI/GraphQL schema, sends them to a running API, and fails when responses violate the schema or a selected check.

Schemathesis is property-based testing for APIs: it fuzzes inputs from the schema and asserts the server stays within the contract. It sits alongside diff/lint as a runtime conformance gate.

What it does

The run command loads a schema (file or URL) and a target --url, generates many requests per operation, and applies checks (status code conformance, response schema conformance, and more). It exits non-zero when any check fails and prints a minimal reproducing example.

Common usage

Terminal

schemathesis run --url http://localhost:8000 openapi.yaml
# select checks and run a subset of operations
schemathesis run --url http://localhost:8000 \
  --checks all --include-path-regex '^/v1/' openapi.yaml

Options

Flag	What it does
-u, --url <base>	Base URL of the API under test
-c, --checks <name\|all>	Which checks to run (e.g. response_schema_conformance)
--include-path-regex <re>	Only test operations matching the pattern
-H, --header <k: v>	Extra request header (e.g. auth)
--max-examples <n>	Cases generated per operation
--report junit	Emit a JUnit report for CI

In CI

Start the API (or a Prism mock), then run schemathesis run --checks all against it as a job; the non-zero exit on a failing check blocks the PR. Emit a JUnit report so failures show in the test UI, and set --max-examples to trade coverage against runtime.

Common errors in CI

A failed run prints FAILED per operation with a check name such as response_schema_conformance and a Falsifying example (curl reproduction), then a non-zero exit. "Failed to load schema" means a bad path or an unreachable schema URL. "Connection refused" or "Max retries exceeded" against --url means the API is not up yet; wait for readiness before running.