Skip to content
Latchkey

Cassandra "Connection refused" on port 9042 (service not ready) in CI

Cassandra accepts client CQL connections on port 9042 only after its startup sequence (gossip, commitlog replay, schema load) finishes. A job that connects the moment the container starts hits "Connection refused" because the listener is not bound yet.

What this error means

The driver fails immediately with "Connection refused" to 127.0.0.1:9042, often within seconds of the container starting, even though the container later logs that it is listening for CQL clients.

cqlsh
Connection error: ('Unable to connect to any servers', {'127.0.0.1:9042':
ConnectionRefusedError(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error:
Connection refused")})

Common causes

The CQL listener has not bound yet

Cassandra takes tens of seconds to start. Until it logs "Starting listening for CQL clients on /0.0.0.0:9042", nothing is listening, so the connection is refused.

No readiness gate before the test step

The workflow runs tests as soon as the container is created, instead of waiting for the node to report it can serve queries.

How to fix it

Wait until cqlsh can run a trivial query

  1. Poll the node with cqlsh before starting tests.
  2. Treat the node as ready only when a SELECT now() succeeds.
  3. Give the loop a generous ceiling; first boot can take 60s or more.
Terminal
for i in $(seq 1 60); do
  cqlsh -e "SELECT now() FROM system.local" 127.0.0.1 9042 && break
  echo "waiting for cassandra ($i)"; sleep 5
done

Add a container health check on 9042

Use a service health check so dependent steps only run once the CQL port answers.

.github/workflows/ci.yml
services:
  cassandra:
    image: cassandra:5.0
    ports: ['9042:9042']
    options: >-
      --health-cmd "cqlsh -e 'describe cluster'"
      --health-interval 10s --health-timeout 5s --health-retries 12

How to prevent it

  • Always gate on a real CQL query, not just TCP reachability.
  • Allow at least 60s for first boot on cold runners.
  • Pin the Cassandra image tag so startup timing is consistent.

Related guides

Tired of flaky CI? Latchkey auto-heals failed jobs and retries them for you. Start free →