Kafka "UnknownTopicOrPartitionException" in CI

The client addressed a topic-partition the broker does not host. In CI this almost always means the topic was never created: auto-creation is disabled (the default in many images), and the producer or consumer ran before any setup step created it.

What this error means

A producer or consumer fails with "org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition." while other broker calls succeed.

Terminal

org.apache.kafka.common.errors.UnknownTopicOrPartitionException:
This server does not host this topic-partition.

Common causes

The topic was never created

With auto.create.topics.enable=false, the broker does not create a topic on first produce, so any reference to it is unknown.

A readiness race created it after the client started

A consumer subscribed before the create-topic step ran, so at subscription time the topic-partition did not exist yet.

How to fix it

Create the topic explicitly before tests

Run kafka-topics.sh --create as a setup step so the topic exists with a known partition count.

Terminal

kafka-topics.sh --bootstrap-server localhost:9092 \
  --create --topic orders --partitions 3 --replication-factor 1

Enable auto topic creation for the test broker

If your tests rely on first-produce creation, turn the setting on in the service container.

.github/workflows/ci.yml

services:
  kafka:
    image: bitnami/kafka:3.7
    env:
      KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE: 'true'

How to prevent it

Create required topics in a setup step rather than relying on auto-create.
Wait for the create-topic step to finish before starting consumers.
Pin the partition count so tests do not depend on broker defaults.