What Kafka is and when to use it, how it differs from traditional brokers, its architecture, topics and partitions, creating topics, replication and ISR, ZooKeeper's historical role, what happens when it fails — and KRaft, which replaced ZooKeeper entirely in Kafka 4.0.
Published September 25, 2026
Kafka questions often come from material written before Kafka 4.0 (2025) removed ZooKeeper. Answer the ZooKeeper questions correctly for older clusters, then add that KRaft is how Kafka works now. That single update marks you as current.
Short answer: Kafka is a distributed, partitioned, replicated commit log, used as an event-streaming platform. Producers append records to topics. Kafka stores them durably, for a configurable retention period, whether or not anyone has consumed them. Consumers read at their own pace by tracking offsets. It delivers very high throughput through sequential disk I/O, batching, zero-copy transfer and partitioning.
Learn it in depth → Messaging Technology Choices
Short answer:
Short answer:
| Traditional broker (RabbitMQ, ActiveMQ) | Kafka | |
|---|---|---|
| Model | Queue: messages are deleted once acknowledged | Log: records are retained (by time or size), and consumers track offsets |
| Replay | Generally not | Yes: rewind the offsets and reprocess |
| Many independent consumers | Needs fan-out exchanges or copies | Free: each consumer group reads the same log |
| Ordering | Per queue | Per partition |
| Throughput | High | Very high (millions of messages per second across a cluster) |
| Routing | Rich (exchanges, patterns, priorities) | Simple: topic plus key → partition |
| Push/pull | Mostly push | Consumers pull |
Key points to cover:
Short answer:
Short answer: A topic is a named, append-only stream of records for one kind of event (orders.placed). It's split into partitions for parallelism, and each partition is an ordered log in which each record has an offset. Records are retained according to a policy: time or size (retention.ms, retention.bytes), or compaction (keep the latest value per key).
Short answer: Use the CLI, the Admin API, or declarative infrastructure-as-code:
kafka-topics.sh --bootstrap-server broker:9092 --create --topic orders.placed \
--partitions 12 --replication-factor 3 --config min.insync.replicas=2 --config retention.ms=604800000
@Bean
NewTopic ordersPlaced() { // Spring Kafka's KafkaAdmin creates it at startup if missing
return TopicBuilder.name("orders.placed").partitions(12).replicas(3)
.config(TopicConfig.MIN_IN_SYNC_REPLICAS_CONFIG, "2").build();
}
Key points to cover:
auto.create.topics.enable=false), and manage topics through GitOps or Terraform.Short answer: A topic's partitions are spread across brokers. A producer chooses a partition by:
Partitions matter because they are:
orderId).Key points to cover:
Short answer: Each partition has replication-factor copies on different brokers:
acks=all and min.insync.replicas are configured properly.Key points to cover:
broker.rack) spreads replicas across failure domains.Short answer: In older Kafka (before KRaft), ZooKeeper stored the cluster metadata:
It also helped elect the controller broker, which in turn managed partition leadership, and told brokers about metadata changes.
Key points to cover:
Short answer: In ZooKeeper-based clusters, it was the source of truth for metadata, and the anchor for controller election. Without it, the cluster couldn't change leadership or configuration safely. That centrality was also its weakness: a second distributed system to deploy, secure and tune, slow controller failover on large clusters (metadata had to be reloaded), and a practical limit on partition count. KRaft removed those problems. Controller failover is fast, the cluster scales to millions of partitions, and there's one system to operate.
Short answer (ZooKeeper-based clusters):
Key points to cover:
Short answer: In older clusters: brokers keep their cached metadata and continue serving, while metadata operations are blocked until a ZooKeeper quorum returns. When it does, the controller reconciles the state. Operators minimise the risk with an odd-sized ensemble across AZs, dedicated disks and monitoring.
The modern answer is KRaft. There's no ZooKeeper to lose. Metadata is replicated among 3 or 5 KRaft controllers using Raft, and losing a minority of controllers doesn't stop the control plane.
Q: How do you migrate from ZooKeeper to KRaft? A: Kafka 3.x provides a migration mode. You stand up KRaft controllers, migrate the metadata while brokers run in dual-write mode, roll the brokers into KRaft mode, then finalise and decommission ZooKeeper. You have to be on a bridge 3.x release before upgrading to 4.0.
Q: What's the ISR, and what is min.insync.replicas?
A: The ISR is the set of replicas fully caught up with the leader. min.insync.replicas (commonly 2, with a replication factor of 3) is the minimum ISR size required for an acks=all write to succeed. That guarantees every acknowledged write exists on at least two brokers.
Q: How many partitions should a topic have? A: Enough for the peak consumer parallelism you need (target throughput ÷ per-consumer throughput), plus headroom, and not thousands without reason. More partitions mean more open files, more metadata, and longer rebalances.
Q: What is log compaction? A: A retention mode that keeps at least the latest record per key, and removes older ones. It suits changelog or state topics, such as current customer profiles, and it's what Kafka Streams uses for its state stores.