Kafka Exactly-Once Semantics
What common mistakes do candidates make around Kafka Exactly-Once Semantics, and how do you avoid them?
Answers use simple, clear English.
Quick interview answer
Common mistakes: Assuming EOS covers DB update without idempotency key. Best practices: Outbox table + relay; idempotency keys on consumers.
Detailed answer
Common mistakes: Assuming EOS covers DB update without idempotency key. Best practices: Outbox table + relay; idempotency keys on consumers. Core: Idempotent producer + transactional writes (read-process-write in one transaction) give EOS within Kafka. End-to-end EOS needs idempotent sinks or outbox pattern. Real-time example: Stream processor reads payments, updates balance topic and audit topic atomically in one transaction. Pros: No duplicate downstream messages within pipeline. Cons: Higher latency; transactional overhead; not magic for external DB writes. Common mistakes: Assuming EOS covers DB update without idempotency key. Best practices: Outbox table + relay; idempotency keys on consumers. Audience level: Engineering Manager.
Full explanation
Idempotent producer + transactional writes (read-process-write in one transaction) give EOS within Kafka. End-to-end EOS needs idempotent sinks or outbox pattern.
Real example & use case
Stream processor reads payments, updates balance topic and audit topic atomically in one transaction.
Pros & cons
Pros: No duplicate downstream messages within pipeline. Cons: Higher latency; transactional overhead; not magic for external DB writes.
Common mistakes
Assuming EOS covers DB update without idempotency key.
Best practices
Outbox table + relay; idempotency keys on consumers.
Follow-up questions
Open one as its own read / solve / listen card