Consumer Groups and Rebalance
Design a small subsystem that relies on Consumer Groups and Rebalance. Outline components, data flow, failure modes, and metrics.
Answers use simple, clear English.
Quick interview answer
Use Consumer Groups and Rebalance as the core idea. Example shape: 6 partitions, 3 consumers → each gets 2 partitions; add 4th consumer triggers rebalance..
Detailed answer
Use Consumer Groups and Rebalance as the core idea. Example shape: 6 partitions, 3 consumers → each gets 2 partitions; add 4th consumer triggers rebalance.. Watch for: Rebalance pauses consumption; sticky assign reduces churn.. Measure success via latency/error/saturation. Core: Each partition consumed by one consumer in group. Rebalance on member join/leave — can cause duplicate processing if offsets not committed carefully. Real-time example: 6 partitions, 3 consumers → each gets 2 partitions; add 4th consumer triggers rebalance. Pros: Elastic scale-out; at-least-once default with offset commit. Cons: Rebalance pauses consumption; sticky assign reduces churn. Common mistakes: Long processing without pause; committing before side effects complete. Best practices: Idempotent consumers; static membership; cooperative sticky assignor. Audience level: Junior.
Full explanation
Each partition consumed by one consumer in group. Rebalance on member join/leave — can cause duplicate processing if offsets not committed carefully.
Real example & use case
6 partitions, 3 consumers → each gets 2 partitions; add 4th consumer triggers rebalance.
Pros & cons
Pros: Elastic scale-out; at-least-once default with offset commit. Cons: Rebalance pauses consumption; sticky assign reduces churn.
Common mistakes
Long processing without pause; committing before side effects complete.
Best practices
Idempotent consumers; static membership; cooperative sticky assignor.
Follow-up questions
Open one as its own read / solve / listen card