Consumer Groups and Rebalance
What are the key trade-offs of Consumer Groups and Rebalance? When would you choose it vs alternatives?
Answers use simple, clear English.
Audio N/AQuick interview answer
Pros: Elastic scale-out; at-least-once default with offset commit. Cons: Rebalance pauses consumption; sticky assign reduces churn.
Detailed answer
Pros: Elastic scale-out; at-least-once default with offset commit. Cons: Rebalance pauses consumption; sticky assign reduces churn. Decision: pick when benefits outweigh operational cost. Core: Each partition consumed by one consumer in group. Rebalance on member join/leave — can cause duplicate processing if offsets not committed carefully. Real-time example: 6 partitions, 3 consumers → each gets 2 partitions; add 4th consumer triggers rebalance. Pros: Elastic scale-out; at-least-once default with offset commit. Cons: Rebalance pauses consumption; sticky assign reduces churn. Common mistakes: Long processing without pause; committing before side effects complete. Best practices: Idempotent consumers; static membership; cooperative sticky assignor. Audience level: Mid-level.
Full explanation
Each partition consumed by one consumer in group. Rebalance on member join/leave — can cause duplicate processing if offsets not committed carefully.
Real example & use case
6 partitions, 3 consumers → each gets 2 partitions; add 4th consumer triggers rebalance.
Pros & cons
Pros: Elastic scale-out; at-least-once default with offset commit. Cons: Rebalance pauses consumption; sticky assign reduces churn.
Common mistakes
Long processing without pause; committing before side effects complete.
Best practices
Idempotent consumers; static membership; cooperative sticky assignor.
Follow-up questions
- How would you test Consumer Groups and Rebalance?
- What metrics prove Consumer Groups and Rebalance is healthy in prod?
- How does Consumer Groups and Rebalance change at 10× traffic?