Consumer Groups and Rebalance
You are a Senior on-call. A production issue might involve Consumer Groups and Rebalance. How do you diagnose and mitigate?
Answers use simple, clear English.
Audio N/AQuick interview answer
Mitigate first, then root-cause. Check symptoms against: Long processing without pause; committing before side effects complete..
Detailed answer
Mitigate first, then root-cause. Check symptoms against: Long processing without pause; committing before side effects complete.. Validate with: Idempotent consumers; static membership; cooperative sticky assignor.. Context: Each partition consumed by one consumer in group. Rebalance on member join/leave — can cause duplicate processing if offsets not committed carefully. Core: Each partition consumed by one consumer in group. Rebalance on member join/leave — can cause duplicate processing if offsets not committed carefully. Real-time example: 6 partitions, 3 consumers → each gets 2 partitions; add 4th consumer triggers rebalance. Pros: Elastic scale-out; at-least-once default with offset commit. Cons: Rebalance pauses consumption; sticky assign reduces churn. Common mistakes: Long processing without pause; committing before side effects complete. Best practices: Idempotent consumers; static membership; cooperative sticky assignor. Audience level: Senior.
Full explanation
Each partition consumed by one consumer in group. Rebalance on member join/leave — can cause duplicate processing if offsets not committed carefully.
Real example & use case
6 partitions, 3 consumers → each gets 2 partitions; add 4th consumer triggers rebalance.
Pros & cons
Pros: Elastic scale-out; at-least-once default with offset commit. Cons: Rebalance pauses consumption; sticky assign reduces churn.
Common mistakes
Long processing without pause; committing before side effects complete.
Best practices
Idempotent consumers; static membership; cooperative sticky assignor.
Follow-up questions
Open one as its own read / solve / listen card