Retention vs Log Compaction
As a Junior engineer, explain Retention vs Log Compaction. What problem does it solve and how would you describe it in an interview?
Answers use simple, clear English.
Quick interview answer
At Junior depth: start with the problem, then mechanism, then a short example. Time/size retention deletes old segments.
Detailed answer
At Junior depth: start with the problem, then mechanism, then a short example. Time/size retention deletes old segments. Compacted topics keep latest value per key — ideal for changelog / KTable state. Core: Time/size retention deletes old segments. Compacted topics keep latest value per key — ideal for changelog / KTable state. Real-time example: user-profile topic compacted by userId; consumers rebuild cache from log. Pros: Infinite history of latest state per key without full replay cost. Cons: Tombstones need retention to delete keys; not for event-sourcing all revisions. Common mistakes: Compacting high-churn unique keys (no benefit). Best practices: Compact config/state topics; append-only audit with time retention. Audience level: Junior.
Full explanation
Time/size retention deletes old segments. Compacted topics keep latest value per key — ideal for changelog / KTable state.
Real example & use case
user-profile topic compacted by userId; consumers rebuild cache from log.
Pros & cons
Pros: Infinite history of latest state per key without full replay cost. Cons: Tombstones need retention to delete keys; not for event-sourcing all revisions.
Common mistakes
Compacting high-churn unique keys (no benefit).
Best practices
Compact config/state topics; append-only audit with time retention.
Follow-up questions
Only answered follow-ups are shown — click to open with full answers