From memtable through SSTables to compacted stores
An LSM tree begins with an in-memory buffer (memtable) receiving all writes. When the memtable fills, it is flushed as an immutable sorted file (SSTable) on disk. New writes go to a fresh memtable. Multiple SSTables accumulate; reading requires checking each to find the latest version of a key. Compaction merges overlapping SSTables into fewer, larger files, improving read performance and freeing disk space.
This append-only design avoids random writes: memtable writes are sequential RAM, SSTable writes are sequential disk I/O. Compaction runs in the background, not blocking active reads or writes. The result is throughput suitable for write-heavy workloads, at the cost of read latency (checking multiple files) and background CPU/disk usage.
Compaction levels and tuning strategies
Level-based compaction groups SSTables by level. Level 0 contains fresh flushes; Level 1 is larger, Level 2 even larger. Compaction merges adjacent levels, maintaining size invariants. This keeps the number of levels (and thus read amplification) logarithmic in total data size. Tiered compaction instead groups SSTables into equal-sized tiers, trading slower reads for fewer compactions.
Compaction tuning is critical: too-frequent compaction wastes CPU and I/O; too-infrequent leaves excessive SSTables and slows reads. Parameters like level multiplier, compaction ratio, and target file sizes control the tradeoff. Write-heavy systems tune for low write-amplification; read-heavy systems prioritize read performance, accepting more compaction overhead.