Apache Iceberg V3: How Deletion Vectors Rewrote Delete Economics
A deep dive into why deleting rows in Apache Iceberg is hard, how v2's delete files work, and how v3's deletion vectors reshape update economics.
Deleting a single row in an object-storage-backed data lake turns out to be far harder than it sounds, because systems like S3 never allow in-place edits to existing files. Apache Iceberg addresses this immutability constraint with two strategies: copy-on-write, which rewrites the whole affected file, and merge-on-read, which leaves the original file untouched and instead records separate 'delete notes.' Iceberg v2 made the merge-on-read approach production-ready through position-based and equality-based delete files, enabling real CDC pipelines and targeted row deletes.
At scale, though, v2's position delete files revealed a serious accumulation problem. A frequently updated data file can end up referenced by dozens of small delete files across commits, forcing readers to locate, open, and merge all of them just to know which rows survive. As AWS's analytics team has noted, this pattern places heavy strain on query engines through excessive small file reads and costly in-memory reconciliation.
Iceberg v3's deletion vectors are designed to fix exactly this pain point, replacing scattered delete-file sprawl with a single, more stable representation that is far cheaper to apply on read. The shift is more than a storage implementation detail — it directly determines how well lakehouse architectures can support update-heavy workloads like CDC, streaming ingestion, and targeted deletes.