Lakehouse Encryption Explained: Parquet, Iceberg, and the Interop Puzzle
A deep dive into Parquet and Iceberg 1.11 encryption architecture, key terminology, and the interoperability challenges of multi-engine lakehouses.
Server-side bucket encryption only protects against physical storage compromise; a leaked credential or overly broad IAM role still reads plaintext, because the storage service decrypts transparently for any authorized caller. Real protection requires client-side encryption, where data is encrypted before it reaches storage under keys the storage layer never holds. In a lakehouse, where multiple query engines from different vendors must read the same files, this becomes a genuinely hard distributed-systems problem spanning key management, metadata, and integrity guarantees.
By 2026 the pieces finally arrived: Apache Parquet's modular encryption matured into broad practical use, and Apache Iceberg 1.11 shipped table-level encryption as a headline feature, a full envelope-encryption design with a three-tier key hierarchy, encrypted metadata, and the catalog acting as key broker.
What's still missing is widespread understanding of how these pieces fit together, and in cryptography, partial understanding is more dangerous than none, a misconfigured system can look secure while providing no real protection. Engineers need to understand DEK/KEK envelope hierarchies, authenticated encryption via AES-GCM, the role of AAD in binding context to ciphertext, and operational practices like key rotation and crypto-shredding, all of which must work consistently across a fleet of heterogeneous query engines.