File Encryption for the Lakehouse: The Terminology, the Machinery, and the Hard Problem of Interoperable Encrypted Tables

Short summary

A deep technical guide on lakehouse encryption covering Parquet Modular Encryption and Apache Iceberg 1.11's new table-level encryption with envelope-encryption and three-tier key hierarchy. The article explains why client-side encryption is needed beyond server-side SSE, and works through the interoperability challenge of multiple query engines reading encrypted tables. It also covers operational realities like key rotation, crypto-shredding, and disaster recovery.

• •Iceberg 1.11 shipped table-level encryption with envelope encryption and catalog as key broker
• •Server-side encryption (SSE) doesn't protect against bucket-level breaches; client-side encryption is needed for regulated data
• •Interoperability across heterogeneous query engines is the core distributed-systems challenge for encrypted lakehouses

Generated with AI, which can make mistakes.