Cargando el Technology Radar
Open table lakehouses use open table formats such as Apache Iceberg, Delta Lake, and Apache Hudi to bring database-like reliability to object storage. Open table formats store tabular data in files and maintain metadata about data and operations separately, enabling ACID transactions, time travel, schema enforcement and evolution, data skipping, and CRUD operations (Delta Lake).
Apache Iceberg describes itself as an open table format for analytic datasets that lets engines such as Spark, Trino, Flink, Presto, Hive, and Impala safely work with the same tables at the same time (Apache Iceberg). This engine interoperability is why open table formats are now a default foundation for analytics, ML training, and AI-ready data products.
This belongs in Adopt for new analytical data products. The main architectural decision is no longer whether to use a table format, but which table format, catalog, governance model, and compute engines should define the platform standard.
Format choice still matters. Delta Lake, Iceberg, and Hudi have different metadata structures, transaction models, APIs, ecosystem strengths, and governance integrations, so uncontrolled multi-format adoption can fragment the platform (Delta Lake).
Performance still requires data engineering. File sizes, clustering, compaction, partition evolution, delete handling, metadata growth, and query-engine tuning must be managed deliberately.
Catalog and governance are as important as the files. A lakehouse without clear ownership, lineage, access control, retention, and table lifecycle policies can recreate the same trust problems as an unmanaged data lake.
Migration can expose hidden coupling. Legacy Hive-style partitions, downstream Spark assumptions, BI connector limitations, and vendor-specific table features can complicate portability.
Adopt an open table lakehouse for new data products where long-term portability, governance, cost control, reproducibility, and AI-ready data access matter. Standardize on a primary table format and catalog, and define explicit rules for compaction, schema evolution, delete handling, access control, lineage, and engine compatibility.
Avoid letting each team choose its own format by default. Multi-format support can be valuable, but the platform should make interoperability an intentional capability rather than an accidental architecture.