Apache Iceberg
Recent items mentioning Apache Iceberg across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Databricks has significantly advanced its support for Apache Iceberg, with Unity Catalog now offering GA support for Managed Iceberg, Iceberg v3, and Foreign Iceberg, establishing it as a comprehensive and production-ready catalog 4. A major development is the GA release of Iceberg v3, which unifies data layers to enable file sharing across Delta and Iceberg tables without rewriting 2. Looking ahead, Databricks is working towards a unified metadata layer in Delta 5 and Iceberg v4, aiming for full unification later this year 2, and has introduced Lakehouse//RT, a new SQL warehouse designed for real-time analytics directly on data lake formats like Iceberg 1.
Generated daily from the 4 most recent items mentioning Apache Iceberg. Click any [N] to jump to the source.
Delta Lake 3.3.3
Delta 3.3.3 fixes transaction log retention bugs that broke time travel and CDF reads, and a Delta Sharing deletion vector cache bug causing long-running queries to fail. It adds opt-in RANDOMIZE_FILE_PREFIXES to spread S3 object keys for high-throughput workloads and upgrades Delta Sharing client for improved OAuth and retry handling.
oreign Iceberg tables via HMS federation fail with browse_only_table on all compute — request Public
Delta Lake 4.3.1
Delta Lake 4.3.1 fixes OAuth authentication failures in the Delta REST Catalog caused by incorrect key lowercasing and enables S3A fast listing when using OSS UnityCatalog's CredScopedFileSystem wrapper. It also prevents the reserved is_managed_location property from persisting into managed table metadata.
ReleasesLakehouse//RT, the real-time Lakehouse powered by Reyden — Reynold Xin, Co–founder & Chief Architect
Databricks introduces Lakehouse//RT, a new SQL warehouse powered by the Raiden engine, designed to provide millisecond performance and massive concurrency for real-time analytics directly on data lake formats like Delta and Iceberg. This innovation aims to unify data warehousing and serving stacks, eliminating the need for separate systems and data copies.
EventsNo one needs to care about table formats with Databricks' Ryan Blue, creator of Apache Iceberg
Databricks announced the GA release of Iceberg v3, which unifies data layers so files can be shared across Delta and Iceberg tables without rewriting. The company is also working towards a unified metadata layer in Delta 5 and Iceberg v4, aiming for a full unification vision later this year.
Delta Lake 4.3.0
Delta 4.3.0 deepens Unity Catalog integration by making it the source of truth for managed table operations via the UC Delta REST API, introduces replaceOn/replaceUsing DataFrame APIs for selective row-level data replacement, and improves UniForm with atomic Iceberg conversion and incremental metadata updates. Delta Sharing gains streaming support, Change Data Feed capabilities, and Trigger.AvailableNow, plus performance improvements like better V2 checkpoint parallelization and variant column statistics for data skipping.
Advancing Apache Iceberg on Databricks: Iceberg v3 GA, Open Sharing, and Unified Governance
Unity Catalog now offers GA support for Managed Iceberg, Iceberg v3, and Foreign Iceberg, making it the most comprehensive and production-ready Apache Iceberg catalog with open APIs, catalog federation, and secure sharing. Future versions of Iceberg and Delta will converge on a unified metadata structure, eliminating the tradeoff between interoperability and performance.
NewsApache Iceberg V3 on Databricks: From Ingestion to Analytics
The video demonstrates Apache Iceberg v3 on Databricks, showcasing how its new variant column type natively handles semi-structured data and how row-level concurrency enables simultaneous data ingestion and corrections. It also highlights cross-platform data accessibility from open-source Spark via the Iceberg REST catalog, ensuring no vendor lock-in.
Interoperability Between Unity Catalog and Google BigQuery via Catalog Federation
Google Cloud now supports catalog federation to Unity Catalog, enabling BigQuery users to read tables in Unity Catalog without duplication. Unity Catalog also supports catalog federation to Google Cloud's Lakehouse, allowing it to read Iceberg tables written from BigQuery and other engines.
Delta Lake 4.2.0
Delta 4.2.0 enables atomic REPLACE TABLE, RTAS, and DPO for catalog-managed tables, enhances streaming capabilities, and adds a Kernel-based Flink connector. The release makes Variant generally available, adds geospatial and collation support, and includes comprehensive security hardening.
Delta Lake 4.1.0
Delta Lake 4.1.0 supports Apache Spark 4.1.0 and introduces conflict-free enablement of Deletion Vectors and Column Mapping on existing tables without blocking concurrent writes. The release requires Java 17 and Spark 4.0.1 or higher (dropping Spark 3.5), adds full catalog-managed table support in Delta Kernel for Unity Catalog integration, and fixes MERGE/INSERT struct expansion bugs.
EventsAnnouncing full Apache Iceberg™ support in Databricks
Databricks now fully supports Apache Iceberg, offering significantly higher performance for Iceberg tables compared to other vendors. This integration leverages Databricks' optimized engine and Unity Catalog for faster access and better clustering of open-format data.
NewsDatabricks: What’s new in July 2025? Updates & Features Explained! #databricks
NewsIoT for Fun & Prophet: Scaling IoT and predicting the future with Redpanda, Iceberg & Prophet
Delta Lake 3.3.0
Delta Lake 3.3.0 adds Identity Columns for automatic unique keys, VACUUM LITE for faster transaction log-based cleanup, and enables Row Tracking backfill on existing tables for row-level lineage tracking. UniForm Iceberg can now be enabled on existing Delta tables without data rewriting, and Type Widening is now supported in Delta Kernel for reading type-evolved tables.
Events
