Apache Iceberg
Recent items mentioning Apache Iceberg across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Databricks has significantly advanced its support for Apache Iceberg, with Unity Catalog now offering GA support for Managed Iceberg, Iceberg v3, and Foreign Iceberg, establishing it as a comprehensive and production-ready catalog 4. A major development is the GA release of Iceberg v3, which unifies data layers to enable file sharing across Delta and Iceberg tables without rewriting 2. Looking ahead, Databricks is working towards a unified metadata layer in Delta 5 and Iceberg v4, aiming for full unification later this year 2, and has introduced Lakehouse//RT, a new SQL warehouse designed for real-time analytics directly on data lake formats like Iceberg 1.
Generated daily from the 4 most recent items mentioning Apache Iceberg. Click any [N] to jump to the source.
ReleasesIntroducing Lakehouse//RT and Reyden — Reynold Xin, Co–founder and Chief Architect
Databricks introduces Lakehouse//RT, a new SQL warehouse powered by the Raiden engine, designed to provide millisecond performance and massive concurrency for real-time analytics directly on data lake formats like Delta and Iceberg. This innovation aims to unify data warehousing and serving stacks, eliminating the need for separate systems and data copies.
EventsNo one needs to care about table formats with Databricks' Ryan Blue, creator of Apache Iceberg
Databricks announced the GA release of Iceberg v3, which unifies data layers so files can be shared across Delta and Iceberg tables without rewriting. The company is also working towards a unified metadata layer in Delta 5 and Iceberg v4, aiming for a full unification vision later this year.
Delta Lake 4.3.0
Databricks practitioners can now integrate Spark with the Unity Catalog Delta REST API for managed Delta tables and selectively replace data using new `replaceOn` and `replaceUsing` DataFrame APIs. UniForm for Iceberg conversion is now atomic and incremental, and Delta Sharing supports streaming and Change Data Feed for shared tables.
Unity Catalog and the next era of Apache Iceberg
Advancing Apache Iceberg on Databricks: Iceberg v3 GA, Open Sharing, and Unified Governance
Unity Catalog now offers GA support for Managed Iceberg, Iceberg v3, and Foreign Iceberg, making it the most comprehensive and production-ready Apache Iceberg catalog with open APIs, catalog federation, and secure sharing. Future versions of Iceberg and Delta will converge on a unified metadata structure, eliminating the tradeoff between interoperability and performance.
NewsApache Iceberg V3 on Databricks: From Ingestion to Analytics
The video demonstrates Apache Iceberg v3 on Databricks, showcasing how its new variant column type natively handles semi-structured data and how row-level concurrency enables simultaneous data ingestion and corrections. It also highlights cross-platform data accessibility from open-source Spark via the Iceberg REST catalog, ensuring no vendor lock-in.
This release adds the dbt invocation ID to default query comments and enforces the 255-character identifier length limit for Databricks relations. It also resolves several issues, including spurious warnings for MicrobatchConcurrency and insert_overwrite, and improves handling of materialized views and Iceberg tables.
Interoperability Between Unity Catalog and Google BigQuery via Catalog Federation
Google Cloud now supports catalog federation to Unity Catalog, enabling BigQuery users to read tables in Unity Catalog without duplication. Unity Catalog also supports catalog federation to Google Cloud's Lakehouse, allowing it to read Iceberg tables written from BigQuery and other engines.
Delta Lake 4.2.0
Databricks practitioners gain enhanced Unity Catalog support with new REPLACE TABLE/RTAS and Dynamic Partition Overwrite capabilities, alongside improved streaming reads for catalog-managed tables including `startingTimestamp` and `skipChangeCommits` options. This release also introduces general availability for Variant columns and support for Geospatial and Collations table features, while fixing several bugs related to data skipping, DML operations, and decimal predicates.
Delta Lake 4.1.0
Delta Lake 4.1.0 introduces enhanced support for Unity Catalog managed tables, including batch/streaming read/write and conflict-free feature enablement for Deletion Vectors and Column Mapping. It also requires Java 17 and Spark 4.0.1+, dropping support for Spark 3.5.
EventsAnnouncing full Apache Iceberg™ support in Databricks
Databricks now fully supports Apache Iceberg, offering significantly higher performance for Iceberg tables compared to other vendors. This integration leverages Databricks' optimized engine and Unity Catalog for faster access and better clustering of open-format data.
NewsDatabricks: What’s new in July 2025? Updates & Features Explained! #databricks
NewsIoT for Fun & Prophet: Scaling IoT and predicting the future with Redpanda, Iceberg & Prophet
Delta Lake 3.3.0
Delta Lake 3.3.0 introduces Identity Columns, faster VACUUM LITE, and the ability to enable Row Tracking on existing tables for row-level lineage. It also allows enabling UniForm Iceberg on existing tables without data rewrite and supports Type Widening in Delta Kernel.
Events
