Skip to content
All topics

Apache Iceberg

Recent items mentioning Apache Iceberg across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.

47 recent items11 releases15 news21 community threads
What's happening in Apache IcebergAI synthesis · updated Nov 18, 2026

Unity Catalog now supports multi-statement transactions that execute atomic writes across multiple Delta and Iceberg tables, removing the need for complex reconciliation logic or separate governance layers 1. Cross-platform interoperability also advanced with dbt Mesh enabling zero-copy Iceberg table sharing between Databricks and Snowflake catalogs 10, alongside new Databricks Go SDK support for Iceberg-backed machine learning data sources 2.

Generated daily from the 10 most recent items mentioning Apache Iceberg. Click any [N] to jump to the source.

Reddit

Spark Declarative Pipelines - possible to write UC Managed Iceberg Tables instead of materialized views?

Hi All, Based on my experience using SDP, the tables are typically rendered as "Materialized Views". My company is making a decision to move towards Managed Iceberg for tables and I was wondering if creating Managed Iceberg tables is supported by Databricks's native SDP capabilities? submitted by /u/RazzmatazzLiving1323 [link] [comments]

00RazzmatazzLiving13233d ago
Reddit

Inside Apache Iceberg V3: Deletion Vectors and Puffin Files

submitted by /u/Low_Brilliant_2597 [link] [comments]

00Low_Brilliant_25975d ago
Reddit

Streamhouse and Lakehouse with Apache Iceberg

submitted by /u/codingdecently [link] [comments]

00codingdecently6d ago
Reddit

Catalog Linked Database - Snowflake writes UC Managed Iceberg tables to Unity Catalog

Is Unity Catalog REST API only accessible via public internet from Snowflake when attempting to set-up a private link connection (catalog integration) to a Unity Catalog (Catalog Linked Database)? I need a private link connection between Databricks and Snowflake. The use case is Snowflake needs to be able to write Managed Iceberg tables to Unity Catalog. submitted by /u/RazzmatazzLiving1323 [link] [comments]

00RazzmatazzLiving13231w ago
Reddit

Apache Iceberg TCO: The Real Cost of Lakehouse Table Maintenance

submitted by /u/codingdecently [link] [comments]

00codingdecently4w ago
Reddit

Real-Time Lakehouse Guide: Apache Iceberg in Production

submitted by /u/codingdecently [link] [comments]

00codingdecently1mo ago
Reddit

Automatic CDF

The recipe is simple: combine two great features of Delta, time travel and row tracking ID, and you will get Automatic Change Data Feed. As Elizabeth Bowman, databricks PM said: "Teams should be able to use table changes without asking an administrator to enable a separate feed. Automatic CDF makes this possible on supported Delta and managed Iceberg tables while reducing the write overhead of legacy CDF." https://medium.com/databrickscommunity/automatic-change-data-feed-cdf-is-here-to-save-space-improve-write-time-and-feed-change-data-076131a6abb0 https://www.sunnydata.ai/blog/automatic-cdf-databricks submitted by /u/hubert-dudek [link] [comments]

00hubert-dudek1mo ago
Reddit

Writing to externally managed Iceberg tables from Databricks: am I missing something?

I recently joined a new company as a data architect and I'm trying to understand how things work here today (new to Databricks, main experience with RedShift and BigQuery), and I'd like corrections from people who use Databricks daily. I've been testing how Databricks handles Iceberg tables that are managed by Glue. What I'm seeing: Can read tables as Databricks' federation lets me mount the external catalog and query the tables as foreign tables. Can't write (is this accurate?) INSERT, MERGE, and UPDATE operations against those foreign Iceberg tables fail, and the Databricks docs seem to confirm this. Question: Has anyone gotten writes to non-UC Iceberg tables working from Databricks? That could be through a config, a preview feature, or an open-source Spark Iceberg runtime with a REST catalog on a cluster. If I've got any of this wrong, please correct me as I am still getting up to speed with the platform. submitted by /u/DonutSignificant4174 [link] [comments]

00DonutSignificant41741mo ago
Reddit

The Story of iceberg-rust: The Official Rust Implementation of Apache Iceberg

The story of iceberg-rust is a great example of how open-source communities work and how new projects are created and developed. That’s the spirit of the community-over-code philosophy. submitted by /u/Low_Brilliant_2597 [link] [comments]

00Low_Brilliant_25972mo ago
Reddit

How AI Agents Query Apache Iceberg Data with MCP

submitted by /u/codingdecently [link] [comments]

00codingdecently2mo ago
Reddit

Unity Catalog: Pros and Cons

Apache Iceberg won the open table format war when Databricks acquired Tabular, followed by its subsequent adoption across the industry. Then, the catalog war began. In a lakehouse, storing data in object storage and using the Apache Iceberg format is only part of the story. You also need a catalog that helps lakehouse query engines like Spark, Flink, or RisingWave discover tables, manage metadata, enforce access control, and work with governed data across different systems. That is where Unity Catalog comes in. submitted by /u/Low_Brilliant_2597 [link] [comments]

00Low_Brilliant_25972mo ago
Reddit

Apache Iceberg Table Cleanup: A Production Guide

A guide to Iceberg table cleanup — snapshot expiration, orphan file removal, manifest rewriting, delete file resolution, streaming challenges, compliance, and cost. Why sequencing matters, where teams break tables, and how to automate the full lifecycle. submitted by /u/codingdecently [link] [comments]

00codingdecently2mo ago
Reddit

Apache Iceberg Compaction Best Practices

submitted by /u/codingdecently [link] [comments]

00codingdecently2mo ago
Reddit

Automating Apache Iceberg Table Maintenance

submitted by /u/codingdecently [link] [comments]

00codingdecently2mo ago
Reddit

Apache Iceberg Performance Optimization: Queries to Tables

submitted by /u/codingdecently [link] [comments]

00codingdecently2mo ago
Reddit

Data Lakehouse with Apache Iceberg: A Guide

submitted by /u/codingdecently [link] [comments]

00codingdecently2mo ago
Reddit

Maintaining Apache Iceberg Tables: Compaction, Snapshots, Metadata and Orphan Files

submitted by /u/codingdecently [link] [comments]

00codingdecently2mo ago
Databricks CommunityWarehousing & Analytics

Got error using Opensharing Get Table API to foreign iceberg table

002mo ago
Reddit

[Announce] Apache Iceberg Virtual Meetup Series

submitted by /u/PrideDense2206 [link] [comments]

00PrideDense22063mo ago
Databricks CommunityData Engineering

oreign Iceberg tables via HMS federation fail with browse_only_table on all compute — request Public

003mo ago
Databricks CommunityAnnouncements

Unity Catalog and the next era of Apache Iceberg

005mo ago

Get Tuesday's version of this

Tracking Apache Iceberg? The Tuesday email carries what moved across the whole ecosystem, not just this topic. Free, one-click unsubscribe.

Read past issues first