The Convergence of Open Table Formats and Open Catalogs: Catalog Commits is Generally Available
Summary
Catalog Commits for Unity Catalog managed tables is now generally available, aligning Delta Lake with a catalog-oriented model to coordinate table discovery, access, and state across engines. The architectural upgrade eliminates metadata drift between external engines, centralizes cross-engine governance without static storage paths, and unlocks multi-statement, multi-table transactions on the lakehouse.
Summary generated by brickster.ai. For the full article, follow the source link above.
More from Databricks Blog
Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks
A complete reference architecture is now available for building a real-time, multi-stage e-commerce recommendation engine on Databricks using AI Search for candidate retrieval, Lakebase for online feature serving, and Model Serving for low-latency inference. By unifying batch precomputation and real-time scoring on a single governed platform, this design replaces fragmented ML stacks to eliminate glue code and deliver end-to-end lineage from raw clickstream to production predictions.
Read Restrictions and Catalog Labels: Unifying governance across engines and catalogs
Apache Iceberg has introduced read restrictions and catalog labels, two new specifications designed to standardize policy enforcement across different engines and catalogs. These additions eliminate fragmented enforcement by delegating access decisions to trusted engines and making governance metadata portable across federated catalogs.
Lakebase Postgres branch-based restores for fast recovery at scale
Lakebase Postgres introduces branch-based restores, a metadata operation that instantly recovers databases by pointing to immutable history in decoupled storage rather than copying data. This capability cuts recovery times for 100 TB databases down to seconds, eliminating prolonged downtime and enabling seamless branching and undo workflows for AI agents.
Connecting customer context to measurable ROI with agentic marketing
Agentic marketing relies on AI agents grounded in real-time, governed customer context and identity resolution to recommend next-best actions within defined guardrails. Moving incrementality measurement directly into this decision loop proves what changed because marketing acted, providing a shared basis for marketing and finance investment decisions.
IP Functions are Generally Available, bringing high-performance network analytics to the Lakehouse
Native IP functions are now generally available on Databricks Runtime 18.3+, providing built-in support to parse, validate, canonicalize, and join IPv4 and IPv6 addresses and CIDR blocks without UDFs or regex. Optimized in Photon across SQL, PySpark, and Scala, these native functions run demanding network workloads in seconds and complete IP CIDR joins up to 3.1x faster and 6.4x cheaper than another leading cloud data warehouse.
Introducing ai_decide: make fast decisions on your governed data
Databricks has introduced ai_decide, a new AI Function that processes unstructured text to return structured decisions faster and at lower cost than traditional LLMs. Optimized for decision-making rather than text generation, it runs directly on governed data at batch scale in SQL or in real time over REST for tasks like prompt routing, metadata tagging, and agent quality evaluation.