Unity Catalog

The prototyping tax is killing your AI roadmap

Platform-native agents grounded in business semantics—like Databricks Genie plus Unity Catalog—eliminate the "prototyping tax" of fragmented context and siloed domain knowledge, hitting 77% accuracy versus 56–72% for general coding agents at roughly half the cost. Abacus Insights proved the approach even in regulated settings, cutting new-client onboarding time by about 50% and manual data-mapping effort by 40% inside a HIPAA-grade environment.

How a major freight railroad scaled pipeline creation with Genie Code

Freight railroad rolled out Databricks Genie Code to automatically modernize legacy data pipelines, cutting delivery time from days per table to minutes while hitting over 90% automation for new table ingestion. The approach used Genie Code's Agent Skills and Unity Catalog integration to preserve enterprise standards, metadata grounding, and expert review across hundreds of pipelines.

How to ground Genie Agents in both structured data and documents without losing governance

Genie Agents can be grounded across both structured data and unstructured Unity Catalog Volumes, allowing a single agent to answer questions across all your enterprise data. By anchoring governance at the catalog layer with features like Automatic Identity Management, Row Filters, and Column Masks, the agent runs as the user's identity so every answer

What is Tool Calling?

Tool calling enables AI models to interact with external systems and APIs, transforming basic chatbots into agents that execute real-world actions through a structured request-and-response loop. Databricks Agent Bricks provides a governed platform to build these agents grounded in enterprise data, featuring native support for the Model Context Protocol and Unity Catalog

Release: v2.13.0 (#2089)

V2.13.0 adds AI tools support and enables Unity Catalog in Databricks Remote SSH mode. The extension can now start SSH tunnels directly, with the Databricks CLI updated to v1.11.0.

MLflow 3.15.0 introduces an MCP Registry for registering and sharing Model Context Protocol servers, enhances the Assistant with multi-provider LLM support and per-session token usage tracking, and enables proxy-less artifact transfers via presigned URLs to reduce server load and timeouts on large files. Additional improvements include sharable Runs table views, multi-modal image attachments for LLM judges to evaluate vision tasks, and numerous bug fixes across tracing, evaluation, gateway, and UI components.

Agents for production lines: Trusted decisions in real time

Agents like ProdLine CoPilot analyze live plant state from the Databricks Data Intelligence Platform to turn line-manager recovery decisions from a next-morning report into an in-shift answer. Zerobus streams OT telemetry into Delta tables alongside MES, ERP, and LIMS under Unity Catalog, so domain specialists can run real solvers against planners' existing constraints while recommendations stay as draft work orders and holds pending human approval.

Adds catalog_database in v2 catalogs.yml to route Unity catalog models to independent physical catalogs (requires dbt-core 1.12+). Resolves bugs affecting Iceberg Python models, dbt clone operations, non-ASCII column incremental strategies, streaming table tags, and governed tag column drops.

Workspace ID validation for unified-provider resources shifted from plan time to apply time, reducing unnecessary API calls and eliminating false positives for restricted credentials and dynamic workspace IDs. The release adds a databricks_recipients data source for Delta Sharing, trace_location support for MLflow experiment traces in Unity Catalog, and fixes for VIEW column comment updates and access control rule set drift detection.

Simplify AI agent orchestration with Lakebase Postgres

Learn how to turn two Lakebase Postgres tables into a durable, crash-resilient queue for long-running AI agent tasks without operating an external broker, cache, or scheduler. This fully Databr

Why R&D Data Belongs in the Lakehouse - and Why Agents Need It There

Category:* Platform

How Dow Built a Carbon Footprint Ledger on Databricks to Accelerate Sustainability at Scale

CFL) on the

Announcing the Public Preview of Discover and Domains, powered by Unity Catalog

Databricks has released

UnityCatalog 0.5.1

0.5.1 improves credential caching in the Spark connector by keying credentials to resource scope instead of per-request identifiers, reducing latency for repeated access to the same UC tables while maintaining session isolation. The release also fixes permission GET endpoints to work correctly when server-side authorization is enabled, resolving HTTP 500 errors that occurred on secured deployments.

Building a soccer coaching app on Databricks

Coach's Corner is a Databricks App that processes 25 fps match tracking data into a sub-second 2D/3D tactical bench with replays, event analytics, a scout chat, and an opponent-dossier agent. The end-to-end solution is powered entirely on the Databricks platform, utilizing Lakeflow pipelines for data refinement, DBSQL and Lakebase for rapid querying, and Unity Catalog-governed AI tools like Genie, Vector Search, and MLflow tracing.

Meta’s Spark Muse 1.1 is now available on Databricks, fully governed by Unity AI Gateway

Meta's new Spark Muse 1.1 model is now available on Databricks through Unity AI Gateway's Model Provider Services, allowing teams to securely access and govern any model from day one. By registering providers once in Unity Catalog, platform teams can eliminate API key sprawl, enforce centralized permissions and guardrails, and automatically track token usage, costs, and audit logs for end-to-end observability.

Data-Native AI Agents: Why Agents Must Move to Your Data

AI agents must run directly within your data stack rather than in a separate, external stack to avoid compounding penalties like fragmented governance, high egress costs, and latency. By deploying data-native agents on the Databricks Data Intelligence Platform, enterprises can leverage an integrated stack of Unity Catalog, AI Search, MLflow, Lakebase, and AI Gateway to ship secure, trusted AI features faster.

Take insights anywhere with Genie One on mobile

Genie One is now available as a native mobile app for iOS and Android, allowing business users to chat with the AI coworker, schedule tasks, explore dashboards, and use apps directly from their phones. The mobile app delivers the same business context as the web version while fully inheriting your existing security controls, including Unity Catalog permissions, SSO, and network security with no separate mobile backend.

How Unity Catalog managed tables bring interoperability, performance, and unified governance to the Lakehouse

Unity Catalog managed tables now support direct creation and writing from external engines like Apache Spark, Apache Flink, and DuckDB, while maintaining centralized governance and full interoperability. Practitioners can also upgrade existing external tables in place without rewriting data, leveraging Predictive Optimization to automatically improve query performance and lower storage costs.

Introducing Feature Views

Databricks has introduced Feature Views, a managed framework that allows practitioners to define an ML feature once in Unity Catalog and use it consistently across training, batch inference, and real-time serving. This release eliminates training-serving skew and complex infrastructure management, enabling users to rapidly productionize features with a few API calls and serve streaming features at a 200ms end-to-end p99 latency.

Introduces catalogs.yml v2 support, a `skip_optimize` config for opting out of post-materialization OPTIMIZE, and Rust kernel backend for SQL warehouses. Fixes numerous incremental model bugs around constraints and tags, but now requires `--full-refresh` to apply changes to primary/foreign key expressions.

How Imperial College London is accelerating dementia research with a modern data platform

Imperial College London unified IoT, clinical, and research data on a modern data platform, accelerating dementia research and model development. This new architecture leverages Unity Catalog for governed data access and reduced IoT integration timelines from six months to one.

Barracuda makes security logs conversational with Genie

Barracuda Managed XDR now uses Genie to enable natural language search of security logs, letting analysts investigate threats across thousands of customers without SQL or schema expertise. Unity Catalog's row-level security enforces tenant isolation at the data layer, ensuring safe multi-tenant threat detection.

Automatic Upgrades: best practice features for your lakehouse tables

Automatic Upgrades now bring best-practice features like improved performance and reliability to your Unity Catalog managed tables. This first-of-its-kind capability verifies workload compatibility before enabling features, all while keeping them configurable per table.

Reimagining Data Modeling on the Lakehouse: Introducing Vibe Data Modeling

Vibe Data Modeling is now available, an LLM-powered agent that creates analytical Silver-layer business models directly from plain English descriptions, reducing deployment from months to hours. Iterate in natural language to produce new versioned models, validated against 251 rules and redeployed to Unity Catalog, with one logical model supporting many physical layouts.

Breaking changes in this release remove browse-related secret fields from Unity Catalog APIs and replace the Name field with FullName in IAM users. New functionality includes List and ListEffective methods for workspace grants, SQL condition support for Jobs triggers, and added fields across ML and Postgres services.

How the English Office for Students leverages Databricks to enhance higher education standards and drive better student outcomes

The English Office for Students modernized its analytics environment on Databricks to manage millions of student records and support data-informed higher education regulation. By unifying structured, qualitative, and near-live data on a governed platform with Unity Catalog and AI capabilities, they accelerated analysis, improved collaboration, and enabled faster, more trusted decision support.

From test bench to lakehouse: how AVL modernizes measurement data analytics with Impulse

AVL modernized their measurement data analytics with Impulse, an open-source Databricks Labs framework for sensor data analysis. Impulse on Databricks scales time-series analytics to hundreds of terabytes, cutting analysis time from days to minutes while ensuring reproducibility, shareability, and Unity Catalog governance.

What if the answer was already in your data?

Kythera Labs' AI agents, built on Databricks, now provide health system leaders with governed, trustworthy answers to strategic questions from 339 billion claims. A Louisiana health system saw 150% more visibility into patient encounters and $3.8M in estimated annualized value in 10 days.

Databricks positioned highest in execution and furthest in vision for the second consecutive year in Gartner Magic Quadrant

Databricks is recognized as a Leader in the 2026 Gartner Magic Quadrant for AI Platforms for Data Science and Machine Learning, positioned highest in execution and furthest in vision for the second consecutive year. This reflects the market shift towards deploying agentic applications that reason on governed data, enabled by Databricks' unified data, AI, and governance platform with Unity Catalog and Unity AI Gateway.

python-v1.6.1: Column Mapping write support

This release adds column mapping write support and BlindDeltaTable for stats-free appends, alongside improvements to data skipping and partition pruning. S3DynamoDbLogStore has been removed in preparation for 1.0.0.

Genesis Workbench: A blueprint for industry AI in life sciences, powered by Databricks and NVIDIA

Genesis Workbench is a new Databricks blueprint integrating NVIDIA BioNeMo and Parabricks into a secure, no-code environment for end-to-end drug discovery. It centralizes data and eliminates external API dependencies, streamlining research from hypothesis to therapeutic candidate with Unity Catalog governance.