News from the Databricks ecosystem.
Posts from databricks.com, MLflow, and dbt Labs — three trusted sources covering the platform, the open-source projects around it, and the data tooling layer most teams pair with it. Summarized for scanning.
Last week
27 articlesIntroducing Omnigent: A Meta-Harness to Combine, Control and Share Your Agents
Omnigent, an open source meta-harness, is now available to combine, control, and share your AI agents across various models and interfaces. It enables building agent teams, controlling them with policies, and sharing live sessions with teammates.
From Wall Street to Data Platforms
Databricks values deep industry expertise, as shown by Kim Hatton’s transition from finance to helping financial institutions solve modern data challenges. Our collaborative environment encourages employees to grow beyond their core roles and contribute to industry innovation, building practical tools that turn complex data tasks into streamlined successes.
Enabling Evolutionary Database Development: Database branching with Lakebase, the conclusion
Lakebase now supports database branching, enabling evolutionary database development. This concludes the series on Lakebase's operationalization of evolutionary database design.
Talk to all your data, wherever it lives
Lakehouse Federation is now available, allowing you to query data across all sources without migration delays. Unity Catalog serves as the single source of truth for both federated and managed data, enabling secure AI workloads and natural language querying.
What is customer segmentation?
Customer segmentation combines multiple types and methods, from rule-based to AI/ML-driven models, but its success hinges on unifying fragmented customer data into a governed Customer 360. Databricks' CustomerLake, an Agentic CDP, builds segments directly on governed data with AI-driven identity resolution and natural-language audience creation, eliminating data copies and extra vendors.
Unlocking semantics for AI: How Mercedes-Benz Korea built trusted “Talk to Data” at scale
Mercedes-Benz Korea built a trusted "Talk to Data" solution at scale by making 500+ KPI definitions available in an AI-ready semantic layer on Unity Catalog metric views, accelerating the transition with an automated DAX-to-Metric-View transpiler. This governed semantic layer supports both existing BI and new "Talk to Data" experiences, with Genie and Agent Bricks providing consistent answers and shaping a playbook for persona-based AI agents across markets.
Forward Deployed Engineering: Delivering Business Outcomes with AI
Databricks is launching its Forward Deployed Engineering (FDE) organization to accelerate customer business outcomes with AI, pairing the Lakehouse platform with embedded, engineering-led delivery. This new approach moves beyond migration and pipeline building to solve business problems with production AI agents, as demonstrated by customers like Fox, JPMC, and Qualcomm.
Ingesting the Milky Way: Petabyte-Scale with Zerobus Ingest
Zerobus Ingest, a new serverless streaming API, enables instant deployment of petabyte-scale data pipelines on Databricks without manual infrastructure management. Its dynamic partitioning architecture automatically scales compute and sustains over 12 GB/s throughput to a single table, efficiently handling unpredictable data volumes.
How ERGO Hestia reduced time-to-market with Lakebase and Mosaic AI Model Serving
ERGO Hestia modernized its real-time pricing engine with Databricks Lakebase and Mosaic AI Model Serving, reducing time-to-market by unifying data, features, and decisions for millisecond pricing. This eliminated extraction overhead and fragmented governance from their previous multi-hop architecture, enabling faster model deployment and instant market response.
Welcoming the first cohort of Databricks student fellows
Databricks launched its inaugural Student Fellows cohort, selecting a diverse group of students to bridge academic theory and real-world data and AI practice. These fellows will host workshops, hackathons, and mentorship programs at their universities, with five standout individuals from top schools already making significant contributions.
Geospatial Unbounded: Spatial SQL GA with AI/BI Maps, Delta Sharing, and Iceberg v3
Spatial SQL is now Generally Available on Databricks, bringing native geospatial data types, 90+ ST_* functions, and AI/BI Dashboards that render maps natively. This release also includes major performance improvements, open lakehouse support via Delta Sharing and Iceberg v3, and Apache Spark 4.2 compatibility for geo columns.
Azure Databricks at Data + AI Summit 2026 featuring Industry Leaders and Partners
Azure Databricks at Data + AI Summit 2026 featured new joint product announcements and integrations, alongside key sessions on zero-copy federated analytics and ecosystem co-engineering. Learn how joint customers are modernizing data estates, scaling AI, and unlocking business value with Azure Databricks.
Empower your healthcare agents with ready-to-use MCP on Databricks Marketplace
Databricks Marketplace now offers ready-to-use biomedical and clinical Model Context Protocol (MCP) servers from partners like Climb and Atropos Health, empowering healthcare agents. Easily build and deploy bespoke agents to production, leveraging a securely governed, centralized MCP Catalog that also supports your own custom MCP servers or data.
How Ecolab rebuilt retail intelligence on Databricks and Anthropic Claude
Ecolab rebuilt retail intelligence on Databricks and Anthropic Claude, converting 700-page FDA manuals into real-time answers for frontline staff using Foundation Model APIs and cutting compliance report compilation from two weeks to under two minutes. The solution, a native Databricks App with Lakebase Postgres and Unity Catalog, unifies nine siloed data sources and employs a multi-agent orchestration framework with Judge LLMs and MLflow tracing for personalized, continuously refined intelligence.
Stop building data products. Start building data services.
The one-product-per-use-case model breaks down under acquisition-driven growth and agentic consumption; a services layer is more adaptable to what comes next. Moving data mastering and quality checks closer to ingestion makes integration cycles measured in weeks possible, reducing insight lag.
Scaling AI Through Data Fluency
Aer Lingus built a solid data foundation with governance and quality, treating data literacy as a core business skill with a custom curriculum. This enabled real-time insights for optimizing flight loads, pricing, and operations decision-making.
AWS and Databricks at Data + AI Summit 2026: Accelerating real-world AI innovation
AWS and Databricks are accelerating real-world AI innovation, from Mastercard experimentation to production-scale AI, as showcased at Data + AI Summit 2026. Explore breakout sessions, industry forums, and hands-on demos covering agentic AI, governance, open data architectures, and multi-engine interoperability with Amazon Bedrock and Kiro.
Announcing the Public Preview of Custom URLs
Databricks accounts can now use a single, branded custom URL like mycompany.databricks.com, replacing individual workspace URLs. This simplifies login and navigation across multiple workspaces, enabling account-wide features like Genie and Unity Catalog lineage.
AI Serving Platform That Adapts to Your Model
Databricks now offers a fully managed AI serving platform that automatically adapts to your model's resource needs, from scikit-learn to 70B LLMs, without manual configuration. This results in up to 90% lower infrastructure costs and <10ms p99 latency overhead for customers migrating from self-managed stacks.
Announcing the Databricks storage ecosystem: Governing the enterprise data estate, wherever it lives
The Databricks Storage Ecosystem now natively connects hybrid and on-premises storage platforms to Databricks via OpenSharing, enabling centralized data governance and GenAI scaling across your entire hybrid infrastructure. Run Databricks Serverless Compute, Genie, and LLMs directly on your on-premises datasets with a zero-copy architecture, instantly turning isolated data into active, AI-ready assets.
Modern BSA/AML compliance on Databricks
Databricks now offers a unified, AI agent and ML-augmented experience for BSA/AML compliance, consolidating siloed systems and accelerating SAR report building. AML teams can expect 8-10x faster case processing, a 75% reduction in false positives, and $50-150 million in annual cost savings.
Claude Fable 5 is now available on Databricks, fully governed through Unity AI Gateway
Claude Fable 5 is now available on Databricks, accessible through Unity AI Gateway for centralized governance, cost controls, and observability. This Anthropic model offers state-of-the-art performance across enterprise workflow automation, agentic search, data reasoning, and multimodal document understanding.
Announcing the winners of the 2026 Databricks Customer Awards
The 2026 Databricks Customer Awards winners have been announced, recognizing 10 customers for excellence, innovation, transformation, social impact, and leadership. These winners span diverse industries and regions, showcasing how they leverage Databricks to solve complex data and AI challenges.
Announcing the 2026 Databricks Customer Awards Industry winners
The 2026 Databricks Customer Awards Industry winners have been announced, recognizing ten organizations across diverse sectors like financial services, healthcare, and manufacturing. These winners showcase compelling data and AI stories, demonstrating how they've leveraged Databricks to solve complex challenges and achieve measurable results.
Transforming solar and wind maintenance reports with Genie and AI agents
Plenitude now converts unstructured solar and wind maintenance PDFs into a unified, queryable data model using Databricks Genie and AI agents. This enables natural-language querying and visualizations across plants, accelerating multi-plant analysis and laying the groundwork for predictive maintenance.
Your AI isn't broken. Your data model is.
Databricks practitioners, your AI isn't broken; your data model is. The gap between successful AI proof of concepts and failed production deployments stems from your data model, not your AI model.
Enterprise Data Strategy Roadmap for Business Outcomes
* A robust enterprise data strategy connects organizational data assets to specific business objectives through governance, architecture, and analytics frameworks that scale with evolving business needs. * Effective data governance, data quality management, and master data manage
Week of Jun 1
21 articlesEnabling Evolutionary Database Development: database branching with Lakebase, continued
This series revisits the methodolgy of Evolutionary Database Design, twenty years...
Data + AI Summit 2026: Insider’s Guide for Financial Services Leaders
Data + AI Summit 2026 offers a financial services executive guide to key banking, insurance, payments, and capital markets sessions. Learn how leading organizations like Morgan Stanley and JPMorganChase are approaching AI transformation, responsible AI, and operational modernization, with practical strategies for maximizing summit value.
Your guide to the Telecommunications Industry Experience at Data and AI Summit 2026
The Data + AI Summit 2026 will feature a Telecommunications Industry Experience, showcasing how global carriers are leveraging data and AI to address customer experience, network operations, and fraud. Attendees will gain insights into AI agents for autonomous networks, churn prevention, and Genie-powered conversational intelligence for frontline teams.
3x Faster Search: Parallel Test-Time Scaling with Instructed-Retriever-1
Instructed-Retriever-1 now delivers 3x faster search for Agent Bricks Knowledge Assistant. This parallel test-time scaling update also improves quality for Databricks practitioners.
Apache Spark Real-Time Mode for Gaming: A Better Way to Do Real-Time Sessionization
Apache Spark Real-Time Mode now enables real-time gaming sessionization for millions of active device sessions, replacing custom applications with sub-second precision for both input processing and timer-driven output. Learn how transformWithState timers power proactive, timer-driven heartbeats, generating output on a schedule independent of incoming data.
Bring Databricks into Kiro IDE with the AI Dev Kit Power
The Databricks AI Dev Kit Power now offers a one-click setup to integrate Kiro IDE with the full Databricks platform, providing AI-assisted development grounded in your workspace's Unity Catalog metadata. This new path, alongside a lighter PAT-based option, ensures your AI assistant writes SQL with actual columns and respects all row, column, and tag-based grants.
Building a data stack for trusted AI
Databricks now offers a data stack for trusted AI, providing governed, consistent, and contextual data. Learn how to build it without tying yourself down.
Scaling Enterprise Conversational Intelligence: Cross-industry Technology and Functional Solutions Powered by Databricks Genie
Databricks Genie now powers cross-industry conversational intelligence solutions from leading partners, offering ready-to-deploy offerings for sales, marketing, HR, finance, and other enterprise functions. These innovative solutions accelerate AI transformation by addressing technology and function-specific use cases across the enterprise.
Beyond parsing X12: Closing the gap for revenue cycle workflows in healthcare
Healthcare billers now have an operational workbench built on Unity Catalog gold views, providing a purpose-built UI with a denials queue, remittance drawer, and timely-filing age alerts directly on their fully parsed 835/834/837 EDI data. This solution integrates GenAI via Databricks Foundation Model APIs to auto-draft appeal letters, moving billers beyond manual spreadsheet and SQL work to review and approve instead of writing from scratch.
dbt Labs Named Snowflake Data Integration Product Partner of the Year
dbt Labs was named Snowflake Data Integration Product Partner of the Year. This post details dbt Labs' two Snowflake Partner honors, including the CoCo Adoption Award.
Agentic BI: A Practical Guide for BI Teams and Business Users
Agentic BI, which embeds autonomous AI agents into analytics workflows, automates data prep, query execution, and insight delivery to replace static dashboards and address dissatisfaction with current insight generation. A governed semantic layer is critical for trustworthy agentic analytics, and adoption can be incremental, starting with a pilot and expanding based on documented outcomes.
Data Science vs Data Analytics: Compare Careers, Skills, and Degrees
Data analytics explains what already happened using SQL and Power BI, while data science builds ML models to automate future decisions. Choosing between them depends on your appetite for technical depth, comfort with unstructured data, and preference for stakeholder communication vs. system deployment.
AI in Defense: How Artificial Intelligence Is Reshaping National Security
AI is rapidly reshaping national security as nations accelerate military AI development, creating a global race with strategic consequences. Responsible AI governance, model validation, and human oversight are essential safeguards as defense organizations deploy autonomous systems and machine learning in combat operations.
Data Governance Architecture: A Complete Blueprint for Modern Organizations
This blueprint details a complete data governance architecture, outlining the policies, roles, and technologies needed to manage data assets. It emphasizes a modern strategy combining automated lineage, RBAC, and federated models to ensure data quality and regulatory compliance at scale.
Query Tags: The Context Your Warehouse Queries Have Been Missing
Databricks SQL warehouses now support query tags, enabling cost attribution by team or project and automatic tagging for dbt, PowerBI, and Tableau. Tag queries from any source, including the SQL Editor, Notebooks, Dashboards, APIs, connectors, and drivers.
Introducing Cross-Engine ABAC
Unity Catalog now enforces attribute-based access controls (ABAC) on external engines, allowing you to define tag-based row filters and column masks once for enforcement from any engine. This centralized governance at the catalog layer, built on Iceberg REST Catalog scan APIs, ensures policies are enforced before data reaches the engine.
Personalizing Genie Code with instructions, skills, memory, and MCP
Genie Code now personalizes to your conventions with Instructions, Skills, and MCP Servers, allowing reuse of team workflows, internal docs, and external tools without repeated pasting. Leverage personal skills for individual work, workspace skills for shared team workflows, and admin-approved MCP servers for scalable external context in agent mode.
Debunking 8 data layout myths: why Liquid Clustering outperforms partitioning
Liquid Clustering is the data layout for open table formats that outperforms partitioning, and this post debunks 8 common myths keeping teams tied to partitioning. Customers using Liquid Clustering report dramatic improvements in query latency, write throughput, storage efficiency, and data freshness, with the largest gains compounding at petabyte scale.
What we announced at Snowflake Summit and why it matters
dbt State, dbt Wizard, dbt Core v2.0, and the Fivetran merger
Fivetran + dbt Labs Complete Merger to Create the Data Infrastructure for Trusted AI Agents
Fivetran and dbt Labs have completed their merger, creating a unified company focused on building the data infrastructure for trusted AI agents. This new entity aims to provide the foundational data layer necessary for the agentic AI era.
Week of May 25
12 articlesEnabling Evolutionary Database Development: database branching with Lakebase
Why this series existsThe methodology described in Evolutionary Database Design and...
AI Doesn't Scale Until You Stop Calling It Innovation
Schneider Electric solutions leveraging Databricks can reduce energy costs by up to 20 percent, demonstrating that scaling AI requires focusing on business value and customer need over technology selection. The fastest-scaling companies combine domain expertise with AI knowledge through dedicated, end-to-end teams.
Databricks at SIGMOD 2026
Spark Declarative Pipelines (SDP) are simplifying complex ETL and streaming workloads, pioneering the next generation of data engineering. Get a deep dive into Enzyme, our incremental view maintenance engine, which won an honorable mention at SIGMOD.
Winning under CMS TEAM: Building the learning health system to realize success in VBC today and tomorrow
Databricks helps healthcare providers succeed under the mandatory CMS TEAM program by building an AI-enabled data foundation for proactive, data-driven intervention. This enables a unified view across clinical and claims data, embedding predictive insights into care workflows to reduce SNF costs by 15% and readmissions by 12%.
How enterprise leaders are scaling AI agents across their organization
Databricks practitioners can learn five key practices for scaling agentic AI responsibly across enterprise core workflows like HR, finance, and fraud detection. This post helps leaders deliver rapid gains from AI agents while maintaining governance, trust, and cost control.
Advancing Apache Iceberg on Databricks: Iceberg v3 GA, Open Sharing, and Unified Governance
Unity Catalog now offers GA support for Managed Iceberg, Iceberg v3, and Foreign Iceberg, making it the most comprehensive and production-ready Apache Iceberg catalog with open APIs, catalog federation, and secure sharing. Future versions of Iceberg and Delta will converge on a unified metadata structure, eliminating the tradeoff between interoperability and performance.
Reliable LLM Inference at Scale
Databricks now offers model units, a VM-like abstraction for allocating and scaling GPU resources per customer, enabling cost-aware load balancing and autoscaling that saved over 80% in GPU costs. Runtime reliability mechanisms like black-box health checks and multimodal bottleneck profiling further improve throughput and recover from silent failures automatically.
BI Serving Pointers; Maximizing for Performance and TCO
Databricks now offers Unity Catalog Metric Views for a headless semantic layer, enabling governed business metrics across all BI tools and AI agents. Maximize performance and TCO by structuring your physical layer with star schemas, liquid clustering, and Predictive Optimization, and leverage aggregate-aware materialization for OLAP-style performance.
Introducing Always-On pricing: automatic savings for Databricks Lakebase
Databricks Lakebase now offers Always-On pricing, providing serverless flexibility with a 25% lower price on baseline capacity for established production workloads. Activate with a single toggle to disable scale-to-zero and set an autoscaling range, then after 24 hours of continuous use, baseline capacity bills at the Always-On rate while spikes bill at standard Autoscaling rates.
How the lakebase architecture stays resilient to cloud failures
Lakebase's architecture is built for resilience to cloud failures, not patched for it, by using stateless Postgres compute on zone-redundant storage and separating hot-path control-plane operations. This approach, validated through chaos testing and per-database availability tracking, addresses the unique reliability demands of agent workloads that start tens of millions of databases daily.
Announcing Lakebase Change Data Feed (CDF)
Lakebase Change Data Feed (CDF) is now in Public Preview, eliminating pipeline sprawl from operational databases by exposing every table's changes through Unity Catalog Managed Tables. This enables native CDC governed end-to-end without sidecar infrastructure, allowing operational data to function as the native Bronze layer in the medallion architecture.
Building a FHIR-native health data platform on Databricks Lakebase
Health Samurai's Aidbox now runs natively on Databricks Lakebase, providing a FHIR-native health data platform that standardizes clinical data at ingestion and makes it instantly available for Spark, ML, and AI. This architecture inherently delivers compliance with CMS-0057 and ONC mandates, eliminating the need for separate compliance workstreams.