Everything across the Databricks world, ordered for you.
News, releases, videos, GitHub projects, and community Q&A merged into one feed. Pick a role below to reorder it for your work.
Reorder for your role
From Wall Street to Data Platforms
Databricks values deep industry expertise, as shown by Kim Hatton’s transition from finance to helping financial institutions solve modern data challenges. Our collaborative environment encourages employees to grow beyond their core roles and contribute to industry innovation, building practical tools that turn complex data tasks into streamlined successes.
The Hidden Logic: How AI Transforms Your Data 🧐
AI models implicitly convert string-based categorical data, like sentiment (positive, negative, mixed), into numerical representations. This conversion is essential for performing mathematical operations, such as calculating an average sentiment.
AI-Powered Data Cleaning in Databricks! 📊🤖
Databricks demonstrates using an AI assistant to clean data by providing an image of desired output. The AI transforms the existing data to match the structure and content shown in the attached image.
Is Your Azure Databricks Storage Exposed? (Enable Firewall now)
The video demonstrates how to enable firewall support for an Azure Databricks workspace storage account, preventing public network access. It walks through creating private endpoints, an access connector, and then executing a PowerShell command to configure the firewall and network security perimeter.
Databricks: Future of Storage Security Revealed!
Databricks is onboarding existing workspace storage accounts with enabled firewalls to Network Security Perimeter (NSP). This allows users of Databricks serverless to leverage enhanced storage security.
Import Local Files to Databricks Easily! ✨
Databricks Lake Designer now allows users to easily import local files by dragging and dropping them onto the canvas. This feature simplifies bringing personal datasets into Databricks for analysis, addressing the common need to use data not yet stored in the platform.
Pro Tip: Add Multiple Tables Fast! 🚀
Users can quickly add multiple tables to a canvas by dragging them directly from the Catalog Explorer left panel. This method streamlines the process of adding several tables from the same schema or catalog, avoiding the need to create individual source nodes.
Enabling Evolutionary Database Development: Database branching with Lakebase, the conclusion
Lakebase now supports database branching, enabling evolutionary database development. This concludes the series on Lakebase's operationalization of evolutionary database design.
Data + AI Summit - AI Recap + Q&A
If you couldn't make it to San Fran, or keep up with four days of announcements, demos, and deep-dive sessions from across the pond, then don't worry, we’ve got you covered. Join Gavi and James for a no-nonsense recap of the key highlights, and more importantly, what they actually mean for your org…
Building Real AI Agents (Fast!) | Microsoft Agent Framework Foundations | Part 2
The video demonstrates building AI agents using the Microsoft Agent Framework, covering basic agent setup, tool integration for external data, and managing conversation context and personalized interactions. It highlights the framework's simplified development, built-in telemetry, and modular design for creating robust AI agents.
databricks/databricks-sql-python
Databricks SQL Connector for Python
★ 231 · Python
What is customer segmentation?
Customer segmentation combines multiple types and methods, from rule-based to AI/ML-driven models, but its success hinges on unifying fragmented customer data into a governed Customer 360. Databricks' CustomerLake, an Agentic CDP, builds segments directly on governed data with AI-driven identity resolution and natural-language audience creation, eliminating data copies and extra vendors.
Unlocking semantics for AI: How Mercedes-Benz Korea built trusted “Talk to Data” at scale
Mercedes-Benz Korea built a trusted "Talk to Data" solution at scale by making 500+ KPI definitions available in an AI-ready semantic layer on Unity Catalog metric views, accelerating the transition with an automated DAX-to-Metric-View transpiler. This governed semantic layer supports both existing BI and new "Talk to Data" experiences, with Genie and Agent Bricks providing consistent answers and shaping a playbook for persona-based AI agents across markets.
Forward Deployed Engineering: Delivering Business Outcomes with AI
Databricks is launching its Forward Deployed Engineering (FDE) organization to accelerate customer business outcomes with AI, pairing the Lakehouse platform with embedded, engineering-led delivery. This new approach moves beyond migration and pipeline building to solve business problems with production AI agents, as demonstrated by customers like Fox, JPMC, and Qualcomm.
Ingesting the Milky Way: Petabyte-Scale with Zerobus Ingest
Zerobus Ingest, a new serverless streaming API, enables instant deployment of petabyte-scale data pipelines on Databricks without manual infrastructure management. Its dynamic partitioning architecture automatically scales compute and sustains over 12 GB/s throughput to a single table, efficiently handling unpredictable data volumes.
How ERGO Hestia reduced time-to-market with Lakebase and Mosaic AI Model Serving
ERGO Hestia modernized its real-time pricing engine with Databricks Lakebase and Mosaic AI Model Serving, reducing time-to-market by unifying data, features, and decisions for millisecond pricing. This eliminated extraction overhead and fragmented governance from their previous multi-hop architecture, enabling faster model deployment and instant market response.
Stop Leaving Your Azure Storage Open to the Public!
The video demonstrates how to enable firewall support for an Azure Databricks workspace storage account, preventing public network access. It walks through creating private endpoints, an access connector, and using a PowerShell command to configure the firewall.
databricks/databricks-sdk-py — v0.117.0
The `resource_id` field in `bundledeployments.Operation` is no longer required, which is a breaking change. Several bug fixes improve performance and stability, including caching OIDC tokens, making `WorkspaceClient.dbutils` lazy, and better handling of Spark Connect runtimes.
Welcoming the first cohort of Databricks student fellows
Databricks launched its inaugural Student Fellows cohort, selecting a diverse group of students to bridge academic theory and real-world data and AI practice. These fellows will host workshops, hackathons, and mentorship programs at their universities, with five standout individuals from top schools already making significant contributions.
Deploying Azure Databricks with Terraform? Watch this first!
This video demonstrates how to deploy an Azure Databricks workspace using Terraform by cloning a provided script, configuring variables, and executing Terraform commands. It walks through setting up prerequisites, authenticating Azure CLI, and populating a Terraform variables file to successfully provision the workspace.
Geospatial Unbounded: Spatial SQL GA with AI/BI Maps, Delta Sharing, and Iceberg v3
Spatial SQL is now Generally Available on Databricks, bringing native geospatial data types, 90+ ST_* functions, and AI/BI Dashboards that render maps natively. This release also includes major performance improvements, open lakehouse support via Delta Sharing and Iceberg v3, and Apache Spark 4.2 compatibility for geo columns.
Azure Databricks at Data + AI Summit 2026 featuring Industry Leaders and Partners
Azure Databricks at Data + AI Summit 2026 featured new joint product announcements and integrations, alongside key sessions on zero-copy federated analytics and ecosystem co-engineering. Learn how joint customers are modernizing data estates, scaling AI, and unlocking business value with Azure Databricks.
Empower your healthcare agents with ready-to-use MCP on Databricks Marketplace
Databricks Marketplace now offers ready-to-use biomedical and clinical Model Context Protocol (MCP) servers from partners like Climb and Atropos Health, empowering healthcare agents. Easily build and deploy bespoke agents to production, leveraging a securely governed, centralized MCP Catalog that also supports your own custom MCP servers or data.
How Ecolab rebuilt retail intelligence on Databricks and Anthropic Claude
Ecolab rebuilt retail intelligence on Databricks and Anthropic Claude, converting 700-page FDA manuals into real-time answers for frontline staff using Foundation Model APIs and cutting compliance report compilation from two weeks to under two minutes. The solution, a native Databricks App with Lakebase Postgres and Unity Catalog, unifies nine siloed data sources and employs a multi-agent orchestration framework with Judge LLMs and MLflow tracing for personalized, continuously refined intelligence.
databricks/databricks-sdk-go — v0.144.0
The Databricks SDK for Go now includes a TypeOverrides field for database.SyncedTableSpec. This field is also added to postgres.SyncedTableSyncedTableSpec.
Stop building data products. Start building data services.
The one-product-per-use-case model breaks down under acquisition-driven growth and agentic consumption; a services layer is more adaptable to what comes next. Moving data mastering and quality checks closer to ingestion makes integration cycles measured in weeks possible, reducing insight lag.
databricks/cli — v1.3.0
The `direct` deployment engine is now Generally Available and the default for new deployments, with an option to revert to Terraform. New commands `databricks quickstart` and `databricks version --check` are added, alongside fixes for authentication and bundle deployments.
Scaling AI Through Data Fluency
Aer Lingus built a solid data foundation with governance and quality, treating data literacy as a core business skill with a custom curriculum. This enabled real-time insights for optimizing flight loads, pricing, and operations decision-making.
Announcing the Public Preview of Custom URLs
Databricks accounts can now use a single, branded custom URL like mycompany.databricks.com, replacing individual workspace URLs. This simplifies login and navigation across multiple workspaces, enabling account-wide features like Genie and Unity Catalog lineage.
AWS and Databricks at Data + AI Summit 2026: Accelerating real-world AI innovation
AWS and Databricks are accelerating real-world AI innovation, from Mastercard experimentation to production-scale AI, as showcased at Data + AI Summit 2026. Explore breakout sessions, industry forums, and hands-on demos covering agentic AI, governance, open data architectures, and multi-engine interoperability with Amazon Bedrock and Kiro.
AI Serving Platform That Adapts to Your Model
Databricks now offers a fully managed AI serving platform that automatically adapts to your model's resource needs, from scikit-learn to 70B LLMs, without manual configuration. This results in up to 90% lower infrastructure costs and <10ms p99 latency overhead for customers migrating from self-managed stacks.
Python Based Time Series Analytics on Databricks
Databricks partnered with AVL to create Impulse, an open-source Python framework for time series analytics on petabyte-scale automotive sensor data. Impulse standardizes raw sensor data into a silver layer data model, allowing engineers to query vast measurement data efficiently within the Databricks Lakehouse.
databricks/dbt-databricks — 1.12.1 (v1.12.1)
This release exposes Databricks Jobs IDs in dbt run results and adds support for SPOG vanity URLs. It also fixes issues with streaming tables, materialized views, snapshots, and managed Iceberg incremental models, while introducing a breaking change requiring `contract.enforced: true` for column-le…
Announcing the Databricks storage ecosystem: Governing the enterprise data estate, wherever it lives
The Databricks Storage Ecosystem now natively connects hybrid and on-premises storage platforms to Databricks via OpenSharing, enabling centralized data governance and GenAI scaling across your entire hybrid infrastructure. Run Databricks Serverless Compute, Genie, and LLMs directly on your on-premises datasets with a zero-copy architecture, instantly turning isolated data into active, AI-ready assets.
databricks/databricks-sdk-go — v0.143.0
This release adds an `AcceleratedSync` field to `database.SyncedTableSpec` and `postgres.SyncedTableSyncedTableSpec`. These changes enable configuration of accelerated sync for synced tables within Databricks.
Databricks on Databricks: How Marketers Use Data 3x More with Genie, an AI Analytics Assistant
Databricks built "Marge," an AI analytics assistant powered by their Genie platform, to help its marketing team access and utilize data more efficiently. Marge provides conversational analytics by unifying marketing data in a lakehouse and offering governed, trusted insights in seconds, significantly reducing reliance on manual analyst reports.
databricks/databricks-sdk-java — v0.119.0
This release adds new services for AI Search and Bundle Deployments, along with numerous fields across existing services for managing catalogs, connections, schemas, MLflow, pipelines, and vector search. It also includes a breaking change by removing the `bundle` package and its associated service.