Serverless

Can't start Serverless Warehouse after extend budget

Serverless Compute connectivity issues with .com.br domains vs. Classic Clusters Spark hangs

Beta alert: Materialized Views and Streaming Tables in Serverless Notebooks

Hi folks, Wanted to share a new feature that's in [beta](https://docs.databricks.com/aws/en/ldp/dbsql/compute#serverless-general-compute) \- creating and refreshing materialized views and streaming tables from serverless compute! Users can create MVs natively in SQL or using `spark.sql("CREATE MATERIALIZED VIEW test_mv AS SELECT * from samples.wanderbricks.booking_updates")` in their notebooks and jobs attached to serverless compute. Workspace admins can enable the beta feature, "MV and ST in Serverless Notebooks and Jobs" in their preview settings. It’s currently available in [select regions](https://docs.databricks.com/aws/en/resources/feature-region-support#serverless-aws). Would love to hear y'all's feedback!

Inquiry regarding Serverless outbound IP ranges and Serverless compute firewall configuration

Serverless SQL Warehouses Strategy

Hi, we're a big industrial company and have some pretty diverse use cases in terms of data volume, speed requirements etc. Many of them are quite sporadic (serving data to PowerBI dashboards which are queried a few times per day, but need to be performant then). We are currently thinking on how to provision SQL Serverless Warehouses to our users. How do you do this in your companies: \- Do you have one (or a few) larger warehouses that serve all different use cases? Or \- Do you create / have users create their own warehouses per use case? \- Or do you use a/multiple shared classic warehouses running 24/7? Cost allocation wise the latter one is easier to track, but from a compute cost point of view I imagine the former one is probably more efficient?

How Switching from JDBC/ODBC Clusters to Serverless SQL Warehouses Boosted Our Power BI Performance

Live Cost Estimator

I'm building a **live cost estimator** that doesn't have to wait for the system tables or billing data to update. It gives me immediate cost feedback every second and I'm sharing the development journey on YouTube. I already have live costs estimates for **all-purpose clusters, SQL warehouses and interactive serverless compute.** I would love some feedback, suggestions and if you want to try it out or contribute let me know!

Serverless compute throws OUT_OF_MEMORY exception

Any tips for DABs in CI/CD? Seems pretty useless so far.

We've used DAB-commands like Validate and Plan for a while - to print Github PR-comments on what the PR will change, delete and create. But we are struggling to catch breaking changes before they are committed to main. Some examples: After migrating a pipeline to serverless compute, our branch passed the plan in the PR stage, but failed in main due to `You must use the Advanced edition when using serverless compute. (400 INVALID_PARAMETER_VALUE)` which is something I would expect CI to catch. Another example is Lakeflow generating a new pipeline-ID, which means during deploy it will try to apply itself to an existing pipeline and fail on mismatching pipeline-ids. Again, would've loved to fail in CI instead of main. How are you solving this?

DABs Python Mutators: Stop Copy-Pasting the Same Config Across 50 Jobs

# Situation You've got 30, 50, maybe 100 jobs in your Declarative Automation Bundle. Every single one needs failure notifications. Every single one needs cost-center tags. Every single one needs the right cluster policy. And every time someone adds a new job, they forget at least one of those things. You could write **one Python function** that enforces it automatically at deploy time. That's what DABs Python mutators do. # What Are Mutators? A mutator is a Python function that runs during `databricks bundle deploy`. It receives every job (or pipeline) in your bundle, whether defined in YAML or Python, and returns a modified copy. Think of it as middleware for your deployment config. Write a tag, permission, or compute standard once, and apply it automatically to every resource at deploy time. No drift. Decorate a function with  \`@job\_mutator\`, \`@pipeline\_mutator\`,  \`@schema\_mutator\`, or \`@volume\_mutator\`. The function receives the resource + bundle context, and returns a transformed copy. You register them in `databricks.yml`: python: mutators: - 'mutators:add_pipeline_mutators' # Example This example defines common pipeline standards for every pipeline in your bundle: * Specifies common tags. * Enforces serverless compute. * Defines default notifications group and when to trigger an alert. ​ from databricks.bundles.core import Bundle, pipeline_mutator @pipeline_mutator def add_pipeline_mutators(bundle: Bundle, p: Pipeline) -> Pipeline:     p = replace(p, tags=_add_common_tags(bundle, p.tags))     p = replace(p, serverless=True)     default = Notifications.from_dict(        { "email_recipients": "${var.recipients}",          "alerts": ["on-update-failure", "on-update-fatal-failure", "on-flow-failure"] }     )     p = replace(p, notifications=[default])     return p Other resources: * The [bundle-examples](https://github.com/databricks/bundle-examples) repo has a working example at [knowledge\_base/job\_programmatic\_generation](https://github.com/databricks/bundle-examples/tree/main/knowledge_base/job_programmatic_generation). * And as well the documentation page: [https://docs.databricks.com/aws/en/dev-tools/bundles/python/#modify-resources-defined-in-yaml-or-python](https://docs.databricks.com/aws/en/dev-tools/bundles/python/#modify-resources-defined-in-yaml-or-python) # Use Cases https://preview.redd.it/2v2ikiexd4yg1.png?width=632&format=png&auto=webp&s=fa39246e3830b857f1da43777aacfb1079a261a8 Job Mutator Examples: * Enforce default email notifications, owners, tags. * Standardize job clusters / serverless environments. * Inject common job parameters or health/queue settings. Pipeline Mutator Examples: * Enforce pipeline cluster / environment settings. * Apply consistent configuration, catalog/schema, or triggers across all pipelines. Schema Mutator Examples: * Apply standard permissions or tags to all schemas. * Enforce naming conventions or lifecycle settings. Volume Mutators: * Set default storage locations, ACLs, or lifecycle flags. * Add org‑wide tags or conventions to all volumes.

VNet Data Gateway unable to connect to Azure Databricks Serverless SQL via Private Endpoint

Heading into the May 2026 Databricks Data Engineer Associate Exam? Read this first.

So if you've been scrolling through older study guides for the Databricks Data Engineer Associate exam — be careful. The syllabus got a pretty big update this month, and the focus has shifted toward the platform's newer declarative features. I spent some time going through the new guidelines. Here's what I found. Lakeflow is the new standard. The exam has moved away from manual ETL logic. You need to understand Lakeflow Spark Declarative Pipelines (formerly DLT) and how Streaming Tables and Materialized Views actually differ. If your notes still say "DLT" everywhere, time to update them. DABs are no longer a side topic. Databricks Asset Bundles — basically infrastructure-as-code for workflows — is now a core part of the exam. They want to see that you can deploy through DABs, not just click around the UI. Unity Catalog is the default assumption. No more legacy Hive Metastore questions. The exam lives in a UC-enabled world now. Three-tier namespace (catalog.schema.table), Volumes for unstructured data, column-level lineage — that's where your time should go. Serverless Compute is showing up more. When do you pick Serverless SQL Warehouses or Serverless Jobs over classic clusters? That tradeoff — less config overhead vs. less control — is fair game now. The weightings that surprised me → 31% on Processing (Lakeflow, Spark, Streaming Tables) → 18% on Productionizing (DABs, Workflows, deployment) That's almost half the exam right there. Honestly, if you just understand why Databricks is pushing toward declarative tools — letting the platform handle the boring parts so you can focus on the actual logic — a lot of the questions start to make sense. For practice material, BricksNotes has an updated practice test that follows the May 2026 format — 45 questions, 90 minutes, same weightings. → [bricksnotes.com/blog/databricks-data-engineer-associate-new-exam-guide-may-2026](http://bricksnotes.com/blog/databricks-data-engineer-associate-new-exam-guide-may-2026) Good luck to everyone testing this month! Drop questions below if you're stuck on any of the new topics — happy to help where I can.