Databricks CLI
Recent items mentioning Databricks CLI across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
The Databricks CLI recently received a fix for a broken --profile fallback in v0.107.0 of the Databricks SDK for Java 4. Community discussions also highlighted a mitigation for "error downloading Terraform" during bundle deployments 3 and a new VS Code extension for inspecting Databricks Asset Bundles locally 9.
Generated daily from the 10 most recent items mentioning Databricks CLI. Click any [N] to jump to the source.
Databricks Connect v2.10.7 wants admin permissions on local machine
Anyone experienced this already? When I (auto)updated databricks connect plugin in VSCode today I needed to create a new Auth profile. I was taken to a new login screen where I needed to give Databricks admin permissions (which I can't give on company resources). Anyone experienced this / has way around it? Somehow it mostly seems to affect the Databricks connect plugin as my CLI seems to work,but this doesn't bode well for the (near) future. (edited company info out) https://preview.redd.it/m27ignwc232h1.png?width=434&format=png&auto=webp&s=f05168a6c95cdba6b753628ad15256245d129ddc # packages/databricks-vscode # (2026-05-07) * Add remote mode for initial Remote Development compatibility (#1861) ([9e768db](https://github.com/databricks/databricks-vscode/commit/9e768db)) * Rename "Databricks Asset Bundles" → "Declarative Automation Bundles" (#1864) ([62a94e1](https://github.com/databricks/databricks-vscode/commit/62a94e1)) * Preserve profile name in Databricks CLI auth provider (#1877) ([3f54441](https://github.com/databricks/databricks-vscode/commit/3f54441)) * Fix new profile sign in using already existing host under different profile (#1893) ([c4c25fb](https://github.com/databricks/databricks-vscode/commit/c4c25fb)) * Include profiles with `account_id` in `listProfiles` results (#1894) ([d6e2e5d](https://github.com/databricks/databricks-vscode/commit/d6e2e5d)) * Update minimal python and dbconnect versions for serverless (#1884) ([5a1a1d5](https://github.com/databricks/databricks-vscode/commit/5a1a1d5)) * Update Databricks CLI to v0.297.2 (#1882) ([ea77424](https://github.com/databricks/databricks-vscode/commit/ea77424)) — see the [CLI release notes](https://github.com/databricks/cli/releases) for changes since v0.286.0
Lakeflow Connect | HubSpot (GA)
Hi all, Lakeflow Connect's HubSpot connector is now GA! It provides a managed, secure, and native ingestion solution for the HubSpot Marketing Hub — ingesting marketing campaigns, and email analytics into Databricks. Try it now: 1. [**Set up HubSpot as a data source**](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/hubspot-source-setup) 2. [**Create a HubSpot Connection in Catalog Explorer**](https://docs.databricks.com/aws/en/connect/managed-ingestion#hubspot) 3. [**Create the ingestion pipeline via the UI, a Databricks notebook, or the Databricks CLI**](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/hubspot-pipeline)
Mitigation for "error downloading Terraform" during bundle deployments
If your CI/CD pipelines suddenly started failing out of nowhere with this error: "𝐞𝐫𝐫𝐨𝐫 𝐝𝐨𝐰𝐧𝐥𝐨𝐚𝐝𝐢𝐧𝐠 𝐓𝐞𝐫𝐫𝐚𝐟𝐨𝐫𝐦: 𝐮𝐧𝐚𝐛𝐥𝐞 𝐭𝐨 𝐯𝐞𝐫𝐢𝐟𝐲 𝐜𝐡𝐞𝐜𝐤𝐬𝐮𝐦𝐬 𝐬𝐢𝐠𝐧𝐚𝐭𝐮𝐫𝐞: 𝐨𝐩𝐞𝐧𝐩𝐠𝐩: 𝐤𝐞𝐲 𝐞𝐱𝐩𝐢𝐫𝐞𝐝" and you’re using Databricks CLI - you’re probably hitting the same issue I did. The Databricks CLI needs to be upgraded. Databricks released patched CLI versions after an expired verification key started breaking bundle deploy commands in CI/CD environments. Hopefully this saves someone else some troubleshooting time. https://preview.redd.it/7ree9cuvo21h1.png?width=861&format=png&auto=webp&s=47c46407d20e4b552d1b5e79258e761cba8f48cb [Mitigation for "error downloading Terraform" during bundle deployments · Issue #5022 · databricks/cli](https://github.com/databricks/cli/issues/5022)
This release fixes an issue where the Databricks CLI `--profile` fallback was broken. It also introduces new API methods for `workspaceClient.supervisorAgents()` and `workspaceClient.vectorSearchEndpoints()`, along with several breaking changes related to `Example` and `Tool` fields, and `minQps` in Vector Search endpoints.
Lakeflow Connect | Smartsheet (Beta)
Hi all, Lakeflow Connect's Smartsheet connector is now in beta! It provides a managed, secure, and native ingestion solution for Smartsheet sheets and reports into Databricks. Try it now: 1. **Enable the Smartsheet Beta:** Workspace admins can enable the Beta via Settings → Previews → "LakeFlow Connect for Smartsheet" 2. [**Set up Smartsheet as a data source**](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/smartsheet-source-setup) 3. [**Create a Smartsheet Connection in Catalog Explorer**](https://docs.databricks.com/aws/en/connect/managed-ingestion#smartsheet) 4. [**Create the ingestion pipeline via the UI, a Databricks notebook or the Databricks CLI**](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/smartsheet-pipeline)
Release: v2.10.7 (#1895)
This release adds initial compatibility for Remote Development and renames "Databricks Asset Bundles" to "Declarative Automation Bundles." It also includes fixes for profile management, such as preserving profile names and correctly signing in with existing hosts under different profiles.
Databricks Data Engineer Associate Exam Updated for 2026
The Databricks Data Engineer Associate exam changed on May 4, 2026. The exam now has 7 domains instead of 5. Two new domains were added. The first new domain is CI/CD. This includes: • Databricks Repos • Git integration • Branching and commits • Deploying Declarative Automation Bundles • Using the Databricks CLI • Moving code from dev to test to production Databricks Asset Bundles is now called Declarative Automation Bundles, so learn the new name. If you have never used Git or the Databricks CLI inside Databricks, spend some time practicing in the Free Edition. Connect a Git repo, make commits, and deploy bundles. Hands-on practice will help a lot. The second new domain is Troubleshooting, Monitoring, and Optimization. This includes: • Reading the Spark UI • Finding bottlenecks like data skew and excessive shuffling • Understanding Liquid Clustering • Predictive optimization • Troubleshooting cluster and memory issues Many courses do not teach Spark UI deeply, so try running queries yourself and checking the Spark UI. Compare good queries with inefficient ones to understand the difference. Some existing domains also changed. Ingestion now includes Lakeflow Connect along with Auto Loader and COPY INTO. Governance now includes: • Column-level masking • Row-level security • Attribute-based access control You now need to understand security beyond basic GRANT permissions. Lakeflow Jobs also tests three trigger types: • Scheduled • File arrival • Table update Know when to use each one. Some product names also changed: • Databricks Asset Bundles → Declarative Automation Bundles • Delta Live Tables → Lakeflow Declarative Pipelines The exam uses the new terminology, so update your study material if you are using older resources. The exam format is still: • 45 scored questions • 90 minutes • $200 There may also be extra unscored questions mixed into the exam. For preparation, the original Academy courses still help for the old domains. But for the two new domains, hands-on practice is very important. Practice: • Spark UI • Git integration • Databricks CLI • Deployments using bundles Also read the latest official exam guide PDF from the Databricks page. Good luck to everyone preparing for the exam.
Lakeflow Connect | Outlook (Beta)
Hi all, Lakeflow Connect's Outlook connector is now in beta! The Lakeflow Connect Outlook connector provides a managed, secure, and native ingestion solution for Microsoft Outlook email data — ingesting messages and attachments into Databricks. Try it now: 1. [**Set up Outlook as a data source**](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/outlook-source-setup) 2. [**Create an Outlook Connection in Catalog Explorer**](https://docs.databricks.com/aws/en/connect/managed-ingestion#outlook) 3. [**Create the ingestion pipeline via the UI, a Databricks notebook, or the Databricks CLI**](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/outlook-pipeline)
Lakeflow Connect | GitHub (Beta)
Hi all, Lakeflow Connect's GitHub connector is now in beta! The Lakeflow Connect GitHub connector provides a managed, secure, and native ingestion solution for GitHub organizational metadata and activity data — ingesting commits, pull requests, team members, and more into Delta tables. Try it now: 1. [**Set up GitHub as a data source**](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/github-source-setup) 2. [**Create a GitHub Connection in Catalog Explorer**](https://docs.databricks.com/aws/en/connect/managed-ingestion#github) 3. [**Create the ingestion pipeline via the UI, a Databricks notebook, or the Databricks CLI**](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/github-pipeline)
I built a VS Code extension for inspecting Databricks Asset Bundles locally
I kept catching issues too late, broken dependencies, misconfigured parameters, stale parameters in notebooks, only after running `databricks bundle validate`. So I built something to make it easier to review locally before deployment. It uses the validation output from the Databricks CLI to help you inspect bundle resources, jobs, tasks, dependencies, parameters, and validation output, directly in VS Code. It is still early, but I would love to know: **what additional features would you expect from a tool like this, or what do you think is missing?** At the moment, it works best for jobs, but I will be rolling out to pipelines soon. GitHub repo: [https://github.com/uncoverthestack/databricks-bundle-inspector](https://github.com/uncoverthestack/databricks-bundle-inspector) VS Code Marketplace: [https://marketplace.visualstudio.com/items?itemName=UncoverTheStack.databricks-bundle-inspector](https://marketplace.visualstudio.com/items?itemName=UncoverTheStack.databricks-bundle-inspector) There is also a demo of how it works in the README as well.
I can't seem to download larger files from Databricks
100% Databricks newbie here, but pretty seasoned nerd. I've been tasked with downloading a rather large dataset from Databricks. It's 15 files of various sizes, but the larger ones (300GB, 1.2TB and 2.7TB respectively) are giving me trouble. I started with the [Databricks CLI](https://github.com/databricks/cli), which worked fine but the download died after an hour or so, very consistently. I then noted that the first line of the README says "This project is in Public Preview." Great. I then moved to Firefox under Linux, where I was able to start the downloads. They seem to die after exactly 16.1GB, after which I am able to resume them, and they start where they left off. Yay, I only have to click resume 2.7TB/16.1GB=167 times to get my file. Trouble is, after a while my session expires, and I can no longer resume the downloads. I'm also getting pretty shit speeds (100Mbit/s) or so combined, on a 1Gbit business fiber connection, but if I could at least get something stable, I'd be happy. It should probably be mentioned that I'm on the freebie tier of databricks. Edit: People have asked for background as to why I'm doing this, which is a 100% legitimate question. A company in our line of work has released this very large dataset into the public domain. They picked Databricks, I didn't. We wish to download this dataset to our on-prem systems so we can process it using our fairly niche and highly resource intensive algorithms. It's not really an option to run things on Databricks, for a number of good reasons.
I built a reusable DABs template for multi-environment bundle projects (open source)
I've been working with Databricks Asset Bundles (recently renamed to Declarative Automation Bundles, same DABs acronym) on my project for over a year now. At some point I realized the setup I'd landed on was general enough to be reusable, so I spent about three months of evenings and weekends turning it into a proper Databricks CLI template. It ended up being more comprehensive than what I run on my own project, honestly. You run `databricks bundle init <repo-url>`, answer some prompts (cloud provider, compute type, CI/CD platform, environment setup), and it generates a complete bundle project with: - Multi-environment targets (user/stage/prod, optional dev) - Schema-per-user dev isolation (dbt-style approach: everyone shares the dev catalog, schemas prefixed with username) - CI/CD pipelines for GitHub Actions, Azure DevOps, or GitLab - Medallion architecture schemas as bundle resources - Configurable compute (classic, serverless, or both) - Optional RBAC with environment-aware groups It uses the new direct deployment engine (requires CLI v0.296.0+), so no Terraform dependency. The generated project comes with docs, a quickstart guide, and sample pipelines to start from. Repo: https://github.com/vmariiechko/databricks-bundle-template Example output: https://github.com/vmariiechko/databricks-bundle-template-example MIT licensed. Happy to hear feedback or answer questions about the design decisions. And if something doesn't fit your setup, issues and PRs are welcome.
Unified host detection is now automatic, removing the `Experimental_IsUnifiedHost` field and enabling a single configuration profile for both account and workspace operations. The file-based OAuth token cache has been removed, defaulting to an in-memory cache, and new API methods were added for Temporary Volume Credentials and Knowledge Assistants.
This release adds Azure MSI authentication and improves `.databrickscfg` profile resolution. It also fixes issues with non-JSON error responses and Databricks CLI token scope mismatches.
This release fixes a bug in Databricks CLI bundle commands. Users previously encountered an "unable to verify checksums signature: openpgp: key expired" error, which is now resolved.
Databricks CLI authentication now correctly errors on token scope mismatches, prompting re-authentication instead of silently using incorrect permissions. New `dataclassification` and `knowledgeassistants` services and corresponding workspace-level APIs have been added.
Release: v2.10.5 (#1834)
- Update Databricks CLI to v0.286.0
NewsDatabricks Breaking News: Week 2026 02: 5 January 2026 to 11 January 2026 #databricks news
Databricks now allows changing catalog and schema during dashboard deployments, addressing a previous issue with environment-specific configurations. The Databricks CLI has a breaking change with plan version 2, altering the structure of deployment plans.
Release: v2.10.4 (#1821)
- Update Databricks CLI to v0.280.0
Release: v2.10.3 (#1772)
This release updates the Databricks CLI to v0.266.0, which includes breaking changes. Practitioners should review the CLI release notes for details on these changes.
Tutorials51 Setup Azure DevOps Pipeline with Databricks Asset Bundles (DABs) | Complete CICD Process
The video demonstrates how to set up an Azure DevOps pipeline to deploy Databricks Asset Bundles (DABs) to higher environments like QA. It covers configuring service principal permissions, setting up Azure pipeline variables for environment-specific details, and writing the YAML pipeline code to validate and deploy Databricks assets.
Tutorials50 Databricks Asset Bundles | Configure Production grade DABs | CICD using DABs (IAC)
The video demonstrates how to configure and deploy Databricks Asset Bundles (DABs) for managing Databricks assets like notebooks, jobs, and pipelines across different environments. It covers creating a structured DAB project, defining resources and targets in YAML, and deploying using both the Databricks UI and CLI, including setting up environment-specific configurations and variables.
Tutorials49 Databricks CLI | Install and Authenticate Databricks CLI | U2M and M2M Authentication
The video demonstrates how to install the Databricks CLI on Windows and authenticate it using both User-to-Machine (U2M) and Machine-to-Machine (M2M) methods. It then shows how to run various CLI commands to interact with Databricks workspaces and account consoles, such as listing catalogs, creating schemas, and managing groups.
Release: v2.10.2 (#1738)
- Update Databricks CLI to v0.259.0
Release: v2.10.1 (#1704)
This release updates the Databricks CLI to v0.253.0 and improves virtual environment management by using UV when available. It also adds support for complex variables in the UI and prevents `sys.exit` calls in Jupyter initialization scripts.
Release: v2.9.4 (#1670)
- Rollback Databricks CLI to v0.245.0 to fix auth problems
Release: v2.9.3 (#1665)
- Update Databricks CLI to v0.248.0