Service Principal

Azure Databricks, two-week-old account: partner pay-per-token endpoints (Claude, Grok) have never once succeeded — "Databricks-set rate limit of 0" — and the Assistant has "no daily token allowance". Open models work. What gates this?

Azure Databricks, Premium, one account with three workspaces, Unity Catalog, Azure Marketplace billing (no card, no trial credits left). The account was bootstrapped on a trial workspace about two weeks ago and moved to Premium six days later. I've spent two days on this and want a sanity check from anyone who has seen it. The facts, from system.serving.endpoint_usage: • Every open-weight pay-per-token endpoint has worked since the trial: gpt-oss-120b has ~50 successful calls going back to the trial period, Llama 3.3 70B likewise, zero refusals. • No partner pay-per-token endpoint has ever returned a success on this account. The very first call anyone made to databricks-claude-sonnet-5 was refused, and every call since, on every Claude endpoint and on Grok, in all three workspaces, for users and service principals alike: 403 PERMISSION_DENIED: The endpoint is temporarily disabled due to a Databricks-set rate limit of 0. • It is not our AI Gateway config: I removed every rate limit from one Claude endpoint and invoked it — same 403. The endpoints' config shows nothing else. • The notebook Assistant worked during the trial. On the paid account it refuses everyone, admins included: *"This workspace has no daily token allowance for the assistant."* Underneath, /ajax-api/2.0/conversation/llmproxy/ returns 429 {"type":"daily_token_limit_reached","daily_limit_tokens":0,"message":"…Daily token limit of 0 reached. Resets at midnight UTC. (DTB)"} and it recomputes to 0 every midnight. • Once, the Databricks Apps in the workspace were stopped by the platform with *"App compute was stopped due to workspace or account status"* while every workspace showed RUNNING. apps start brought them back. Red herring, for completeness: we also created a Unity AI Gateway per-user budget whose first version had a $0 threshold with BLOCK_USAGE. That blocked Genie for a day, exactly as documented, and was fixed. It was created a day *after* the first Claude refusal, so it isn't the cause of any of the above. What I've found: two Community threads describe "Databricks-set rate limit of 0" as a workspace trust-tier gate — trial-born accounts sit in TRIAL_VERIFIED, pay-per-token partner models are gated to PAYABLE_VERIFIED, a card alone doesn't flip it, only Databricks Sales/Support moves the tier. Both were AWS/personal accounts, neither resolved on-page. The account console's own API reports our account's feature_tier as STANDARD_W_SEC_TIER although the workspaces are Premium; I can't find what that field means. Questions • Does Azure Databricks with Marketplace billing go through the same PAYABLE_VERIFIED gate? Does it clear on its own after the first settled invoice, or does someone have to move it? • If you were moved: who did you contact (account team, or "Contact us" in the console) and how long did it take? • Is the Assistant's "daily token allowance" the same gate, or a trial allowance that simply goes to 0 when the trial ends on an unverified account? • Does feature_tier: STANDARD_W_SEC_TIER on the account object mean anything to anyone? submitted by /u/sumit671 [link] [comments]

Avoid Unity Catalog external location permission bugs by using managed storage

Hey, I work as a Databricks Engineer at Abilytics, so here's my take. If you are migrating existing tables to Unity Catalog, defining external locations incorrectly will break your grants. When you register an external location via CREATE EXTERNAL LOCATION, Unity Catalog validates your storage credential against the underlying cloud storage container. However, users often forget that read and write access on the external location is distinct from the IAM role or service principal trust policy. If a user has SELECT on a table, but lacks BROWSE on the external location, queries against parquet files directly via path-based access will fail with permission denied, even if the table-level grant succeeded. Always verify your storage credentials have explicit container-level policies attached before mapping external volumes. Furthermore, remember that path-based access like spark.read.load('s3://my-bucket/path') requires explicit external location grants, whereas managed tables abstract this entirely. Prefer managed tables default storage roots to bypass manual external location ACL synchronization issues entirely. Happy to go further — more on Databricks | Platform Engineering | AI at Abilytics , AI Systems, Data Engineering & Platform Engineering Services. submitted by /u/AbilyticsEng [link] [comments]

Databricks now deletes unused service principal secrets after 90 days

Lakebase Data API (GCP) returns jwk not found for valid service principal tokens

Issues with oidc/v1/token endpoint (REST for zerobus-ingest)

Anyone else work with the zerobus-ingest? There is an auth-flow to get an "oauth token" from a service principal with an "oauth secret". In that flow, you are supposed to submit the authorization details. See docs: https://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-kafka I had been requesting access (via authorization_details) to a table as shown in the example: "type": "unity_catalog_privileges", "privileges": ["SELECT"], "object_type": "TABLE", "object_full_path": "main.default.air_quality" ... but it wouldn't work. Was banging my head on it all day. The privileges granted to the service principal for this table were unrestricted " ALL PRIVILEGES ". It turns out that, in addition to "ALL PRIVILEGES", this "token" endpoint wanted me to grant "SELECT " to the service principal in UC governance as well. Else the authorization_details are not valid. This "token" api has a lot of sharp edges. Any chance there is a nuget client to wrap around it and protect us from these unfortunate nuances? Is it documented that ALL PRIVILEGES doesn't actually give us all privileges, if we request "SELECT" by name? submitted by /u/SmallAd3697 [link] [comments]

Ontology ranked the snippet. I wrote the gate I wanted before trusting a Genie space.

A commercial user asked what yield was last season. Two definitions live in the same lakehouse. One is plot-level, moisture-adjusted, borders out. The other is as-harvested, and a dashboard still uses it. Genie returned the second number. Not a hallucination. OntoRank did what it does: creator, usage, link to a certified asset, freshness. The loud dashboard won. Nobody had certified the measure the board pack is allowed to quote. That is the failure I care about. A green demo is not a release gate. What I actually require before that space faces a user who will paste the sentence into Slack: Golden questions written before the space exists. If you write them after you have tuned instructions, you are scoring an overfit prompt. Benchmarks are not instructions. Genie does not learn from the benchmark SQL. The number lives in a Metric View. A column comment and a knowledge-store formula are hints. Ontology will rank all three. Only the Metric View is a contract every surface compiles. Same KPI from MEASURE(), from the certified dashboard, and from Genie. If they disagree, the space stays off Slack. Permissions checked as the asker. Inside a Genie Agent, compute is the author’s warehouse and data access is the end user. That split does not automatically hold in Teams, Copilot on maker credentials, or a Databricks App on its default service principal. The row filter still runs. It runs for whoever showed up on the SQL. A written kill line. I use Databricks’ “above 80% before UAT” as the hold line, not the ship line. Ship is 90% and zero fails on a regulated definition. Any miss on that definition takes the space down. A person does that. Not a service principal. Genie Code can draft the Metric View and the MERGE. ZeroOps can propose a fix and wait for Approve. Neither owns the definition. If the job stays green for eleven days because a key was slightly wrong, that is still your incident. I wrote the four pieces up so I would stop re-explaining this in reviews. Argue the 80 vs 90 line if you want. I will not move a regulated KPI to “looks fine in the demo.” submitted by /u/Worth_Phase1358 [link] [comments]

Databricks pipeline started failing after moving tables to Unity Catalog — how would you debug it?

Suppose a pipeline was previously using: hive_metastore.schema.table and is migrated to: catalog.schema.table The transformation logic hasn't changed. But the pipeline now fails during execution. Possible areas to investigate: Catalog permissions Schema permissions Service principal Pipeline identity Cluster access mode External location Storage credential Table ownership Hardcoded table references The interesting part is that the SQL may be perfectly valid. How would you troubleshoot this systematically? submitted by /u/hanshu6576 [link] [comments]

Databricks pipeline works in one workspace but fails in another — where would you start?

Imagine the same pipeline and code are deployed to two Databricks workspaces. Workspace A: catalog.schema.table → Query works → Pipeline works Workspace B: catalog.schema.table → Catalog not accessible → Pipeline fails The catalog exists, the table exists, and the SQL itself is valid. Would you first investigate: Workspace-to-catalog assignment Unity Catalog permissions USE CATALOG USE SCHEMA SELECT privileges Service principal permissions External locations Storage credentials Cluster access mode SQL Warehouse identity I'm curious about the actual troubleshooting order people follow for these kinds of Unity Catalog issues. What would you check first? submitted by /u/hanshu6576 [link] [comments]

"Do we need a metastore per workspace?" - the answer, and why the question keeps coming up

Every new joiner on our platform team has asked some version of this, so here is the short version. You do not. The Unity Catalog metastore is created in the account console and you have one per region, linked to as many workspaces in that region as you like. Dev, test and prod attach to the same one and share a single catalog.schema.table namespace. The mental model of "each workspace has its own metastore" comes from the legacy Hive metastore, and that is exactly what Unity Catalog replaces. What actually belongs where: - Account: billing, account admins, users, groups, service principals, and the metastore itself. - Workspace: notebooks, clusters, jobs, dashboards, the place people work. - Metastore: catalogs, schemas, tables, views, volumes, and the permissions on them. Identity federation means a group is configured once at account level and assigned to workspaces, so a grant on data applies wherever that group works. Workspace-local groups still exist in non-federated workspaces, but they cannot be granted Unity Catalog permissions, which is a good reason to migrate them. Storage: metastore-level managed storage is optional, and anything already in a bucket becomes an external location with a storage credential. Drew it out as a 2-3 minute animated diagram: https://youtu.be/2yy9-BL_RDo Series in order: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4 How are you separating environments inside one regional metastore: catalog per environment, workspace-catalog bindings, or separate regions? submitted by /u/Broad-Cherry-5999 [link] [comments]

"Do we need a metastore per workspace?" - the answer, and why the question keeps coming up

This comes up constantly in real platform design, and the confusion is almost always about which layer owns what. The account is the container for the organisation, normally one per cloud provider. Billing, account admins and identities live there. A workspace is a single deployment of the UI: notebooks, clusters, jobs, dashboards. Most teams run dev, test and prod as separate workspaces. The Unity Catalog metastore is the part people place wrongly. It is created at account level, not inside a workspace, and the docs say you have one metastore per region, linked to any number of workspaces in that region. So dev, test and prod in the same region share one metastore and one catalog.schema.table namespace, instead of each keeping a separate legacy Hive metastore. Identities follow the same shape: users, groups and service principals are configured once in the account console and then assigned to workspaces, so you grant a group access to data once rather than repeating it per workspace. Storage is still yours. Metastore-level managed storage is optional, and existing buckets are registered as external locations backed by a storage credential. One line to remember it by: the account holds identities and billing, workspaces hold the tools and compute, the metastore holds the data and who can see it, shared across every workspace in the region. I drew it as a short animated diagram because the hierarchy sticks better visually (2:32): https://youtu.be/2yy9-BL_RDo Episode 3 of a series, in order here: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4 For those running dev and prod in the same region: are you isolating with separate catalogs, workspace bindings, or something else? submitted by /u/Broad-Cherry-5999 [link] [comments]

Can Databricks PAT authentication be used to mint an embed token?

Iam working on a Databricks integration where we support both PAT token and Service Principal authentication. The PAT connection works fine for making Databricks API calls, but when I tried to generate an embed token for dashboard embedding, it doesn’t seem to work. If PAT is not supported, is there any official reason or limitation behind this? Would appreciate any clarification from people who have implemented Databricks dashboard embedding. submitted by /u/Bhanuprakash_1947 [link] [comments]

What credential should I use to authenticate Databricks Zerobus?

I’m learning how to use Databricks Zerobus and I’m confused about authentication. When connecting to Zerobus, should I use a Databricks personal access token, OAuth credentials, a service principal, or another type of credential? I’m testing this for a learning project and want to follow the recommended secure approach. I won’t post any actual credentials. What authentication method are people using, and are there any setup steps or permissions I should know about? submitted by /u/Stu-dent999 [link] [comments]

Managing Service Principal Permissions at Scale: External Locations vs. Managed Volumes

System generated Service Principal -App

Serverless Access Control is here!

We finally have a simple way to control access to serverless in the workspace. Two built-in objects have been introduced: - Default Interactive Compute - Default Automated Compute The first one supports notebooks and databricks connect. Second one jobs and SDP pipelines. To limit who can use serverless: Click Compute in the workspace sidebar. In the Serverless tab, click the kebab menu next to Default Automated Compute , then click Edit permissions . Remove the All Users group, or the group that includes all workspace users. Add only the specific users, groups, or service principals that you want to authorize. https://preview.redd.it/sg59jole9hkh1.jpg?width=957&format=pjpg&auto=webp&s=0c33d19d04c57856c51f57f33c44719317fe8347 submitted by /u/szymon_dybczak [link] [comments]

Declarative Asset Bundle Service Principals best practices

Zerobus ingestion fails in Databricks Free Edition using Service Principal OAuth

Why Databricks Genie returns MessageStatus.FAILED in App API (but works fine in UI) — solved

Hey fellow Databricks builders, Our team at **Enqurious** just wrapped a hackathon building a full medallion architecture (Bronze → Silver → Gold) with Genie AI integration for natural language querying. We hit a wall that cost us 6+ hours — sharing the full fix so you don't have to. # What was the setup? * Gold layer tables in Unity Catalog * Genie Space configured and working in UI * Databricks App deployed successfully # What causes MessageStatus.FAILED in Databricks Genie App API? The root cause is a missing service principal permission chain. The Databricks App runs under a **service principal** — not your user account. Genie working in the UI only proves *your user* has access, not the service principal the App uses at runtime. This is why the same question works in the Genie UI and fails via the App API every time. # What is the complete permission fix? The service principal needs **all five** of these — missing any one causes silent failures: sql -- 1. Catalog level (most commonly missed) GRANT USE CATALOG ON CATALOG your_catalog TO `your-app-service-principal`; -- 2. Schema level GRANT USE SCHEMA ON SCHEMA your_catalog.your_schema TO `your-app-service-principal`; -- 3. Table level — each table individually GRANT SELECT ON TABLE your_catalog.your_schema.table_1 TO `your-app-service-principal`; GRANT SELECT ON TABLE your_catalog.your_schema.table_2 TO `your-app-service-principal`; **Plus in the UI:** * SQL Warehouses → your warehouse → Permissions → Add service principal → **Can Use** * Genie Space → Share → Add service principal → **Can Run** # Where do you see the real error (not just MessageStatus.FAILED)? The actual error is in the **Genie Monitoring tab** inside your Genie Space — not the App logs. App logs only show the generic `MessageStatus.FAILED`. The Monitoring tab shows the specific SQL error or permission denial that caused it. This single tip would have saved us 3 of those 6 hours. # How long do Databricks permissions take to propagate? Unity Catalog permissions take **5–15 minutes** to propagate. Testing immediately after granting permissions is the most common reason the fix appears not to work. Wait 10 minutes, hard-refresh your browser, then retest. # Quick debug checklist If you're still hitting `MessageStatus.FAILED`: □ Check Genie Monitoring tab for the actual SQL error □ Confirm service principal has USE CATALOG (not just SELECT) □ Confirm SELECT on each table individually — schema-level alone is not enough □ Confirm SQL Warehouse is Running, not Stopped □ Wait 10 minutes after any permission change before retesting □ Hard refresh browser before retesting □ Verify app.yaml uses ["streamlit", "run", "app.py"] — not ["python", "app.py"] # FAQ **Q: Why does Genie work in the UI but fail in the Databricks App?** A: The App runs under a service principal, not your user account. The service principal needs its own separate permission chain regardless of your personal access. **Q: Is USE CATALOG required even if SELECT is granted on all tables?** A: Yes — USE CATALOG is mandatory. Without it the service principal cannot traverse the catalog hierarchy, even with table-level SELECT grants. **Q: Where is the real error when Genie fails in an App?** A: Genie Space → Monitoring tab. App logs only show the generic failure; Monitoring shows the specific cause. **Q: How long do Unity Catalog permissions take to propagate?** A: 5–15 minutes. Always wait before retesting. The Enqurious team wrote up a full walkthrough with SQL commands, screenshots, and the complete [app.py](http://app.py) template Happy to share our Genie Space config or [app.py](http://app.py) in the comments — just ask. **Questions for the community:** * Has anyone else faced similar Genie + App permission issues? * Is there a better way to debug these permission chains? * Would love to hear if Databricks is working on si […truncated]

Read-only SQL wrapper for using LLM agents safely on Databricks (open source)

I've been running LLM agents against my Databricks workspace daily across Claude Code, Codex, and Cursor IDE. Hit two issues: 1. The CLI happily runs whatever SQL you pass. Nothing stops a confused agent from a `DROP TABLE`. 2. The default JSON output bloats the agent's context: `[{"col_0":"1"}]` when you just want one number back. So I built a small wrapper around `databricks experimental aitools tools query` and open-sourced it as a drop-in agent skill in my `databricks-bundle-template` repo. **What it does:** - Allow-lists six read-only prefixes: SELECT, WITH, SHOW, DESCRIBE, DESC, EXPLAIN - Block-lists every destructive verb (INSERT, UPDATE, DELETE, MERGE, DROP, TRUNCATE, GRANT, COPY INTO, and the rest) - Strips comments and quoted strings before validation, so `SELECT '/* DROP TABLE foo */ 1'` doesn't slip past - Shapes output to row geometry: 1 cell becomes a scalar, 1 column becomes lines, NxM becomes TSV. The agent reads `1` instead of `[{"col_0":"1"}]`. The wins compound on larger queries. **Install:** ``` databricks bundle init https://github.com/vmariiechko/databricks-bundle-template --template-dir assets/dbx-ro-query ``` Single file Python, no dependencies. Works with Claude Code, Codex, Cursor IDE, and anything that follows the agentskills.io layout (Gemini CLI, OpenCode, Antigravity, etc. pick up the `.agents/skills/<name>/SKILL.md` convention). **Caveats:** - Upstream `aitools tools query` is in the experimental namespace - Verb-based block-list, so `SELECT 1 AS drop` is rejected (rename the alias) - Not a substitute for warehouse RBAC. It's a defense-in-depth layer; still run agents under a service principal with read-only permissions Repo: https://github.com/vmariiechko/databricks-bundle-template/tree/main/assets/dbx-ro-query Curious what others are using for safe LLM agent access to Databricks data.

Databricks Streaming Table + ABAC policies causing ABAC_POLICIES_NOT_SUPPORTED

Guys... has anyone run into this before? We’re facing an issue in Databricks with a streaming pipeline writing data into a Streaming Table that has ABAC policies applied on some columns (column masking policies). When the pipeline tries to write/update the table, it fails with this error: `ABAC_POLICIES_NOT_SUPPORTED: ABAC policies are not supported on tables defined within a pipeline.` Basically, the service principal running the job cannot update the Streaming Table because the table has ABAC policies applied. What’s confusing is that the Databricks docs mention that Streaming Tables support ABAC. We also tried adding the service principal to the policy whitelist/exemptions, but it still fails. Has anyone seen this behavior before? Is this an actual limitation with Lakeflow/Streaming Tables + ABAC, or are we missing some configuration? Thanks!

What's new in AIBI Dashboards - April 2026

* **Publish with service principal credentials**: Authors can publish dashboards using the data credentials of a service principal. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/share/share#publish-dashboard) * **Service principal ownership**: Workspace admins can transfer dashboard ownership to a service principal in the UI. 📖 [Documentation](https://docs.databricks.com/aws/en/ai-bi/admin/#transfer-ownership) * **Choropleth map admin levels**: Choropleth maps support US admin levels 3 (regions, multi-state groupings) and 4 (states). 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/manage/visualizations/maps) * **SQL editor line numbers**: The SQL query editor displays line numbers to help with legibility and debugging. * **PDF subscription page selection**: Dashboard authors can select which pages to include in PDF email subscriptions. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/share/schedule-subscribe) * **Parameter values in widget titles and descriptions**: Dashboard authors can reference parameter values in widget titles and descriptions, so the text updates dynamically as viewers change parameter selections. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/manage/filters/parameters) * **Table cross-filtering and drill-through**: Tables support cross-filtering and drill-through. * **Counter prefix and suffix**: Numbers in counters support custom prefixes and suffixes. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/manage/visualizations/types#counter) * **Schema browser default dataset type:** Adding a table to a dashboard from the schema browser creates a [local metric view](https://docs.databricks.com/aws/en/dashboards/manage/data-modeling/local-metric-views) by default instead of a SQL dataset. * **Warehouse overload message**: Dashboards show a message explaining when rendering is delayed due to the warehouse being overloaded. * **Tabular attachments in email subscriptions**: Dashboard email subscriptions include tabular attachments. * **Fullscreen scroll position**: Exiting fullscreen mode on a published dashboard returns you to your previous scroll position instead of jumping to the top of the page. * **Local metric views**: A new dataset type lets you create metric views directly in a dashboard using a low-code visual interface, without publishing to Unity Catalog first. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/manage/data-modeling/local-metric-views) * **Edit hex color values inline**: Authors can click directly on a hex color value to edit it in place. * **View SQL for visualization widgets**: Authors can view the SQL behind specific visualization widgets while in draft mode. * **Waterfall chart totals**: Waterfall charts with categorical X-axis support a total bar. * **Scatter plot shape field**: Scatter plots support a shape field to differentiate data points by category. * **Clear applied filters individually**: Dashboard viewers can individually clear applied filters from the active selection bar. * **Text box vertical alignment**: Text box widgets support vertical alignment (top, center, and bottom). * **Choropleth map boundaries**: Choropleth maps support additional boundary types, including ZIP code and NUTS regions. * **“Explain this change” chart types**: The “Explain this change” feature is available for pivot table cells, horizontal bar charts, pie charts, and heatmaps, in addition to time series charts. 📖 [Documentation](https://docs.databricks.com/aws/en/dashboards/genie-spaces#explain-chart-changes)

Governance in Databricks Apps

I built an app using Streamlit and it's running on Databricks Apps. The app has modules that query catalogs, sending the user's token to use Databricks governance. However, my managers didn't like that the user could execute queries within Databricks. I could use the service principal in the catalog permissions instead of authorizing the user, but I would have to create an ACL system within the app, which could make it complex. Have you built something similar and could offer some ideas?

Tried the Lovable + Databricks connector on a hackathon project

I originally thought the Lovable/Databricks connector was kind of a gimmick. Then I had a hackathon project where all the heavy lifting was in Databricks (data processing, enrichment, a bit of ML), but the result had to be shown as a simple app for non-technical users. Tried Lovable mostly out of curiosity, and honestly, it worked better than I expected for an MVP. A couple of practical notes in case anyone else tests it: * service principal needs access not just to the data, but also to the SQL warehouse / compute * I got it working fine on Databricks Free Edition * if you don’t cache responses, repeated queries can get expensive fast because you’re paying for warehouse runtime I still wouldn’t treat this as my default production setup, but for demos / internal prototypes/idea validation, it was surprisingly useful. I wrote a short article with examples - [https://medium.com/@protmaks/databricks-lovable-a-practical-case-study-and-what-it-costs-to-build-an-app-085f61b07126](https://medium.com/@protmaks/databricks-lovable-a-practical-case-study-and-what-it-costs-to-build-an-app-085f61b07126)

Lakebase login via REST for a service principal

Databricks CLI token creation fails with “cannot configure default credentials” after previously working in CI pipeline

I have been generating a Databricks personal access token in my YAML-based CI pipeline using a bash script. The pipeline installs the Databricks CLI and then creates a token using a Service Principal (Azure AD application) credentials. Current working approach (previously working) #!/bin/bash dbx_host="${1}" dbx_client_id="${2}" dbx_client_secret="${3}" # Set the Environment Variables for Databricks authentication export DATABRICKS_HOST=$dbx_host export DATABRICKS_CLIENT_ID=$dbx_client_id export DATABRICKS_CLIENT_SECRET=$dbx_client_secret echo "Creating a new Databricks token" response=$(databricks tokens create \ --lifetime-seconds 31536000 \ --comment "Token for SPN for EDH Data Access. Validity 1 year.") echo "Token Created Successfully" token=$(echo $response | jq -r '.token_value') token_id=$(echo $response | jq -r '.token_info.token_id') expiry_time=$(echo $response | jq -r '.token_info.expiry_time') This used to work fine for generating tokens. Issue Recently, the same pipeline started failing with the following error: Error: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://***, account_id=***, workspace_id=***, profile=DEFAULT, azure_tenant_id=***, client_id=***, client_secret=*** Env: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET The documentation link provided in the error message does not really help in identifying what exactly needs to be changed or how to fix this specific CI/CD use case. Has there been a recent change in Databricks CLI authentication (especially unified authentication) that breaks Service Principal authentication using DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET environment variables? Any guidance or migration steps would be appreciated. UPDATE:Added Tenant ID […truncated]