Skip to content
All topics

Service Principal

Recent items mentioning Service Principal across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.

56 recent items20 releases6 videos30 community threads
What's happening in Service PrincipalAI synthesis · updated Nov 16, 2026

Databricks Java SDK v0.178.0 introduced dedicated fields to support PostgreSQL service principals and credential generation 3. In parallel, practitioner discussions highlight service principal management as a key focus when promoting Databricks Apps into production 4 and tracking deployment history within Databricks Asset Bundles (DABs) 5.

Generated daily from the 7 most recent items mentioning Service Principal. Click any [N] to jump to the source.

Reddit

Governed Spreadsheets on Databricks: Row Zero Acquisition & What It Means for Business Users on Databricks (Demo + Chat)

Hey everyone! Had the wonderful opportunity of recording a demo + chatting with Breck, the founder & former CEO of Row Zero, a governed spreadsheets platform Databricks just acquired. For business users, this will pair up nicely alongside the Genie One / Genie Agents experience. Some quick answers to common questions the video answers (used AI + transcript for them): When did Databricks acquire Row Zero? =The acquisition was announced in September 24, 2026. Can Databricks customers use Row Zero today? =Yes, with a Row Zero account. A built-in Databricks experience doesn't have a date yet. How will it be priced? =Not final. Expected to follow the rest of Databricks (pay for what you use), not per seat. Does it respect Unity Catalog permissions? =Yes. Row Zero queries Databricks as the signed-in user (OAuth recommended, service principals supported), so row-level security carries through. Can you write back to Databricks? =Yes, to tables with a primary key. You review a diff before changes apply. Can you export back to Excel? =Yes, to Excel or CSV. Exports over about 1 million rows get truncated because of Excel's row limit, and admins can turn export off entirely. All roads in data begin and end with a spreadsheet after all! PS: Always love to hear your feedback on the video. submitted by /u/JosueBogran [link] [comments]

00JosueBogran3d ago
Reddit

Seeking feedback on my Databricks data engineering portfolio project

Hi everyone, I’m preparing to apply for junior/entry-level Data Engineer roles focused on Databricks, and I’d appreciate honest feedback on a portfolio project I’ve been building. It is a governed lakehouse project for a synthetic table-tennis e-commerce dataset: - Deterministic input contract and synthetic fixture with intentional duplicates, malformed records and quality issues. - Bronze ingestion designed around Databricks Asset Bundles, Auto Loader, append-only raw tables, lineage metadata and retained checkpoints. - Silver transformation with explicit validation rules and a quarantine layer. - Gold publication with privacy controls: only aggregated, non-PII cells that meet a k-anonymity threshold are published. - Unity Catalog / Terraform / dedicated service principal / least-privilege model. - Manual serverless Databricks Job with materialization followed by a read-only verifier. - 287 local tests covering contracts, transformations, bundle configuration, verifier logic and Terraform privilege rules. The current DEMO-001 cloud milestone produced: - Silver: 15 orders, 10 customers, 7 products - Quarantine: 6 expected rejected rows - Gold: €640 revenue, 5 orders, 0 refunded lines - Pre-suppression controls: €812.20, 13 orders, 2 refunded lines - Status: COMPLETED_WITH_QUARANTINE I have deliberately kept the scope honest: this is a fixed deterministic snapshot, not CDC, streaming, SCD, generic incremental processing, SQL Warehouse, Power BI, or a production monitoring setup. Repository: tt-ecommerce I would especially value feedback on: Does this demonstrate relevant Databricks/Data Engineering capability? What is missing before using it in applications? Would you prioritise finishing governance/consumer access, adding SQL Warehouse/BI, or CI/CD and operational monitoring? Is the documentation clear, or does it feel over-engineered for a portfolio project? What would you expect to see in a strong junior Databricks project that is not here yet? Thanks — direct criticism is welcome. submitted by /u/Alone-Cow-1563 [link] [comments]

00Alone-Cow-15636d ago
Reddit

Databricks Apps: Are you using them in production yet or keeping it to internal prototypes?

Hey everyone, I’ve been diving into Databricks Apps recently and wanted to see how others are adopting it in their organizations. Historically, whenever we wanted to spin up a Streamlit app or a custom Python dashboard connected to our data in Databricks, it meant dealing with a headache of external infrastructure—setting up App Service or EC2, dealing with network security groups, configuring service principals, and managing separate auth flows. Being able to run Streamlit/Dash apps directly inside the Databricks workspace, backed by Unity Catalog for governance and serverless compute, feels like a massive quality-of-life improvement for internal tooling. I’m curious to hear from anyone who has put them into practice: Are you using Databricks Apps for production-grade tools, or mostly for quick internal MVPs and demos? How are you managing your CI/CD deployments for these apps? Have you run into any weird gotchas with state management, scaling, or dependencies? submitted by /u/SalamanderOpen859 [link] [comments]

00SalamanderOpen8591w ago
Reddit

DABs deployment history

DABs can now record deployment versions, status, managed resources, and the user or service principal that performed each deployment. https://medium.com/databrickscommunity/databricks-news-genie-cli-app-builder-agent-memory-dabs-41820427fcc2 submitted by /u/hubert-dudek [link] [comments]

00hubert-dudek1w ago
Reddit

What’s new in Genie Agents — October 2026

Databricks has introduced significant feature upgrades to Genie Agents, enhancing execution speed with Flash mode, expanding external tool actions via MCP, automating volume knowledge search, and introducing granular budget access controls. Agent Models and performance Flash (Beta) : Prioritize rapid execution speed over analytical depth for routine everyday tasks. Agent mode model upgrade : Agent mode now uses upgraded models that better handle long context and respond faster. Integrations and external tools Connect to external tools (Beta) : A Genie Agent can search and take actions in external tools like Google Drive through MCP connectors. Volumes and content search Faster file retrieval (Beta) : Content search parses and indexes your files so the Genie Agent can reason over a larger set of files, draw on specific passages across many files, and answer questions with lower latency. Content search details hidden from viewers (Beta) : The context panel no longer shows technical content search details, such as the number of files indexed, to users with CAN VIEW or CAN RUN permission on a Genie Agent. Automated Content Search and Volume Sync (Beta): When you attach a volume to a gent agent, Databricks automatically enables content search and keeps the index continuously updated as files are added or removed without inquiring any manual intervention. Volumes from restricted catalogs (Beta) : You can attach a volume as a source for a Genie Agent even when the volume’s catalog isn’t accessible from the agent’s workspace by default Authoring and Embedding Guided agent creation : A new guided experience walks you through creating an effective Genie Agent using Genie Code. It reads your data and suggests context that helps Genie answer questions accurately, such as descriptions of your tables and example queries. Deletion warning for sources and instructions : Deleting sources or instructions from a Genie Agent requires explicit permission. Volume file links in embedded agents : Links to volume files in an embedded Genie Agent open the file instead of redirecting users to the workspace. Governance and budget Budgets used to manage Genie costs support access control lists (ACLs) : Account admins and workspace admins can grant the Budget Manager or Budget Viewer role to specific users, groups, or Databricks service principals, delegating who can view and manage each budget. submitted by /u/Youssef_Mrini [link] [comments]

00Youssef_Mrini2w ago
Reddit

Power bi import gateway strategy

How are you guys that still use power bi via import mode designing your gateway authentication? We are new to databricks and have a lot of schemas that require different levels of AD access. Import mode is the company standard so no direct query sso auth. We are unsure how to design a scalable gateway strategy . Do we use a service principal or allow publishers to use their own credentials on the gateway? submitted by /u/curiouscsplayer [link] [comments]

00curiouscsplayer3w ago
Reddit

How do you manage, in a Databricks Asset Bundles-based CI/CD, the permissions and access resources that a bundle cannot declare?

We have a DABs-based CI/CD in a monorepo and want to split it: deploy the DABs from the data-code repo and keep the Terraform infrastructure in a separate repo. The original idea was that Terraform would change at a different pace and its deployments would be manual, with no automated dependency on DABs deploys. However, at work they’re now saying this infrastructure part should be tied to the DABs deployment and not be manual, and I’m unsure where it should live and how to automate it. We mainly run into four cases: Cross-boundary grants: our pipeline needs to read a table owned by another team, so we need to grant SELECT on that table/schema to our service principal. Postgres grants: a job needs to connect to a Lakebase database and requires a GRANT on a Postgres schema, table, or role. Share contents: we need to add or remove tables and views from a Delta Sharing share consumed by an external consumer. Notification destinations: a job or pipeline needs to send alerts to a notification destination (email, Slack, webhook) that the bundle cannot declare. Today, all of these live in Terraform outside the bundle, so every time the data code needs one of these changes, we have to coordinate a separate MR and deployment in addition to the bundle deploy. Where do you put these in your pipeline: inside the bundle, in a separate Terraform stack applied after the bundle deploy, in the platform repo, or another way? What issues have you run into with each approach? Tx so much submitted by /u/Appropriate_Tip_9580 [link] [comments]

00Appropriate_Tip_95801mo ago
Reddit

Run Lakeflow pipelines as a group: governance win or a way around SP gatekeeping?

As of this month you can set a Lakeflow pipeline’s run-as identity to an account group. Tables the pipeline publishes become group-owned in UC. Two patterns I expect to see: 1. Routing around SP provisioning. In a lot of orgs, creating a service principal is process-locked by a platform or identity team. Tickets, approvals, secret rotation. With run-as group, a team that already has an account group with the right grants can skip all of that. The conversation moves from “who can create an SP” to “can we just use the DE group?”. 2. Lower-maturity teams leapfrogging SPs. Teams still running pipelines as individual users will likely jump straight to groups. That’s better than personal identities, since nothing breaks when someone leaves. But it means they may never adopt SPs at all, and “good enough” becomes the permanent pattern. Questions for the sub: If you run pipelines as SPs today, are you planning to switch any to groups? Platform/admin folks: are you going to restrict this, and how? How are you handling auditability when the identity is a group instead of a single SP? Is group run-as a legit middle step for teams coming off user identities, or a dead end? My take: SPs still belong in prod. But I’d bet most orgs end up with group run-as spreading through both of these doors, and won’t notice until an audit asks who ran what. submitted by /u/OkImprovement7010 [link] [comments]

00OkImprovement70101mo ago
Databricks CommunityGenerative AI

Databricks Genie One - Managing past conversation initiated by Databricks service principal account

001mo ago
Databricks CommunityData Engineering

Are PATs (user or service principal) accepted by Databricks Apps?

001mo ago
Databricks CommunityGenie Hub

Service principals created by Genie App builder are not visible in Databricks UI

001mo ago
Reddit

You can now run an SDP pipeline as a group

A pipeline's run-as identity can now be an account group. Updates run with the group's permissions, and the tables the pipeline publishes are owned by the group. What it is Every pipeline runs as some identity. That identity decides what the pipeline can read and write, and who owns the tables it creates. Until now you could pick a user or a service principal. You can now pick an account group too. A Databricks role is a group under the hood, so setting run-as to a group also lets a pipeline run as a role. Why you'd want it If a pipeline runs as one person, it depends on that person's account staying active. When they leave and the account is deactivated, the pipeline can stop running and the tables it published can become inaccessible to everyone else. Service principals already fix part of this. Groups go a step further: a whole team owns the pipeline, and it keeps working as people join and leave the team. No owner reassignment when someone moves on. How to use it https://preview.redd.it/1ml56gev61uh1.png?width=1848&format=png&auto=webp&s=40275cf04ac3a837108289df8e8f16a855acfd98 In the UI: Open the pipeline, click Settings. Click the pencil next to Run as. Pick an account group you belong to, or one you have Assume permission on. Groups show up next to users and service principals. Click Save. The new identity applies on the next update. You can also assume a group’s identity to change / create a pipeline. And you can change the owner of a standalone MV/ST to a group as well! { "name": "sales-etl", "run_as": { "group_name": "sales-eng" } } Declarative Automation Bundles: resources: pipelines: sales_etl: name: sales-etl run_as: group_name: sales-eng What you need CAN_MANAGE on the pipeline. Either membership in the group, or Assume permission on it (for groups used for exclusive access). The group has to be an account group assigned to the workspace. Workspace-local groups don't work, and neither do the built-in admins, users and account users groups. Grant the group every permission the pipeline needs. Updates run as the group, so your own access to the source and data isn't enough. Docs: https://docs.databricks.com/aws/en/ldp/configure-pipeline#set-the-run-as-identity-to-a-group Please let us know what you think! submitted by /u/saad-the-engineer [link] [comments]

00saad-the-engineer1mo ago
Reddit

Azure Databricks, two-week-old account: partner pay-per-token endpoints (Claude, Grok) have never once succeeded — "Databricks-set rate limit of 0" — and the Assistant has "no daily token allowance". Open models work. What gates this?

Azure Databricks, Premium, one account with three workspaces, Unity Catalog, Azure Marketplace billing (no card, no trial credits left). The account was bootstrapped on a trial workspace about two weeks ago and moved to Premium six days later. I've spent two days on this and want a sanity check from anyone who has seen it. The facts, from system.serving.endpoint_usage: • Every open-weight pay-per-token endpoint has worked since the trial: gpt-oss-120b has ~50 successful calls going back to the trial period, Llama 3.3 70B likewise, zero refusals. • No partner pay-per-token endpoint has ever returned a success on this account. The very first call anyone made to databricks-claude-sonnet-5 was refused, and every call since, on every Claude endpoint and on Grok, in all three workspaces, for users and service principals alike: 403 PERMISSION_DENIED: The endpoint is temporarily disabled due to a Databricks-set rate limit of 0. • It is not our AI Gateway config: I removed every rate limit from one Claude endpoint and invoked it — same 403. The endpoints' config shows nothing else. • The notebook Assistant worked during the trial. On the paid account it refuses everyone, admins included: *"This workspace has no daily token allowance for the assistant."* Underneath, /ajax-api/2.0/conversation/llmproxy/ returns 429 {"type":"daily_token_limit_reached","daily_limit_tokens":0,"message":"…Daily token limit of 0 reached. Resets at midnight UTC. (DTB)"} and it recomputes to 0 every midnight. • Once, the Databricks Apps in the workspace were stopped by the platform with *"App compute was stopped due to workspace or account status"* while every workspace showed RUNNING. apps start brought them back. Red herring, for completeness: we also created a Unity AI Gateway per-user budget whose first version had a $0 threshold with BLOCK_USAGE. That blocked Genie for a day, exactly as documented, and was fixed. It was created a day *after* the first Claude refusal, so it isn't the cause of any of the above. What I've found: two Community threads describe "Databricks-set rate limit of 0" as a workspace trust-tier gate — trial-born accounts sit in TRIAL_VERIFIED, pay-per-token partner models are gated to PAYABLE_VERIFIED, a card alone doesn't flip it, only Databricks Sales/Support moves the tier. Both were AWS/personal accounts, neither resolved on-page. The account console's own API reports our account's feature_tier as STANDARD_W_SEC_TIER although the workspaces are Premium; I can't find what that field means. Questions • Does Azure Databricks with Marketplace billing go through the same PAYABLE_VERIFIED gate? Does it clear on its own after the first settled invoice, or does someone have to move it? • If you were moved: who did you contact (account team, or "Contact us" in the console) and how long did it take? • Is the Assistant's "daily token allowance" the same gate, or a trial allowance that simply goes to 0 when the trial ends on an unverified account? • Does feature_tier: STANDARD_W_SEC_TIER on the account object mean anything to anyone? submitted by /u/sumit671 [link] [comments]

00sumit6711mo ago
Reddit

Avoid Unity Catalog external location permission bugs by using managed storage

Hey, I work as a Databricks Engineer at Abilytics, so here's my take. If you are migrating existing tables to Unity Catalog, defining external locations incorrectly will break your grants. When you register an external location via CREATE EXTERNAL LOCATION, Unity Catalog validates your storage credential against the underlying cloud storage container. However, users often forget that read and write access on the external location is distinct from the IAM role or service principal trust policy. If a user has SELECT on a table, but lacks BROWSE on the external location, queries against parquet files directly via path-based access will fail with permission denied, even if the table-level grant succeeded. Always verify your storage credentials have explicit container-level policies attached before mapping external volumes. Furthermore, remember that path-based access like spark.read.load('s3://my-bucket/path') requires explicit external location grants, whereas managed tables abstract this entirely. Prefer managed tables default storage roots to bypass manual external location ACL synchronization issues entirely. Happy to go further — more on Databricks | Platform Engineering | AI at Abilytics , AI Systems, Data Engineering & Platform Engineering Services. submitted by /u/AbilyticsEng [link] [comments]

00AbilyticsEng1mo ago
Databricks CommunityData Engineering

Databricks now deletes unused service principal secrets after 90 days

001mo ago
Databricks CommunityLakebase Discussions

Lakebase Data API (GCP) returns jwk not found for valid service principal tokens

001mo ago
Reddit

Issues with oidc/v1/token endpoint (REST for zerobus-ingest)

Anyone else work with the zerobus-ingest? There is an auth-flow to get an "oauth token" from a service principal with an "oauth secret". In that flow, you are supposed to submit the authorization details. See docs: https://learn.microsoft.com/en-us/azure/databricks/ingestion/zerobus-kafka I had been requesting access (via authorization_details) to a table as shown in the example: "type": "unity_catalog_privileges", "privileges": ["SELECT"], "object_type": "TABLE", "object_full_path": "main.default.air_quality" ... but it wouldn't work. Was banging my head on it all day. The privileges granted to the service principal for this table were unrestricted " ALL PRIVILEGES ". It turns out that, in addition to "ALL PRIVILEGES", this "token" endpoint wanted me to grant "SELECT " to the service principal in UC governance as well. Else the authorization_details are not valid. This "token" api has a lot of sharp edges. Any chance there is a nuget client to wrap around it and protect us from these unfortunate nuances? Is it documented that ALL PRIVILEGES doesn't actually give us all privileges, if we request "SELECT" by name? submitted by /u/SmallAd3697 [link] [comments]

00SmallAd36971mo ago
Reddit

Ontology ranked the snippet. I wrote the gate I wanted before trusting a Genie space.

A commercial user asked what yield was last season. Two definitions live in the same lakehouse. One is plot-level, moisture-adjusted, borders out. The other is as-harvested, and a dashboard still uses it. Genie returned the second number. Not a hallucination. OntoRank did what it does: creator, usage, link to a certified asset, freshness. The loud dashboard won. Nobody had certified the measure the board pack is allowed to quote. That is the failure I care about. A green demo is not a release gate. What I actually require before that space faces a user who will paste the sentence into Slack: Golden questions written before the space exists. If you write them after you have tuned instructions, you are scoring an overfit prompt. Benchmarks are not instructions. Genie does not learn from the benchmark SQL. The number lives in a Metric View. A column comment and a knowledge-store formula are hints. Ontology will rank all three. Only the Metric View is a contract every surface compiles. Same KPI from MEASURE(), from the certified dashboard, and from Genie. If they disagree, the space stays off Slack. Permissions checked as the asker. Inside a Genie Agent, compute is the author’s warehouse and data access is the end user. That split does not automatically hold in Teams, Copilot on maker credentials, or a Databricks App on its default service principal. The row filter still runs. It runs for whoever showed up on the SQL. A written kill line. I use Databricks’ “above 80% before UAT” as the hold line, not the ship line. Ship is 90% and zero fails on a regulated definition. Any miss on that definition takes the space down. A person does that. Not a service principal. Genie Code can draft the Metric View and the MERGE. ZeroOps can propose a fix and wait for Approve. Neither owns the definition. If the job stays green for eleven days because a key was slightly wrong, that is still your incident. I wrote the four pieces up so I would stop re-explaining this in reviews. Argue the 80 vs 90 line if you want. I will not move a regulated KPI to “looks fine in the demo.” submitted by /u/Worth_Phase1358 [link] [comments]

00Worth_Phase13581mo ago
Reddit

Databricks pipeline started failing after moving tables to Unity Catalog — how would you debug it?

Suppose a pipeline was previously using: hive_metastore.schema.table and is migrated to: catalog.schema.table The transformation logic hasn't changed. But the pipeline now fails during execution. Possible areas to investigate: Catalog permissions Schema permissions Service principal Pipeline identity Cluster access mode External location Storage credential Table ownership Hardcoded table references The interesting part is that the SQL may be perfectly valid. How would you troubleshoot this systematically? submitted by /u/hanshu6576 [link] [comments]

00hanshu65761mo ago
Reddit

Databricks pipeline works in one workspace but fails in another — where would you start?

Imagine the same pipeline and code are deployed to two Databricks workspaces. Workspace A: catalog.schema.table → Query works → Pipeline works Workspace B: catalog.schema.table → Catalog not accessible → Pipeline fails The catalog exists, the table exists, and the SQL itself is valid. Would you first investigate: Workspace-to-catalog assignment Unity Catalog permissions USE CATALOG USE SCHEMA SELECT privileges Service principal permissions External locations Storage credentials Cluster access mode SQL Warehouse identity I'm curious about the actual troubleshooting order people follow for these kinds of Unity Catalog issues. What would you check first? submitted by /u/hanshu6576 [link] [comments]

00hanshu65761mo ago
Reddit

"Do we need a metastore per workspace?" - the answer, and why the question keeps coming up

Every new joiner on our platform team has asked some version of this, so here is the short version. You do not. The Unity Catalog metastore is created in the account console and you have one per region, linked to as many workspaces in that region as you like. Dev, test and prod attach to the same one and share a single catalog.schema.table namespace. The mental model of "each workspace has its own metastore" comes from the legacy Hive metastore, and that is exactly what Unity Catalog replaces. What actually belongs where: - Account: billing, account admins, users, groups, service principals, and the metastore itself. - Workspace: notebooks, clusters, jobs, dashboards, the place people work. - Metastore: catalogs, schemas, tables, views, volumes, and the permissions on them. Identity federation means a group is configured once at account level and assigned to workspaces, so a grant on data applies wherever that group works. Workspace-local groups still exist in non-federated workspaces, but they cannot be granted Unity Catalog permissions, which is a good reason to migrate them. Storage: metastore-level managed storage is optional, and anything already in a bucket becomes an external location with a storage credential. Drew it out as a 2-3 minute animated diagram: https://youtu.be/2yy9-BL_RDo Series in order: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4 How are you separating environments inside one regional metastore: catalog per environment, workspace-catalog bindings, or separate regions? submitted by /u/Broad-Cherry-5999 [link] [comments]

00Broad-Cherry-59991mo ago
Reddit

"Do we need a metastore per workspace?" - the answer, and why the question keeps coming up

This comes up constantly in real platform design, and the confusion is almost always about which layer owns what. The account is the container for the organisation, normally one per cloud provider. Billing, account admins and identities live there. A workspace is a single deployment of the UI: notebooks, clusters, jobs, dashboards. Most teams run dev, test and prod as separate workspaces. The Unity Catalog metastore is the part people place wrongly. It is created at account level, not inside a workspace, and the docs say you have one metastore per region, linked to any number of workspaces in that region. So dev, test and prod in the same region share one metastore and one catalog.schema.table namespace, instead of each keeping a separate legacy Hive metastore. Identities follow the same shape: users, groups and service principals are configured once in the account console and then assigned to workspaces, so you grant a group access to data once rather than repeating it per workspace. Storage is still yours. Metastore-level managed storage is optional, and existing buckets are registered as external locations backed by a storage credential. One line to remember it by: the account holds identities and billing, workspaces hold the tools and compute, the metastore holds the data and who can see it, shared across every workspace in the region. I drew it as a short animated diagram because the hierarchy sticks better visually (2:32): https://youtu.be/2yy9-BL_RDo Episode 3 of a series, in order here: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4 For those running dev and prod in the same region: are you isolating with separate catalogs, workspace bindings, or something else? submitted by /u/Broad-Cherry-5999 [link] [comments]

00Broad-Cherry-59991mo ago
Reddit

Can Databricks PAT authentication be used to mint an embed token?

Iam working on a Databricks integration where we support both PAT token and Service Principal authentication. The PAT connection works fine for making Databricks API calls, but when I tried to generate an embed token for dashboard embedding, it doesn’t seem to work. If PAT is not supported, is there any official reason or limitation behind this? Would appreciate any clarification from people who have implemented Databricks dashboard embedding. submitted by /u/Bhanuprakash_1947 [link] [comments]

00Bhanuprakash_19472mo ago
Reddit

What credential should I use to authenticate Databricks Zerobus?

I’m learning how to use Databricks Zerobus and I’m confused about authentication. When connecting to Zerobus, should I use a Databricks personal access token, OAuth credentials, a service principal, or another type of credential? I’m testing this for a learning project and want to follow the recommended secure approach. I won’t post any actual credentials. What authentication method are people using, and are there any setup steps or permissions I should know about? submitted by /u/Stu-dent999 [link] [comments]

00Stu-dent9992mo ago
Databricks CommunityData Engineeringanswered

Managing Service Principal Permissions at Scale: External Locations vs. Managed Volumes

002mo ago
Databricks CommunityData Engineering

System generated Service Principal -App

002mo ago
Reddit

Serverless Access Control is here!

We finally have a simple way to control access to serverless in the workspace. Two built-in objects have been introduced: - Default Interactive Compute - Default Automated Compute The first one supports notebooks and databricks connect. Second one jobs and SDP pipelines. To limit who can use serverless: Click Compute in the workspace sidebar. In the Serverless tab, click the kebab menu next to Default Automated Compute , then click Edit permissions . Remove the All Users group, or the group that includes all workspace users. Add only the specific users, groups, or service principals that you want to authorize. https://preview.redd.it/sg59jole9hkh1.jpg?width=957&format=pjpg&auto=webp&s=0c33d19d04c57856c51f57f33c44719317fe8347 submitted by /u/szymon_dybczak [link] [comments]

00szymon_dybczak3mo ago
Databricks CommunityData Engineering

Declarative Asset Bundle Service Principals best practices

004mo ago
Databricks CommunityAdministration & Architectureanswered

Zerobus ingestion fails in Databricks Free Edition using Service Principal OAuth

004mo ago
RedditDiscussion

Why Databricks Genie returns MessageStatus.FAILED in App API (but works fine in UI) — solved

Hey fellow Databricks builders, Our team at **Enqurious** just wrapped a hackathon building a full medallion architecture (Bronze → Silver → Gold) with Genie AI integration for natural language querying. We hit a wall that cost us 6+ hours — sharing the full fix so you don't have to. # What was the setup? * Gold layer tables in Unity Catalog * Genie Space configured and working in UI * Databricks App deployed successfully # What causes MessageStatus.FAILED in Databricks Genie App API? The root cause is a missing service principal permission chain. The Databricks App runs under a **service principal** — not your user account. Genie working in the UI only proves *your user* has access, not the service principal the App uses at runtime. This is why the same question works in the Genie UI and fails via the App API every time. # What is the complete permission fix? The service principal needs **all five** of these — missing any one causes silent failures: sql -- 1. Catalog level (most commonly missed) GRANT USE CATALOG ON CATALOG your_catalog TO `your-app-service-principal`; -- 2. Schema level GRANT USE SCHEMA ON SCHEMA your_catalog.your_schema TO `your-app-service-principal`; -- 3. Table level — each table individually GRANT SELECT ON TABLE your_catalog.your_schema.table_1 TO `your-app-service-principal`; GRANT SELECT ON TABLE your_catalog.your_schema.table_2 TO `your-app-service-principal`; **Plus in the UI:** * SQL Warehouses → your warehouse → Permissions → Add service principal → **Can Use** * Genie Space → Share → Add service principal → **Can Run** # Where do you see the real error (not just MessageStatus.FAILED)? The actual error is in the **Genie Monitoring tab** inside your Genie Space — not the App logs. App logs only show the generic `MessageStatus.FAILED`. The Monitoring tab shows the specific SQL error or permission denial that caused it. This single tip would have saved us 3 of those 6 hours. # How long do Databricks permissions take to propagate? Unity Catalog permissions take **5–15 minutes** to propagate. Testing immediately after granting permissions is the most common reason the fix appears not to work. Wait 10 minutes, hard-refresh your browser, then retest. # Quick debug checklist If you're still hitting `MessageStatus.FAILED`: □ Check Genie Monitoring tab for the actual SQL error □ Confirm service principal has USE CATALOG (not just SELECT) □ Confirm SELECT on each table individually — schema-level alone is not enough □ Confirm SQL Warehouse is Running, not Stopped □ Wait 10 minutes after any permission change before retesting □ Hard refresh browser before retesting □ Verify app.yaml uses ["streamlit", "run", "app.py"] — not ["python", "app.py"] # FAQ **Q: Why does Genie work in the UI but fail in the Databricks App?** A: The App runs under a service principal, not your user account. The service principal needs its own separate permission chain regardless of your personal access. **Q: Is USE CATALOG required even if SELECT is granted on all tables?** A: Yes — USE CATALOG is mandatory. Without it the service principal cannot traverse the catalog hierarchy, even with table-level SELECT grants. **Q: Where is the real error when Genie fails in an App?** A: Genie Space → Monitoring tab. App logs only show the generic failure; Monitoring shows the specific cause. **Q: How long do Unity Catalog permissions take to propagate?** A: 5–15 minutes. Always wait before retesting. The Enqurious team wrote up a full walkthrough with SQL commands, screenshots, and the complete [app.py](http://app.py) template Happy to share our Genie Space config or [app.py](http://app.py) in the comments — just ask. **Questions for the community:** * Has anyone else faced similar Genie + App permission issues? * Is there a better way to debug these permission chains? * Would love to hear if Databricks is working on si […truncated]

53Square-Mix-13025mo ago

Get Tuesday's version of this

Tracking Service Principal? The Tuesday email carries what moved across the whole ecosystem, not just this topic. Free, one-click unsubscribe.

Read past issues first