MCP

Problems with UC MCP connection

We built a supervisor agent where one of the tools is the native databricks MCP connection (which is in beta). Today, the agent doesn’t work. Removing the MCP tool fixes it, so it’s definitely that. The agent response just says “received an invalid and unexpected value from the API: undefined” Anyone else having problems suddenly with the MCP server? I’m able to hit the target MCP URL locally no problem. And even test the UC MCP in databricks successfully. So it’s something about the handshake between MCP and the supervisor agent submitted by /u/pboswell [link] [comments]

Genie One MCP is now GA

The Genie One MCP server is now GA. It exposes Genie One over MCP, allowing any agent to communicate with Genie One as a peer agent. The MCP exposes tools for asking questions to Genie One, getting query results, checking on incremental progress, and steering responses. These capabilities allow you to integrate Genie One into whatever agent you want without changing your workflow. And you get all of the semantics built-in instead of having to use raw SQL APIs with fragmented skills/stale markdown repos/additional semantic layers like we used to with ai devtools. Here are the docs: https://docs.databricks.com/aws/en/agents/mcp/genie-mcp I've seen some pretty cool use cases with integration into ChatGPT/Claude, but also headless agent workflows where you want to delegate the data questions to Genie One so it can use agents/ontology. Interested in folks' thoughts on the right/wrong use cases for this feature. submitted by /u/lakehouse_vacation [link] [comments]

What’s new in Databricks - August 2026

Databricks shipped many major Generally Available features in August 2026. Here is the breakdown of what just landed: 🚀 Unity AI Gateway Enterprise AI governance layer covering model access, Model Context Protocol (MCP) management, and cost observability. 🔒 Role-Based Access Control (RBAC) Switch to scoped, temporary role assumptions instead of dealing with permission bloat. 🔑 Secrets in Unity Catalog Unified security secrets are now governed, 3-level namespace securable objects. ⚙️ Serverless Compute Access Control Granular admin controls over who can trigger serverless workloads across your organization. ⚡ Lakebase Postgres APIs & LTAP Direct Writes Accelerated synced-table loads and improved transactional data integration. 🤖 Genie Agent Upgrades Official GA releases for both the Agent mode API and Full-page Genie Code view. submitted by /u/Youssef_Mrini [link] [comments]

Multiple Databricks Genie MCP connectors in one Claude workspace collapse into a single shared connector, anyone solved this?

We're connecting Claude (Cowork/Claude Desktop) to several separate Databricks workspaces, each with its own Genie space. Running into a platform-level snag and hoping someone's hit this before. Setup: each workspace exposes the standard Genie MCP endpoint ( https:// /api/2.0/mcp/genie ). We add each as its own connector in Claude so we can ask natural-language questions against the right workspace's data. Problem 1, generic connectors are indistinguishable: every Genie MCP server reports the same tool names and the same generic description, regardless of which workspace it's pointed at. Claude has no way to tell them apart from metadata alone, so it either has to probe each one (asking a neutral "what workspace is this?" question and checking the returned deep_link ) before it can route a question correctly, or it guesses wrong. Problem 2, trying to fix it with named connectors backfires: we tried building a Claude plugin that declares each of the seven as a distinct, fixed-name MCP server (e.g. genie-workspace-a , genie-workspace-b , ...), hoping deterministic names would remove the need for probing entirely. Turns out Claude's client recognizes Databricks Genie as a "verified" connector type in its directory, and collapses all seven fixed-name declarations into a single shared authorization instead of keeping them independent. Authorizing one silently becomes "the" Genie connection, and the others just reflect that same state rather than getting their own. Problem 3, working around the collapse trades one problem for another: we found that changing the declared URL slightly (adding a harmless query param) makes the client stop recognizing it as the verified type, which does force it down the plain custom-connector path and keeps it independent. But then it needs an actual registered OAuth client ID against that Databricks workspace, since it no longer gets whatever implicit OAuth handling the verified integration has. That means registering an app per workspace in Databricks/Azure just to get back to where the plain manual setup already was, more infrastructure, not less. So right now we're back to manually-added generic connectors plus a routing skill that probes and caches per conversation, which works, but costs a lookup the first time each workspace is used per session, and never scales cleanly as we add more workspaces. Has anyone gotten multiple Genie MCP connections into the same Claude account/org without them colliding into one shared authorization? Is there a way to deploy or configure the Genie MCP endpoint itself so each workspace reports distinguishable tool names or descriptions, rather than relying on the client to differentiate them? Btw, in Codex/chatgpt this is not an issue, because we can edit the description/name, and it automatically discovers the right plugin to use. It seems this is a very claude thing issue submitted by /u/Akroma188 [link] [comments]

Whether MCP server support for Genie is available in our workspace/region for free version.

I built an open-source Text-to-SQL agent for Databricks Unity Catalog

I've been working on an open-source side project called [Mega Djinn](https://github.com/ocelma/mega-djinn) to solve the *"where is this data?"* problem, and I wanted to share it with you all.  **1. What it does** You ask a natural language question (e.g., *"Which web pages kept readers engaged the longest last month?"*) and the agent handles the rest: 1. *Table Discovery*: Automatically finds the relevant tables and schemas inside Databricks Unity Catalog. 2. *Governance Enrichment*: (Optional) Pulls business definitions and approved SQL snippets from Alation to ground the prompt. 3. *Human-in-the-Loop*: Generates optimized SQL and displays it for your review before execution. 4. *Execution*: Runs the query on Databricks (via CLI or MCP server), returns the results, and generates an HTML report. 5. *Memory*: Saves successful queries back to the knowledge base so your team can reuse them. **2. Why I built it** Our data teams were drowning in repetitive ad-hoc requests from PMs and business stakeholders; people who have critical data questions but not enough SQL background. # The ultimate goal is to give non-technical users direct access to data insights without always depending on a data analyst. Mega Djinn is much simpler and lightweight than Vanna: it uses your existing data governance metadata (UC schemas + Alation definitions if available) to ground the context, making the generated SQL accurate to your specific business logic.  **3. The Tech Stack** * Core: Python + Databricks SDK * LLM Integration: Built as a Claude Code skill. Also works via Cursor, Codex, and Gemini CLI. [https://github.com/ocelma/mega-djinn](https://github.com/ocelma/mega-djinn) I would love to get your feedback, feature requests, or thoughts on how you handle text-to-SQL governance!

Genie UI MCP Server Connection Issue

Show HN: Mljar Studio – local AI data analyst that saves analysis as notebooks

Hi HN, I’ve been working on mljar-supervised (open-source AutoML for tabular data) for a few years. Recently I built a desktop app around it called MLJAR Studio. The idea is simple: you talk to your data in natural language, the AI generates Python code, executes it locally, and the whole conversation becomes a reproducible notebook (*.ipynb file). So instead of just chatting with data, you end up with something you can inspect, modify, and rerun. What MLJAR Studio does: - Sets up a local Python environment automatically, runs on Mac, Windows, and Linux - Installs missing packages during the conversation - Built-in AutoML for tabular data (classification, regression, multiclass) - Works with standard Python libraries (pandas, matplotlib, etc.) - Works with any data file: CSV, Excel, Stata, Parquet ... - Connects to PostgreSQL, MySQL, SQL Server, Snowflake, Databricks, and Supabase. For AI: use Ollama locally (zero data egress), bring your own OpenAI key, or use MLJAR AI add-on. I built this because I wanted something between Jupyter Notebook (flexible but manual) and AI tools that generate code but don’t preserve the workflow. Most tools I tried either hide too much or don’t give reproducible results and are cloud based Demos: - 60-second demo: https://youtu.be/BjxpZYRiY4c - Full 3-minute analysis: https://youtu.be/1DHMMxaNJxI Pricing is $199 one-time, with a 7-day trial. Curious if this is useful for others doing real data work, or if I’m solving my own problem here. Happy to answer questions. --- top comments --- [MSaiRam10] Notebooks as the output format is funny because notebooks are famously bad for reproducibility. Out of order execution, hidden state, etc. You're solving "chat isn't reproducible" with a format that also isn't really [hasyimibhar] How does this compare to open source Deepnote[0]? We use the cloud version (BYOC) at my previous company to replace self-hosted Jupyter notebooks, and it's pretty great. [0] https://github.com/deepnote/deepnote [2ndorderthought] This is one of those product areas I would call high-risk without a human in the loop. So I am glad you kept a person in the loop. It's really easy to lose tons of money making decisions based on bad statistics or models. Anyone remember how much money zillow lost because of automatic time series models? I do have concerns about the workflow. Data people aren't usually the best programmers. Models hallucinate and make mistakes sometimes subtle sometimes not. Can you think of a way to prevent data scientists from having to be expert code reviewers? I feel like taking away the code gives them the chance to find and fix mistakes in their reasoning but I have no evidence for that. [amirathi] Really cool. If somebody doesn't want to adopt a new platform, take a look at open source Jupyter MCP Server[1]. Once integrated with Claude, it can execute code on the live notebook kernel. I just let Claude write notebooks, run top to bottom, debug & fix errors & only ping me when everything is working. [1] https://github.com/datalayer/jupyter-mcp-server [trymamboapp] "AI saves analysis as notebooks" is fighting the wrong fight ig. The reproducibility issue with notebooks isn't the format. it's out-of-order cell execution and silent kernel state llm generation makes that worse: the model has no memory of what state existed when it wrote cell 7, and neither does the user.

Web scraping -> entity resolution -> normalized model -> API serving layer pipeline

Data is fetched from a variety of sources: XML files from FTP server, public JSON API, web scraping HTML pages, downloading PDF pages that need OCR, ... These sources contain data about private companies and their shareholders. Entities need to be resolved: link two address observations if they are the same, link two people observations if they are the same, ... This needs to be brought togheter into one combined model. This is followed by a very fast serving layer to power my own API that will be directly consumed by users, app and mcp server. There is an initial load of about 10 million company and people rows, as well as 50 million PDF pages that need OCR. Every day about 10k elements are added. Currently I'm doing this in PostgreSQL hosted on Railway, with DuckDB to perform the entity resolution. I have 260 GB of data in total. I have a cron job for each source. These are the schemas: raw (separate schema for each source), xref (entity resolution), core (normalized) and mart (serving layer). I have 1 mono repo with all of the code, most of it is Typescript with Bun. My problem is that it has become hard to manage. Things feel a bit duck taped as I have little observability. I don't have a clear overview of the data pipeline. Additionally, doing intial loads can take many hours. I was thinking Databricks could be a unified data platform from which I can manage this. One thing I'm not sure about is how to manage the scraping as I don't think Databricks is really built for this. Anyone that had to work on a similar problem. How would you solve this?

The real gap isn't connecting Claude to Databricks, it's the 3,000 tokens it costs every time you do

Posted a days ago asking if manually copying Databricks schemas into Claude was a real pain point. Thread here: [Old post](https://www.reddit.com/r/databricks/comments/1srypxz/im_building_an_opensource_tool_that_gives_claude/) The community was right to push back. ai-dev-kit and the managed MCP already solve the connection problem. I was building something redundant. But digging into both tools after those comments, I found something nobody mentioned: **Every existing tool dumps raw JSON back to Claude.** This is what ai-dev-kit returns for a single table schema: json { "table_name": "orders", "columns": [ {"name": "order_id", "type": "LongType", "nullable": false, "metadata": {}, "comment": null}, {"name": "customer_id", "type": "LongType", "nullable": true, "metadata": {}, "comment": null}, {"name": "order_date", "type": "DateType", "nullable": true, "metadata": {}, "comment": null}, {"name": "amount", "type": "DoubleType", "nullable": true, "metadata": {}, "comment": null} ], "partition_columns": ["order_date"], "storage_location": "dbfs:/user/hive/warehouse/...", "table_type": "DELTA" } \~800 tokens. For one table. Two tables + sample rows in a real session = **3,000+ tokens just for context**, before Claude writes a single line of code. If you're iterating — write, fix, optimize, test — that cost repeats every message. This is what the same schema looks like after compression: orders: order_id!bigint customer_id bigint order_date*date amount dbl status str **15 tokens. Same information Claude needs to write correct PySpark.** `!` = primary key. `*` = partition key. Types shortened. Storage paths, nullability metadata, comments — all stripped. Claude never uses any of that for code generation anyway. **What I'm thinking of building:** A thin middleware layer. Not a new MCP server — just a compressor that sits on top of whatever you already use (ai-dev-kit, managed MCP, anything). Intercepts the raw schema response, strips the noise, returns the compressed format. No new auth. No YAML config. No PAT tokens. You keep your existing setup. This just makes each tool call 84% cheaper in tokens. **One honest question before I build it:** Does token bloat from schema fetches actually affect you day to day? Or are you on an API/enterprise plan where token cost isn't something you think about? If most people here are on enterprise plans where this doesn't register, I should know that now rather than after building it.