Anthropic

Claude Code + Databricks AI Dev Kit

Finding Databricks cached token usage (count and/or cost)

I was asked to do a comparison of token usage between Databricks and Azure AI Services. With Microsoft you get your list price (and discount price if you have an EA) and they break the costs into input, output, input cache, output cache, and sometimes many other SKUs. Databricks charges us what looks like a single price for a token. We can find the input and output count, but the only place we've found any cached token counts is in the gateway system table system.ai\_gateway.usage. (which accounts for only about 4% of our overall token volume) Why do we need this? Without it, we're betting on 40B Claude Opus 4-6 (for example) being at full input and output costs. That artificially makes Azure look more expensive, and we have Azure often between 75-95% cached usage. Currently our model gives us a $2.16/1M token cost for Databricks, and its $5.20/1M for Azure marketplace. At around 70% cache usage, the table will turn to Azure being cheaper. https://preview.redd.it/qx0t0xg15d1h1.png?width=1534&format=png&auto=webp&s=362fd144e41bb2170bc682d9f9441d1db5292606 I need to know at least the number count so I can get a percentage then do a fair comparison. Databricks needs a parts catalog like Azure. It seems they bury it on purpose. That or I just haven't found the right place to look. Thanks. BTW: of course Microsoft isn't perfect either. Claude models are marketplace driven, not in the FOCUS cost table, so we have to look in two separate places for model pricing. At least its there.

Modular structure for Databricks Apps (Streamlit)

Hey, I wanted to share something that's been bugging me for a while and get your take. The official Databricks Streamlit tutorial puts everything into a single **app.py.** Fine for a demo. But the moment a real internal app grows past \~500–600 lines, it stops being fun: * Two people on the team touch the same file → merge conflicts every PR. * Hard to write unit tests when UI, data access, and business logic live in one module. * Git diffs become unreadable, and code review suffers. * When I point Cursor/Claude at the repo, it has to re-read the whole monolith on every prompt. Context window and cost both balloon. So I refactored our internal template into something more boring and modular: app. py # entry point only, routing pages/ ├── home. py ├── analytics. py └── settings. py components/ # reusable UI bits services/ # SQL warehouse / UC / SDK calls assets/ ├── styles.css └── logo.png tests/ *This is my own repo, not a product. Sharing because the single-file pattern bit us hard, and I figured others might find it useful -* [*https://github.com/protmaks/databricks\_apps\_streamlit\_mod\_template*](https://github.com/protmaks/databricks_apps_streamlit_mod_template)

FMAPI Anthropic endpoint rejects requests with trailing assistant message — known limitation?

Ali Ghodsi from Databricks: AGI Is Here, Enterprise AI Is Not

Ali was a guest lecturer at Stanford's MS&E Economics of the AI Supercycle class and he shared some really interesting insights on how he sees the AI adoption trends and why he thinks that stock market is overreacting to Claude's capabilities in eating away top SaaS companies.

Claude code + AI Dev kit vs Genie code

Hi everyone, i am testing the limits for agentic engineering for databricks. Has anyone tried using claude code with AI dev kit and comparing it against Genie code? Specifically around these tasks 1) Data engineering 2) ML prototyping 3) AI/BI Dashboard building So far, claude code + AI dev kit seems to work really well. But from a conversation with claude it seems genie code out of the box is better already. Anyone have any experience?

Resident Solution Architect - Anxiety

Hi all, I was approached by a Databricks Recruiter this week regarding a RSA role in Europe. Now I’m proceeding to the Hiring Manager interview next week. I was told that the following steps in the process are: \- HM interview \- Technical Interview (Spark deep dive) \- Technical Interview (live coding Python & SQL) \- Architecture Interview \- Project Delivery Interview Honestly, I’m a bit flabbergasted looking at the process. It seems like a huge effort for both parties and honestly I have a hard time seeing how someone should prep themselves while being in my current job and not abandoning my family for a month. 😄 Furthermore, I get anxiety about having to do live coding. Like I literally have not written a single line of code this year due to AI. Like yeah I review, check, orchestrate, give instructions etc. but being tested on writing live code gives me the chills! I’m a Solution Architect at a consultancy, work with Databricks almost daily and have multiple certs. So the role matches my profile pretty well BUT I really feel intimidated. Is the process really as tough as it sounds? Does nobody use Claude at Databricks? How are the technical deep dives? Any advice on prep or in general? I would love to hear from you and your experience! Thanks!

Chat layer architecture: what should I use for external customers?

May be wrong but as far as I know, we can use multi tenant schema or RLS to do customer isolation. Then give specific read access to tables using customer specific service principles. But after this what should I use? What APIs of databricks? Claude mcp or genie mcp or what would be accessible so that customers can do natural language query on top of data. Is genie internally using claude and adding databricks context so that we can reliably skip claude mcp inside databricks engine?

Show HN: Mljar Studio – local AI data analyst that saves analysis as notebooks

Hi HN, I’ve been working on mljar-supervised (open-source AutoML for tabular data) for a few years. Recently I built a desktop app around it called MLJAR Studio. The idea is simple: you talk to your data in natural language, the AI generates Python code, executes it locally, and the whole conversation becomes a reproducible notebook (*.ipynb file). So instead of just chatting with data, you end up with something you can inspect, modify, and rerun. What MLJAR Studio does: - Sets up a local Python environment automatically, runs on Mac, Windows, and Linux - Installs missing packages during the conversation - Built-in AutoML for tabular data (classification, regression, multiclass) - Works with standard Python libraries (pandas, matplotlib, etc.) - Works with any data file: CSV, Excel, Stata, Parquet ... - Connects to PostgreSQL, MySQL, SQL Server, Snowflake, Databricks, and Supabase. For AI: use Ollama locally (zero data egress), bring your own OpenAI key, or use MLJAR AI add-on. I built this because I wanted something between Jupyter Notebook (flexible but manual) and AI tools that generate code but don’t preserve the workflow. Most tools I tried either hide too much or don’t give reproducible results and are cloud based Demos: - 60-second demo: https://youtu.be/BjxpZYRiY4c - Full 3-minute analysis: https://youtu.be/1DHMMxaNJxI Pricing is $199 one-time, with a 7-day trial. Curious if this is useful for others doing real data work, or if I’m solving my own problem here. Happy to answer questions. --- top comments --- [MSaiRam10] Notebooks as the output format is funny because notebooks are famously bad for reproducibility. Out of order execution, hidden state, etc. You're solving "chat isn't reproducible" with a format that also isn't really [hasyimibhar] How does this compare to open source Deepnote[0]? We use the cloud version (BYOC) at my previous company to replace self-hosted Jupyter notebooks, and it's pretty great. [0] https://github.com/deepnote/deepnote [2ndorderthought] This is one of those product areas I would call high-risk without a human in the loop. So I am glad you kept a person in the loop. It's really easy to lose tons of money making decisions based on bad statistics or models. Anyone remember how much money zillow lost because of automatic time series models? I do have concerns about the workflow. Data people aren't usually the best programmers. Models hallucinate and make mistakes sometimes subtle sometimes not. Can you think of a way to prevent data scientists from having to be expert code reviewers? I feel like taking away the code gives them the chance to find and fix mistakes in their reasoning but I have no evidence for that. [amirathi] Really cool. If somebody doesn't want to adopt a new platform, take a look at open source Jupyter MCP Server[1]. Once integrated with Claude, it can execute code on the live notebook kernel. I just let Claude write notebooks, run top to bottom, debug & fix errors & only ping me when everything is working. [1] https://github.com/datalayer/jupyter-mcp-server [trymamboapp] "AI saves analysis as notebooks" is fighting the wrong fight ig. The reproducibility issue with notebooks isn't the format. it's out-of-order cell execution and silent kernel state llm generation makes that worse: the model has no memory of what state existed when it wrote cell 7, and neither does the user.

Vibe coding on the Databricks free addition

Has anyone used Genie code on Databricks free addition? Have you faced any issues. Is it better to use something like Claude/ Cursor ( I have a subscription already) in combination with AI Dev Kit on the free addition to not hit the rate limits?

Genie Spaces - What do you think?

I've been having a lot of success using Claude Cowork with the Databricks AI Dev Kit to create Genie Spaces that have fully developed Knowledge Stores set up. It saves so much time! The industry I'm in means I need lots of the same space except for a specific facility with all of its own permissions and slightly different context. They are early days so we will see how the users respond, but so far they love them. With the new push to Genie as a whole, I can see us getting a lot of use out of the mobile app paired with Genie's capabilities. What are the thoughts here in the community on Genie Spaces and what do you use them for?