Anthropic
Recent items mentioning Anthropic across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Databricks has released official guidance on using Claude alongside Genie and ChatGPT to study for its new Context Engineer Associate certification 7, while developers are deploying Claude Code to audit platform spend 1. Furthermore, Claude is being integrated into multi-agent frameworks like OmniGen for collaborative execution 4 and leveraged in knowledge distillation pipelines to train low-cost student models 6.
Generated daily from the 10 most recent items mentioning Anthropic. Click any [N] to jump to the source.
Databricks Cost Optimizer: Audit Spend with Codex or Claude Code
Managing AI Coding Costs at Scale
--- top comments --- [extr] I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fable 5 High + 5.6 Sol XHigh come up with a design + plan. Agree on the high level plan. (Usually this just comes down to choosing where the change belongs on the spectrum between minimal patch <-> full redesign) - Use Opus 5 or Sol Med to execute - Auto-fix bugs and CI until green + thermonuclear review skill x3. - Manual interrogation of change/nits - Come up with QA plan and have Codex Computer Use execute on it - Manually spot check the final result (usually a sizable diff, thousands of lines, complete feature E2E, etc) I probably spend like $80 a day at least but I produce the output of 3 or 4 2022 engineers and probably at better quality. So it's easily worth it. Would I save money by switching to GLM 5.2 and such...perhaps? IDK. At our scale it's not worth the time spent building the eval harness to actually understand the performance tradeoff. [lbriner] There are a surprising number of articles like this along the lines of, "we started using AI tools and ended up spending millions per year". On what planet do people start paying for things without keeping an eye on the costs and no-one notices until you have spent a crazy amount? I don't understand. You are either paying a fixed amount which you are happy about in-advance or you are PAYG in which case you would ballpark how much it costs. Otherwise it reads a bit like a fake problem, because it didn't really happen, you just foresaw it (as you should) and added a few guide rails. [sashank_1509] I suspect that when it comes to hard complex software products, you’re better off ignoring agents and doing “trad coding”. What you lose in short term speed you gain in manageable complex codebases. If you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term. Now of course, there are products that just involve lots of code but are not actually complex. This is generally the project with like hundreds or thousands of features but most of the features are separate and don’t actually interact in complex ways. Think a task management app with hundreds of features like calendar, email integration etc. there I think agents gives you more bang for the buck. Just my thought, using agents at work. [platinumrad] Careful. If you admit to using models that weren't trained by OpenAI or Anthropic then you might hauled in front of Congress: https://www.scmp.com/news/china/diplomacy/article/3362616/us... [dgellow] What I take from this is that models are already commoditized, and it’s pretty clear nobody has a moat: routing for the models, they can be swapped whenever new models are released, AI labs will have to continue to run on the treadmill non stop or be replaced. Long term I cannot imagine that business will be high margin. Routing for the harness, so anything that differentiate a provider vs another isn’t exposed to the user and isn’t too relevant. One more datapoint for the thesis that OpenAI and anthropic aren’t viable, sustainable businesses, and cannot justify their $1T valuation and the level of compute commitment (reminder that OpenAI committed to >$750B in infra spending for 2030)
Pi, Minimal and Performant
--- top comments --- [zhinit] Im looking forward to trying out Pi this week. It seems like anthropic just keeps adding more features that I don't really want and now is ignoring my claude.md and hooks. It feels like i'm adding more and more guardrails just to get to the place i used to be at before all the new features. Hopfully Pi helps
NewsClaude vs GPT: Live AI Debate with Omnigent
OmniGen is a new open-source project that orchestrates different AI agents like ChatGPT and Claude to collaborate and communicate within a single system. The architecture consists of a runner that sandboxes agents into uniform sessions and a server that manages control layers, policies, and chat history.
MLflow 3.15.0 introduces an MCP Registry for registering and sharing Model Context Protocol servers, enhances the Assistant with multi-provider LLM support and per-session token usage tracking, and enables proxy-less artifact transfers via presigned URLs to reduce server load and timeouts on large files. Additional improvements include sharable Runs table views, multi-modal image attachments for LLM judges to evaluate vision tasks, and numerous bug fixes across tracing, evaluation, gateway, and UI components.
TutorialsBuild Your First AI Agent + LLM Tool Calling Explained | Chapter 09
Agents operate through tool-calling loops where the LLM requests specific tool execution with arguments, code executes the tools, and observations return for the LLM to process. The video demonstrates building a Python agent using Grok API with two tools that execute sequentially: retrieving weather for a city and suggesting an activity based on temperature.
NewsHow Distillation Works? #claude #kimik3
Knowledge distillation trains a smaller student AI model by using prompts and reasoning outputs from a larger, more advanced teacher model. This process allows the cheaper student model to replicate the expensive teacher's advanced capabilities at a fraction of the computational cost and without accessing its source code.
The `databricks genie ask` command is now stable (promoted from experimental), enabling practitioners to ask natural-language questions about their data directly from the CLI. Multiple reliability and usability improvements ship, including auth profile validation timeouts, fixes for GCS-backed UC Volumes deletion, bundle validation for sql_warehouse configurations, opt-in spark_python_task file downloads in bundle generation, and instance_pools resource support in Declarative Automation Bundles.
The skills gap behind agentic AI — and how Databricks is closing it with a new context engineer certification and agent trainings
Databricks has launched an industry-first Context Engineer Associate certification and expanded its learning catalog with targeted courses to help practitioners build reliable agentic AI systems. The platform is also the first to publish official guidance on utilizing agents like ChatGPT, Genie, and Claude to help candidates effectively study for these certification exams.
TutorialsHow to Go From an Agent Prompt to a Live Databricks App | Databricks Apps Demo
The Databricks Developer Hub provides copyable agent prompts that developers can paste into coding assistants like Cursor to automatically build, configure, and deploy Databricks applications. Developers can also use specialized template prompts from the hub to add advanced features to their apps, such as integrating a natural language chat interface for querying workspace data.
Bundles now auto-migrate from Terraform to the direct engine when configured and the dry-run is clean, while a new `env:` section in scripts lets you declare environment variables that reference bundle, workspace, and variable values. SSH interactive sessions now properly resolve Python and pip to the environment interpreter instead of system libraries, and bundle init provides clearer error messages for unsupported template URL protocols.
TutorialsHow to Write Efficient Prompts | Prompt Injection Attacks | Chapter 07
This video teaches the fundamentals of prompt engineering, demonstrating how to craft efficient prompts using tasks, context, examples, formats, and personas. It also explains the risks of prompt injection attacks and demonstrates how to prevent them by instructing the model to treat input content strictly as data rather than instructions.
NewsGenie Code - Unleashing Agent Mode!
Databricks has rebranded its Databricks Assistant as Genie Code and introduced a new agent mode. This new feature allows the assistant to build entire systems, including notebooks, tests, jobs, and dashboards, from a single prompt rather than just suggesting code for individual cells.
Benchmarking coding agents on Databricks' multi-million line codebase
--- top comments --- [redmalang] We have an internal proxy (that I've been meaning to open source for ages) that routes all llm usage at our company, which allows us to see data in realtime. Its been fascinating how rapidly Pi has been adopted. Moreover since its pretty hackable, we've been able to automatically aggregate context from pi sessions, which has resulted in Pi efficacy being higher as more people use it, putting in place a interesting virtuous loop. I didn't expect this outcome: for whatever reason I assumed proprietary harnesses fine tuned to work with a companies' models would work better? ps/random aside: there is something slightly off about Pi's edit command, we are planning to investigate this further and patch this as we have quite a few session traces now.. [yodon] I wish they'd do a follow-on post drilling into the impact of the programming language on cost-per-task, specifically looking at cost to complete tasks in mainstream strongly typed languages (eg. C#, TypeScript) vs dynamic languages (eg. Python, JavaScript). Does the additional verbosity of the language help or hurt cost per task? [cpard] This was mostly because Sonnet 5 worked longer and read more to get there, consuming 1.9x more tokens. I have experienced similar behavior between opus and haiku when benchmarking Dara engineering tasks. The “cheaper” model takes many more turns to figure out the task and this is without taking into account other important factors. Another interesting behavior that I observed is that Haiku tended to cheat more maybe because it was having a harder time to find the root cause of the problem. Benchmarking and evaluation of agentic systems is very interesting and if there’s one thing that someone should keep from the Databricks post is how important is for everyone to build and run their own. [anentropic] > the results showed clear clustering of the models and harnesses into 3 capability tiers pretty sure the only thing making that 'clear' is the coloured stripes, if you took that away it'd look like two tiers good result for GLM 5.2 though and Sonnet 5 seems like a waste of time [HarHarVeryFunny] Wow! It's great to see a large-scale real-world benchmark from a user of these tools, as opposed to the the benchmaxxed results from the vendors themselves. Also great to see different harnesses being tested, with considerably different results. Definitely a few surprises here: 1) GLM 5.2 using Pi performs identically in terms of pass rate (~87.5%) to Opus 4.8 high using Claude Code, but significantly cheaper ($1.25 per task vs $2) 2) Absolute best pass rate (90%) was from Opus 4.8 x-high using Pi, beating out Opus 4.8 using Claude Code 3) Pareto frontier performance from any of the models (Opus 4.8, GPT 5.5, GLM 2.5) was using Pi rather than native harnesses Apparently Pi used 3x less context than Claude Code, and one takeaway is to use Pi regardless of what model you are using. The other takeaway is that in real-world performance GLM 5.2 is the equal of Opus 4.8 unless you run Opus 4.8 on x-high in which case you can eke out a 2.5% increase in pass rate at the expense of doubling your cost over GLM 5.2
Show HN: Dex – Cost-aware analytics engineering skills for agents
Hi I’m Marco, co-founder of Exmergo. Me and my team created Dex to help Analytics Engineers do real work with Claude Code (and any other coding agent). We’ve found that Data and Analytics teams are stuck between a rock and a hard place cost-wise: - On one side, they are using some of the most expensive consumption-billed software on the planet (Snowflake, Databricks etc.). - On the other side, Anthropic and Open AI want you to tokenmax (and with data analytics it’s very easy to burn your context window). So we created Dex, our open source skills plugin (Apache-2.0), to solve both of these problems: - Dex forces the agent to use a cost guard when performing exploration queries and transformations. - Dex uses a set of tooling that makes it hard to get burned when transforming data. It achieves these things with tight control scripts that the SKILL.md files are pointed towards when using /dex:explore, /dex:transform and /dex:maintain. Bonus: Dex makes agents really good data exploration, building sql and dbt models and detecting drift. 76% performance on ade-bench with Claude Sonnet 5 (and, per our measures, 2.5x cheaper than Fable 5). Install on any agent with this command in your terminal: npx skills add exmergo/dex Install on Claude Code running these commands (separately): /plugin marketplace add exmergo/exmergo-agent-plugins /plugin install dex@exmergo If you want to see more visual examples you can go through the README or browse here: https://www.exmergo.com/dex Let me know if this helps your analytics workflow and makes your agents more cost-aware.
Contextual Policies in Omnigent: Using session state to better govern AI agents
Omnigent now offers contextual policies for AI agents, enabling session state tracking to evaluate whether subsequent actions should proceed. These policies, applicable across any agent Omnigent wraps, allow for more powerful governance like per-session spending caps or risk-accumulating guardrails.
SSH connect adds `--base-environment` for custom base environments, and aitools install now uses plugins instead of raw skills. Bundle deployments fix drift on model serving endpoints and failed migrations on permissioned resources.
ReleasesIntroducing Omnigent: The Ultimate Meta-Harness for AI Agents
Omnigent is a new open-source meta-harness for AI agents that provides a unified interface for composition, control, and collaboration across multiple models and agent workflows. It enables stateful, data-centric policies for guardrails and allows real-time sharing and steering of live agent sessions with teammates.
NewsAI Stack Explained in 3 Layers (LLM, Agent Harness, Omnigent)
The AI stack now includes a third layer, the meta harness, which sits above individual agent harnesses. This meta harness, exemplified by Databricks' open-sourced Omnigent, allows for routing queries to appropriate agents and orchestrating tasks across multiple agents, enabling seamless interaction and context sharing between them.
TutorialsHow Transformers Work - Attention Explained Step by Step | Chapter 06
The video explains the Transformer architecture, detailing how it processes text input through tokenization, embedding, and a stack of Transformer blocks to generate the next token. It breaks down the attention mechanism, multi-head attention, and feed-forward layers within a Transformer block, highlighting the differences between encoders and decoders.
MLflow 3.14.0 adds one-command agent setup with Databricks support and durable low-latency Claude Code tracing, Review Queues for trace annotation and feedback collection, and @mlflow.test pytest markers for regression testing. Default model serialization formats change for sklearn to skops, PyTorch to pt2, and LightGBM to skops.
Databricks announces 2026 global partner awards
Databricks announced its 2026 global partner awards, recognizing over 60 Consulting and System Integrator and ISV partners including Accenture, Deloitte, Anthropic, and NVIDIA. This year's awards emphasize AI transformation, lakehouse modernization, Unity Catalog governance, and agentic AI at enterprise scale.
Introducing Omnigent: A Meta-Harness to Combine, Control and Share Your Agents
Omnigent, an open source meta-harness, is now available to combine, control, and share your AI agents across various models and interfaces. It enables building agent teams, controlling them with policies, and sharing live sessions with teammates.
How Ecolab rebuilt retail intelligence on Databricks and Anthropic Claude
Ecolab rebuilt retail intelligence on Databricks and Anthropic Claude, converting 700-page FDA manuals into real-time answers for frontline staff using Foundation Model APIs and cutting compliance report compilation from two weeks to under two minutes. The solution, a native Databricks App with Lakebase Postgres and Unity Catalog, unifies nine siloed data sources and employs a multi-agent orchestration framework with Judge LLMs and MLflow tracing for personalized, continuously refined intelligence.
Claude Fable 5 is now available on Databricks, fully governed through Unity AI Gateway
Claude Fable 5 is now available on Databricks, accessible through Unity AI Gateway for centralized governance, cost controls, and observability. This Anthropic model offers state-of-the-art performance across enterprise workflow automation, agentic search, data reasoning, and multimodal document understanding.
NewsAnthropic's SpaceX Deal, ClawPilot, and Databricks Agent-centric Cert | AI Newsround - May 2026
Anthropic signed a deal with SpaceX for AI supercomputing infrastructure, signaling the importance of compute supply in AI development. Google and Microsoft launched personal AI agents, Gemini Spark and Microsoft Scout, emphasizing ecosystem integration, trust, and governance.
The Databricks Python SDK now supports the Vercel AI_AGENT environment variable as a secondary fallback for AI agent detection in User-Agent headers. Unrecognized agent values are now sanitized and passed through up to 64 characters instead of defaulting to agent/unknown.
Beyond parsing X12: Closing the gap for revenue cycle workflows in healthcare
Healthcare billers now have an operational workbench built on Unity Catalog gold views, providing a purpose-built UI with a denials queue, remittance drawer, and timely-filing age alerts directly on their fully parsed 835/834/837 EDI data. This solution integrates GenAI via Databricks Foundation Model APIs to auto-draft appeal letters, moving billers beyond manual spreadsheet and SQL work to review and approve instead of writing from scratch.
MLflow 3.13.0 introduces Role-Based Access Control with Admin UI, automatic trace archival to S3, and one-click observability for Claude Code and other coding agents. Breaking changes include a redesigned permission system (legacy APIs removed), MLServer removal from pyfunc serving, and requirement for MLFLOW_ALLOW_FILE_STORE=true flag for local file-based stores.
The SDK now supports more granular AI agent detection in User-Agent headers and passes unrecognized values as-is. Several API changes introduce new fields for dashboards, apps, ML materialized features, and synced table statuses, along with a `Revert` method for Lakeview dashboards.
Route Claude Code Through MLflow AI Gateway
MLflow AI Gateway now supports routing Claude Code, providing full observability, budget controls, and guardrails for all your coding agent sessions. This integration requires no changes to your existing Claude Code usage.
MLflow 3.13.0rc0 completely overhauls Role-Based Access Control with unified permission APIs and a new Admin UI, and integrates Claude Code, OpenAI, Ollama, and OpenClaw as native assistant providers in the AI Gateway. The release adds trace archival with seamless retrieval, GenAI agent stress-testing, Kubernetes Helm chart support, and database replica routing for horizontal scaling.
NewsEnhancing your Skills with Databricks Genie Code
Databricks Genie Code is an agentic coding system that allows users to build custom "skills" using markdown files, enabling it to generate code and perform tasks according to specific in-house standards and conventions. These skills provide context-on-demand, ensuring repeatable and consistent output for various engineering tasks like schema documentation or metric view creation.
NewsDatabricks Genie, Unity AI Gateway, Project Glasswing, and Model Mania | AI Newsround - April 2026
Databricks Genie is now the business user home screen for Databricks, offering a unified chat interface, external knowledge store connections, and a mobile app. The Unity AI Gateway, integrated with Unity Catalog, provides comprehensive governance for agentic AI, including permissions, auditing, and policy controls for models and tools.
MLflow 3.12.0 adds multimodal tracing with artifact attachments supporting rich rendering, extends coding agent tracing to Codex/Gemini/Qwen platforms, and introduces gateway guardrails for input/output safety. Trace table pagination improves performance, third-party scorer registration enables custom evaluations, and new provider support expands AI Gateway integration options.
MLflow 3.12.0rc0 adds automatic tracing for multiple AI coding assistants (Claude Code, Codex, Qwen Code, Gemini CLI, OpenClaw) through installable TypeScript plugins, and introduces guardrails for production agent safety in the AI Gateway. The release also supports multimodal trace attachments for images, audio, and files with UI visualization, plus a new mlflow.diffusers flavor for diffusion models with LoRA adapters.
The SDK now automatically detects AI coding agents and appends agent information to HTTP request headers, while also removing the unused `experimentalIsUnifiedHost` field from `DatabricksConfig`. A bug fix addresses `X-Databricks-Org-Id` header issues for `SharesExtImpl.list()` on SPOG hosts, and several API method paths have changed, which are breaking changes.
WorkspaceExt upload/download and SharesExt list now include the X-Databricks-Org-Id header for SPOG host compatibility. WorkspaceClient.get_workspace_id avoids an API call when the workspace ID is already known, fixing a SPOG host failure.
NewsGenAI - For Data Engineers Agenda & Introduction | LLM & Agentic AI | LangChain & LangGraph | Claude
This video introduces a new course, "GenAI for Data Engineers," designed to teach data engineers how to leverage generative AI, LLMs, and agentic AI. The course covers basics of LLMs, building agents with LangChain and LangGraph, using Cloud Code, and applying agentic AI within Databricks and data engineering workflows.
This release introduces new workspace-level services for supervisor agents and Unity Catalog secrets, along with an update method for tokens. Several existing API methods for data classification, environments, knowledge assistants, Postgres, and warehouses have breaking changes due to path modifications.
This release adds Azure MSI authentication support and improves `.databrickscfg` default profile resolution. It also fixes issues with non-JSON error responses and Databricks CLI token scope mismatches, alongside several API additions and two breaking changes.
MLflow 3.11.1 introduces AI-powered issue detection in traces, AI Gateway budget alerts and spending controls, trace graph visualization, native Databricks gateway provider, and pickle-free model serialization. TypeScript SDK packages are now @mlflow-scoped and LiteLLM is no longer required for GenAI evaluation.
Testing and Refining Claude Code Skills with MLflow
MLflow tracing and LLM judges can now test Claude Code skills. This enables a self-improvement loop where Claude Code refines its own abilities.
TutorialsDatabricks AI Dev Kit Demo - Install, DataGen, SDP, Dashboard
The video demonstrates installing the Databricks AI Dev Kit on a Mac, then uses it to generate synthetic data, create serverless Spark declarative pipelines for a medallion architecture, and build a Databricks dashboard based on the generated data. It highlights how the AI Dev Kit leverages skills and an MCP server to automate these development tasks.
The SDK now automatically detects AI coding agents and appends `agent/<name>` to HTTP request headers. Two new `DisableGovTagCreation` fields were added to `RestrictWorkspaceAdminsMessage` in both `settings` and `settingsv2`.
The SDK now automatically detects AI coding agents and appends agent information to HTTP request headers. Two new `disable_gov_tag_creation` fields were added to restrict workspace admin settings.
This release introduces AI-powered issue identification for agent traces, budget alerts and limits for AI Gateway spending, and an interactive graph view for trace hierarchies. It also includes native OpenTelemetry GenAI convention support, Opencode tracing integration, UV package manager support, and pickle-free model serialization options for enhanced security.
NewsOpenClaw, Databricks Agentic Data Monitoring & more! | AI Newsround - February 2026 | Advancing AI
The video discusses OpenClaw, an open-source framework for AI agents, and Databricks' new agentic data quality monitoring solution. It also introduces Advancing Analytics' Lake Forge and Pantheon, a framework and AI layer for developing scalable Lake Flow pipelines, and highlights new model releases from Anthropic, Google, and OpenAI.
MLflow 3.10.0 introduces multi-workspace support for organizing experiments and models, alongside new GenAI features like multi-turn evaluation, LLM cost tracking, and AI Gateway usage analytics. The UI has been redesigned for improved navigation, and in-UI trace evaluation is now available.
ReleasesIntroducing Databricks AI Dev Kit - Skills, MCP server, Builder App
The Databricks AI Dev Kit provides agent skills, an MCP server, and a Builder App to enhance AI-driven development on Databricks. It allows users to integrate AI coding tools with Databricks best practices, extending LLM capabilities through specialized functions and offering a chat-based interface for building applications.
5 Tips to Get More Out of Your Claude Code with MLflow
MLflow now offers an MCP server, CLIs, and Skills to extend Claude Code, enabling you to trace tokens and monitor tool usage. These five tips will help you transform your Claude coding agent into a transparent and controllable workflow.
v.3.9.0
MLflow 3.9.0 introduces an in-product MLflow Assistant chatbot and a Trace Overview Dashboard for GenAI experiments, enhancing debugging and performance insights. The AI Gateway is revamped for direct tracking server integration, alongside new LLM judge features for online monitoring and custom prompt building.
MLflow 3.9.0rc0 introduces an in-product AI Assistant for debugging and a new Trace Overview Dashboard for GenAI experiments. The AI Gateway is now integrated into the tracking server, and users can configure LLM judges for online monitoring and build custom judges directly in the UI.
NewsClaude Code: 5 Essentials for Data Engineering
The video introduces five essential concepts for using Claude Code in data engineering: the cloud.mmd file for core project information, skills for packaging expertise, commands for predefined prompts, sub-agents for focused tasks, and Model Context Protocol (MCP) for standardized tool interaction. These components help manage context and memory for effective AI-enhanced development.
TutorialsDatabricks + Cursor IDE: Step-by-Step AI Coding Tutorial
The video demonstrates using Cursor IDE for AI-enhanced Databricks development, focusing on setting up Databricks Connect and leveraging Cursor rules and context for efficient code generation and testing. It shows how to structure projects, write Python and PySpark code, and create unit tests, highlighting the importance of providing clear instructions to the AI agent.
EventsThe Future of AI Agents with Dario Amodei, Co-founder and CEO, Anthropic at Data + AI Summit
Unity Catalog AI 0.2.0
Unity Catalog AI 0.2.0 adds Gemini and LiteLLM integrations for using UC functions as AI tools and introduces function wrapping APIs with dependency management support. AutoGen integration now requires version 0.4.x and Databricks client now only supports serverless endpoints.
Unity Catalog AI 0.1.0
The initial release of Unity Catalog AI introduces the unitycatalog-ai client package, enabling Databricks practitioners to manage and execute Unity Catalog functions as GenAI tools. This release also delivers dedicated integration packages to seamlessly use these functions within LangChain, LlamaIndex, OpenAI, Anthropic, CrewAI, and AutoGen.