Skip to content
All topics

LLM

Recent items mentioning LLM across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.

31 recent items1 release18 news12 community threads
What's happening in LLMAI synthesis · updated Nov 16, 2026

Databricks workflows now support distilling 120B-parameter LLMs into 0.8B student models using Foundation Model APIs, Serverless GPUs, and MLflow evaluate to slash serving costs while preserving accuracy 1. This move toward cost-efficient, task-specific architectures is reinforced by the shrinking performance gap of open-weight models evaluated on custom benchmarks 2 alongside enterprise governance of external LLM traffic via Databricks Unity AI Gateway 3.

Generated daily from the 8 most recent items mentioning LLM. Click any [N] to jump to the source.

Databricks CommunityCommunity Articles

Governing External LLMs with Databricks Unity AI Gateway

001w ago
Reddit

No data? No problem: dltHub agent builds ingestion, model and metrics so Genie can answer from the data

hey folks, i'm one of the dltHub cofounders (the data engineer one) and we've been working on a meaning-driven pipeline generation workflow. A little context: We created an agent that can talk to Gemini and generate a pipeline from requirements, complete with ingestion, transformation and context (ontology, taxonomy, metrics etc for the transformed data). We basically apply the same concept as when unstructured data gets modeled into a graph, to raw structured data: we look at Pages in the catalog to get the meaning, then we generate a bespoke ingestion pipeline, model it into a canonical data model to work as a virtual knowledge graph, and add ontology, taxonomy, metrics to help a LLM reflect the same meaning as in the catalog Page. Then, everything gets deployed to Databricks - the pipeline, the data, the ontology+taxonomny+metrics where the meaning and data access can live in one place and be governed securely together. What makes this really special? - pipeline, data and meaning are automatically in sync, one source of truth&meaning (no more meaning drift, the system can be interrogated about nuance to) - the auto-modeling process (tech agnostic) creates ai-native gold datasets from raw api unnestings. It uses ontology based data access over canonical model, specifically for collapsing semantics into a high singnal/noise model for LLMs which creates the most accurate retrieval as benchmarked before by other data teams such as anthropic's who also describes canonical datasets data load tool what? in case you are unfamiliar with dlt (data load tool), it is the OSS library for loading data that has everything you need to make data ingestion reliable and easy (99.83% oss observed run success). Our AI transformation harness (not oss but source available and you can try it) includes this canonical modeling bit and a runtime agnostic transformation framework that lets you run your code locally on engines like duckdb the same as in cloud on large engines more info on our blog with video demo submitted by /u/Thinker_Assignment [link] [comments]

00Thinker_Assignment1w ago
Reddit

AI_DECIDE benchmark

I benchmarked AI_DECIDE against 20 LLMs. It was definitely the fastest. I was using a moderation test to approve or reject posts. https://medium.com/databrickscommunity/i-tested-databricks-ai-decide-against-20-llms-d37694fbf792 https://www.sunnydata.ai/blog/databricks-ai-decide-benchmark submitted by /u/hubert-dudek [link] [comments]

00hubert-dudek1w ago
Reddit

SAP to Databricks migration

What is the best approach to covert Code from SAP to pyspark or spark sql? Don’t want to use LLMs as the out put can be non deterministic based on the prompt given. I looked into lakebridge and looks like the source list doesn’t support SAP code. submitted by /u/BeeLive9842 [link] [comments]

00BeeLive98422w ago
Databricks CommunityCommunity Articles

I Built Tetris in One Databricks Function, Then Let ai_decide and LLMs Play It

002w ago
Databricks CommunityDatabricks TV

Custom MDM App on Databricks: Combining rules, ML, and LLMs for Entity Resolution

004w ago
Reddit

Databricks Micro Apps, App Spaces and Genie App Generator

Databricks launched App Spaces and Serverless Micro Apps at the end of last week (in Beta). Over the weekend, I tested migrating two of my production Databricks Apps to App Spaces. App Migration 1: Blocked by Zero Egress My Data Portfolio Project Creator required internet access to pull data stacks from live job postings. Then, the LLM needs internet access to research for open data sources to use. In standard Databricks Apps, this runs cleanly. In App Spaces, there is zero external internet egress including for LLMs. The app can reach internal workspace resources, but it cannot touch the outside web. If your app relies on third-party APIs, external databases, or web scraping, App Spaces is a non-starter until Databricks opens network egress. Workload 2: Success on Internal FinOps My client-facing DBU cost observability app reads workspace usage data and writes directly to Lakebase. Because it requires zero external network calls, the migration worked. Both the micro app compute and Lakebase scale to zero when idle. Cold starts take roughly 30 seconds (though in beta, you occasionally need a quick browser refresh once it spins up). For internal, low-frequency administrative tools, this turns a continuous monthly compute bill into pennies. (+ App Spaces, Genie App Generator, and Micro Apps are free while in beta) The next part is less about App Spaces and more just general best practice for Databricks Apps that I see people miss. Stop Using Delta Lake as an OLTP Database Databricks Apps are software applications, not batch analytics notebooks. If your app writes application state, session data, or row-level CRUD directly into analytical Delta tables, you need to rethink that design. For app transactions, use Lakebase (serverless Postgres which also scales to 0). Both are governed under UC, but Lakebase gives your app the low-latency transactional engine that application engineering actually requires. My Verdict on the Beta App Spaces solves the idle compute problem that has plagued Databricks Apps since launch. But until Databricks allows us to deploy directly from existing Git repos and opens external network egress, it remains limited to internal-only use cases. Curious on other peoples experience... How has it been for others? submitted by /u/OkImprovement7010 [link] [comments]

00OkImprovement70101mo ago
Reddit

The deployment decalogue

I am an ML engineer, but I come from a software engineering background: years of full-stack work, with heavy DevOps and Terraform experience. I come from teams that deploy to production five times a day with real continuous deployment. And honestly? Pressing the button still feels weird sometimes. Every engineer knows that feeling, no matter how good the safety net is. So I wrote down the list that settles it. Ten commandments, one flow, written with data scientists and ML teams in mind, but it works for batch jobs, realtime inference, and LLMs alike. Answer honestly, and if all ten are true, you can ship to production anytime, in any form or way. submitted by /u/SuspiciousPavement [link] [comments]

00SuspiciousPavement2mo ago
Reddit

How are you actually setting up AI/LLM evals in Databricks end-to-end? Looking for a step-by-step production workflow

We have a product where Databricks is our backend , and we’re now trying to properly evaluate and improve the quality of the AI-generated answers in our application. I’m looking for advice from people who have actually implemented LLM/GenAI evaluations in Databricks in production . I’d really appreciate an end-to-end, step-by-step explanation of how you would set this up from scratch. Specifically, how would you approach: Define what a “good answer” means Accuracy / correctness Relevance Completeness Groundedness / faithfulness to our data Hallucination rate Citation/source correctness Following user instructions Response consistency Latency and cost Create a benchmark / golden dataset Should we manually create a set of representative user questions? How many questions are enough to start? Should each question have an expected answer? Should we store expected SQL/results, expected sources, or just an expected natural-language response? Where should this benchmark dataset live in Databricks? How do you keep it updated as the product evolves? Set up automated evaluations What Databricks/MLflow tools should we be using today? MLflow evaluation? LLM-as-a-judge? Custom scorers? Human evaluation? How do you combine these rather than relying on one score? Evaluate RAG / data-grounded answers Our AI answers questions based on enterprise data in Databricks. How do you separately evaluate: Retrieval quality Whether the correct tables/documents were selected Context relevance Groundedness Final answer correctness Whether the model invented something that wasn’t in the retrieved data Evaluate text-to-SQL / analytics questions If a user asks something like: “What was average building occupancy last month?” should we evaluate: Generated SQL Tables selected SQL execution result Final natural-language answer separately? What is the recommended architecture for this? Guardrails Where should guardrails sit in the architecture? For example: Prevent hallucinated numbers Prevent querying unauthorized tables Detect PII Prevent prompt injection Enforce tenant/user permissions Block unsupported questions Force answers to cite their source Return “I don’t know” when confidence is too low Should guardrails be part of evaluation, inference, or both? Production monitoring Once this is live, what should we log for every AI request? For example: User question → retrieved context → generated SQL/tool calls → query result → final response → model → prompt version → latency → tokens → cost → evaluation scores → user feedback Is that roughly the right model? Regression testing When we change: System prompt Model Retrieval strategy SQL generation logic Tools Temperature Data sources how do you automatically run the benchmark again and determine whether the new version is actually better? Do you set minimum score thresholds before allowing something to deploy? Human feedback How are people incorporating thumbs-up/down or analyst review into their evaluation datasets? Do production failures automatically become new benchmark cases? Making answers more precise This is ultimately my main goal. If our AI currently gives an answer that is “mostly correct,” what is the systematic process for figuring out why it isn’t fully correct? Is the best workflow something like: Production traces → identify failure → categorize failure → add to benchmark → improve retrieval/prompt/tool → run eval → compare against baseline → deploy → monitor Or is there a better approach? I’m especially interested in what the ideal Databricks-native architecture looks like: User Question → Agent / LLM → Retrieval / SQL / Tools → Databricks data → Response → MLflow tracing → Automated evaluators → Benchmark dataset → Regression testing → Production monitoring If you’ve implemented something like this, I’d love to know what you would build first, second, third, etc. Even a practical example like: Week 1: create benchmark Week 2: add tracing + scorers Week 3: add regression tests Week 4: add guardr […truncated]

00New_Championship39292mo ago
Databricks CommunityCommunity Articles

Solution Accelerator Series | Building Common Sense Product Recommendations With LLMs

004mo ago
Databricks CommunityAnnouncements

Solution Accelerator Series | Building a Chatbot With Large Language Models (LLMs)

005mo ago
Databricks CommunityAnnouncements

Solution Accelerator Series | Large Language Models (LLMs) for Customer Service Analytics

005mo ago

Get Tuesday's version of this

Tracking LLM? The Tuesday email carries what moved across the whole ecosystem, not just this topic. Free, one-click unsubscribe.

Read past issues first