RAG
Recent items mentioning RAG across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
With newer reasoning models from OpenAI and DeepSeek exhibiting higher hallucination rates than their predecessors, enterprise deployments are pairing RAG with systematic evaluation frameworks to curb production risks 1. Databricks now supports this shift with a five-stage production RAG workflow emphasizing hybrid search and independent evaluation of retrieval precision versus generation faithfulness 4, providing the backbone for governed AI assistants to execute autonomous pipeline management 2.
Generated daily from the 5 most recent items mentioning RAG. Click any [N] to jump to the source.
What are AI Hallucinations?
What are AI hallucinations, why does it matter, and what can enterprises do about it? Newer reasoning models from OpenAI and DeepSeek are actually hallucinating more than their predecessors, not less, making detection and prevention a must for any production deployment. Enterprises can curb the risk with retrieval-augmented generation, domain-specific fine-tuning, systematic evaluation frameworks, and strong data governance.
What is an AI Assistant?
Enterprise AI assistants use large language models, retrieval-augmented generation, and agentic frameworks to execute workflows ranging from data analysis to autonomous pipeline management. Choosing the right tool requires evaluating data integration depth, governance controls, extensibility, and alignment with your team's existing workflows and skill levels.
NewsRAG Explained + Build a RAG App From Scratch in Python using LLM | Chapter 08
This video teaches the core concepts of retrieval-augmented generation and demonstrates how to build a complete RAG application from scratch in Python using a Groq language model. The tutorial covers a seven-step pipeline including document ingestion, token-based text chunking, vector embedding generation, in-memory storage, similarity search, prompt augmentation, and response generation.
End-to-End RAG Workflow: How Retrieval Augmented Generation Works
Databricks now offers a five-stage RAG workflow for connecting LLMs to external knowledge bases, enabling accurate, domain-specific answers without model retraining. Production RAG requires careful selection of embedding models, vector database indexing, chunking strategies, and hybrid search, with independent evaluation of retrieval precision and generation faithfulness.
Data Engineering for AI: A Practical Guide for Data Professionals
Data engineering for AI demands new skills and a shift from traditional BI to managing large-scale, unstructured, and real-time data pipelines for ML and generative AI. Master feature engineering, vector databases, RAG, and ethical data practices alongside automation, observability, and unified data architecture to build production-grade AI solutions.
I built a 54-minute hands-on RAG tutorial on Databricks — from PDF loading to retrieval and LLM answers
Hi Everyone I recently published a hands-on tutorial where I build a basic **RAG pipeline on Databricks** from scratch. The goal of the video is not just to use a high-level RAG framework, but to show what actually happens behind the scenes. In the video, I cover: * Loading PDF files inside Databricks * Extracting text from PDF pages * Splitting documents into chunks * Creating embeddings using Databricks embedding endpoints * Building a simple manual retrieval system using vector similarity * Creating prompts from retrieved chunks * Generating grounded answers using Databricks LLM endpoints * Using `databricks-langchain` for embeddings and chat models I intentionally kept the implementation simple so that beginners can understand the core mechanics of RAG before moving to more production-level tools like Vector Search, Unity Catalog, MLflow, etc. Here is the video: [https://youtu.be/7QY1iXPLgRg](https://youtu.be/7QY1iXPLgRg) Would love to hear feedback from people working with Databricks, RAG, LangChain, or enterprise GenAI systems. Also curious: for production RAG on Databricks, would you prefer starting with a simple manual implementation like this first, or directly using Mosaic AI Vector Search / Databricks Vector Search from the beginning?
NewsSponsored by: Firebolt | The Power of Low-latency Data for AI Apps
Firebolt delivers 10-millisecond query latency on Databricks Delta Lake, enabling interactive AI applications where SQL queries complete fast enough to keep users engaged in conversation. SQL-backed retrieval-augmented generation outperforms vector databases for aggregations, time-based filtering, and ACID compliance while scaling to thousands of concurrent queries.
EventsPatrick Wendell, Co-founder and VP of Engineering on Building Production-Quality AI Systems
Databricks provides the Mosaic AI platform, which enables companies to transform general-purpose language models into customized data-intelligent applications. The system leverages tools such as zero-code fine-tuning, the Unity Catalog for data governance, vector search for retrieval-augmented generation, and MLflow for tracing and evaluation.
EventsBuilding and Deploying GenAI Apps at Block with Jackie Brosamer, Head of AI, Data & Analytics
Block implements a scalable and flexible data and AI platform using Databricks to power external generative AI features like automated square menus and internal productivity tools. The architecture utilizes model federation, retrieval augmented generation, and MLflow governance to securely switch between proprietary and fine-tuned open source models while maintaining data security.
NewsLarge Language Models in Healthcare: Benchmarks, Applications, and Compliance
The video explores the current state of large language models in healthcare, focusing on benchmarks, application use cases, and accuracy gaps compared to traditional NLP methods. It demonstrates open-source optimization tools and a retrieval-augmented generation architecture designed for secure, scalable medical data processing.
TutorialsSponsored: AWS|Build Generative AI Solution on Open Source Databricks Dolly 2.0 on Amazon SageMaker
Get Tuesday's version of this
Tracking RAG? The Tuesday email carries what moved across the whole ecosystem, not just this topic. Free, one-click unsubscribe.