Live from the Lakehouse: Machine Learning, LLM, Delta Lake, and data engineering
Summary
Databricks experts discuss recent advancements in machine learning, large language models, Delta Lake data formats, and Unity Catalog governance. The speakers demonstrate how new tools like Delta kernel, universal format replication, and query federation improve data engineering workflows.
Summary generated by brickster.ai from the video transcript.
Topics
More from Databricks
TutorialsHow to Build Customer 360 Profiles with AI | CustomerLake Profile Agents Demo
CustomerLake profile agents uses AI to ingest new data sources and define identity resolution matching rules. The system normalizes data and launches runs to create unified customer profiles.
ReleasesIntroducing Agent Memory on Databricks
Databricks announced Agent Memory, a new tool for Agent Bricks that allows AI agents to store and recall past conversations and user data using lakebase architecture. By connecting session stores and memory stores through automated memory pipelines, agents can retain context across multiple sessions to improve task performance and reduce token usage.
TutorialsBuilding Production-ready AI Agents on Databricks | Agent Bricks Demo
Building production ready AI agents on Databricks involves a six step workflow spanning vector search indexing, agent as code, MLflow tracing, Unity Catalog registration, evaluation, and deployment with review apps. MLflow tracing and evaluation metrics help monitor agent execution steps, relevance, safety, and topic adherence in real time.
NewsShorts extracted from Parallel Coding Agents with Lakebase
Coding agents can use git worktrees and branch databases to work safely in parallel. This allows multiple agents to test features against isolated database branches without interfering with production or each other.
NewsComo a Arezzo&Co centralizou dados com a Databricks
Arezzo and Co utilized Databricks to centralize its data architecture, breaking down silos and improving data consistency across departments. This integration accelerated reporting times by over fifty percent, improved full-price sales, and provided actionable insights that drove strategic business growth.
NewsBuilding Trusted Analytics with Genie & Metric Views on Databricks | Genie Demo
The video demonstrates how to use Databricks Genie and metric views to analyze business data by automatically generating queries and insights from organizational knowledge. It also shows how Genie can connect to external tools like Google Drive and Gmail to retrieve additional information and schedule recurring reports.