Summary
A small data engineering team migrated Wehkamp from a traditional data warehouse to a Databricks lakehouse architecture, improving data democratization while managing complexity. The speaker demonstrated how they implemented custom ingestion frameworks, pseudonymization, and Slack alerting to maintain data quality and lineage during the migration.
Summary generated by brickster.ai from the video transcript.
Topics
More from Databricks
TutorialsBuilding an Agentic Analytics Foundation on the Lakehouse
The video demonstrates how to build an agentic analytics foundation on the Databricks lakehouse platform to ensure AI agents are accurate, secure, and scalable. It explains how to organize data assets into business domains, define metric views, and use Unity Catalog and Unity Gateway for governance and guardrails.
NewsLG Uplus moves from static analytics to real-time AI | Data + AI Executive Series: Fast 5
LG Uplus uses Databricks to operationalize AI and address siloed data challenges across the telecom enterprise. The company integrates AI into core workflows to deliver automated, data-driven customer experiences.
NewsTrain and fine-tune models faster with AI Runtime
Databricks AI Runtime is a managed GPU experience for distributed training and fine-tuning of AI models at scale. The video demonstrates using AI Runtime to train and fine-tune a model and then analyze the run to improve model quality and efficiency.
TutorialsTutorial: Genie for Data Engineers, Data Scientists, and Everybody Else
This tutorial demonstrates how to use Databricks Genie to build a live web app and data pipeline from a 700-million-row IoT dataset. The process covers getting the data from the Databricks Marketplace, running exploratory data analysis, building pipelines and jobs with Genie Code, querying data on mobile, and receiving shared data locally.
TutorialsHow to Build Customer 360 Profiles with AI | CustomerLake Profile Agents Demo
CustomerLake profile agents uses AI to ingest new data sources and define identity resolution matching rules. The system normalizes data and launches runs to create unified customer profiles.
ReleasesIntroducing Agent Memory on Databricks
Databricks announced Agent Memory, a new tool for Agent Bricks that allows AI agents to store and recall past conversations and user data using lakebase architecture. By connecting session stores and memory stores through automated memory pipelines, agents can retain context across multiple sessions to improve task performance and reduce token usage.