43 Lakehouse Federation or Query Federation in Databricks | Query External Database |Foreign Catalog
Description
Lakehouse Federation in Databricks | Query Federation in Databricks | Query External Data Source in Databricks Video explains - What is Lakehouse Federation in Databricks? What is Query Federation in Databricks? How to setup Lakehouse Federation in Databricks? How to query external database in Databricks? What are external connections in Databricks? What are Foreign Catalogs in Databricks? How Lakehouse Federation is different than JDBC connections in Databricks? Chapters 00:00 - Introduction 00:12 - What is Lakehouse Federation or Query Federation in Databricks? 01:47 - Different types of Catalogs in Unity Catalog 02:10 - How to create External Connections in Databricks/Create Lakehouse Federation 07:53 - How to create Foreign Catalog in Databricks? 09:13 - Lakehouse Federation using SQL Queries Databricks Website: www.databricks.com Lakehouse Federation - https://learn.microsoft.com/en-us/azure/databricks/query-federation/ The series provides a step-by-step guide to learning Databricks, a popular unified Data Intelligence Platform. New video in every 3 days ❤️ Follow Subham Khandelwal on LinkedIn and Don't forget to Share - https:// www.linkedin.com/in/subhamkharwal/ Disc…
Description from YouTube. Full content on the video page.
More from Ease With Data
Tutorials02 Machine Learning (ML) for Data Engineers | Basics of Machine Learning |Classification Regression
The video explains core machine learning concepts, including supervised, unsupervised, and reinforcement learning, along with the workflow for building and evaluating models. It details classification and regression models, their applications, and essential data preparation techniques like feature engineering and handling the curse of dimensionality.
News01 From AI to Agents| Fundamentals of AI | ML | DL | LLM & GenAI | How LLMs work | What are Agents
The video explains the fundamental concepts of AI, ML, DL, LLMs, and GenAI, illustrating their hierarchical relationship as subsets of each other. It also defines what models are (mathematical formulas trained on data) and how agents combine LLMs with tools and optional memory to perform autonomous tasks.
NewsGenAI - For Data Engineers Agenda & Introduction | LLM & Agentic AI | LangChain & LangGraph | Claude
This video introduces a new course, "GenAI for Data Engineers," designed to teach data engineers how to leverage generative AI, LLMs, and agentic AI. The course covers basics of LLMs, building agents with LangChain and LangGraph, using Cloud Code, and applying agentic AI within Databricks and data engineering workflows.
Tutorials54 Zerobus Ingest Lakeflow Standard Connector | Ingest Streaming data directly into Delta Table
The video demonstrates how to use Databricks Zero Bus Ingest, a push-based API, to directly stream various data types like IoT, event, and telemetry data into Unity Catalog Delta tables. It highlights Zero Bus Ingest's ability to simplify streaming ingestion by eliminating the need for intermediate message buses and managing their infrastructure.
Tutorials53 Lakeflow Connect SQL Server Managed Connector | Ingest Data using Databricks native connectors
The video demonstrates how to ingest data from SQL Server into Databricks using Lakeflow Connect's managed connector, covering the setup of a SQL Server database, user permissions, and enabling change tracking/change data capture (CT/CDC). It then walks through configuring the Databricks connection, creating gateway and ingestion pipelines, and showcasing how SCD Type 2 changes are automatically managed.
Tutorials52 Lakeflow Spark Declarative Pipelines | New Pipeline Code Editor | AUTO CDC |External Target Sinks
Databricks' LakeFlow Spark Declarative Pipelines (SDP), formerly Delta Live Tables (DLT), offers a unified solution for data ingestion, transformation, and orchestration, now open-sourced with Apache Spark 4.1. The video demonstrates using the new pipeline code editor to build SDPs in Python and SQL, showcasing features like auto CDC (formerly apply changes) and external target sinks.