Workflows
Recent items mentioning Workflows across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Databricks is now supporting Microsoft Outlook in Lakeflow Connect (Beta)
Azure Databricks has introduced a managed Microsoft Outlook connector for Lakeflow Connect, currently available in Beta, enabling organizations to ingest Outlook email data directly into Azure Databricks. With this new connector, teams can now integrate Outlook-based communication data into analytics, governance, automation, and AI workflows more efficiently. https://preview.redd.it/8r0xry5yz50h1.png?width=1402&format=png&auto=webp&s=78ae55a64c3df3d47b0edcb4ed6f246f421e7b08 Key capabilities currently supported: * Incremental ingestion * Unity Catalog governance * UI & API-based pipeline authoring * Databricks Workflows orchestration * Declarative Automation Bundles * Column selection/deselection Supported authentication: * OAuth M2M (Machine-to-Machine) Current Beta limitations: * SCD Type 2 support * Automated schema evolution * API-based row filtering * Multiple tables per pipeline (currently limited to 1) Since the connector is still in Beta, workspace admins must enable the feature from the Previews page before use. Nice to see Databricks continuing to expand Lakeflow Connect integrations across enterprise ecosystems. [Source Link](https://learn.microsoft.com/en-us/azure/databricks/ingestion/lakeflow-connect/outlook-overview)
LLM Vs AI: A Practical Guide to Differences, Use Cases, and Tools
LLMs are a subset of AI, and this guide clarifies their practical differences, use cases, and tools. Understand how LLMs fit into the broader AI landscape and what that means for your Databricks workflows.
Transitioning from ADF to Databricks Workflows: Best Practices in a Multi-Workspace (dev-prod)
TutorialsDatabricks End-To-End Project | Zero-To-Expert | Streaming, AI, Lakeflow, Unity Catalog, AI/BI
This video demonstrates building an end-to-end restaurant analytics platform on Databricks, covering streaming and batch data ingestion, AI-powered sentiment analysis, and dashboard creation. It teaches how to use Unity Catalog, Lake Flow Connect for CDC, Spark declarative pipelines for real-time data from Event Hub, and how to construct a medallion architecture with fact and dimension tables.
NewsFrom Days to Seconds — Reducing Query Times on Large Geospatial Datasets by 99%
This release adds a new CLI command and documentation for migrating Delta Live Tables pipelines to Unity Catalog, including options to include or exclude specific pipelines. It also introduces support for MSSQL and PostgreSQL databases in Hive Metastore Federation, allowing these external metastores to be federated to a Unity Catalog.
EventsEmbracing the Future of Data Engineering: The Serverless, Real-Time Lakehouse in Action
News
