Databricks News: watermark-based incremental ingestion, MCP in AI gateway, Genie, Vector Search
Summary
Databricks now offers watermark-based incremental ingestion from SQL databases without change data feed, allowing for efficient data updates and soft deletion handling. The AI Gateway supports custom MCP servers, enabling integration with external APIs like GitHub for enhanced AI application development.
Summary generated by brickster.ai from the video transcript.
More from databricks MVP Hubert Dudek
NewsDatabricks News: ZeroOps, DABs, Indexes, Genie, sandboxes, migration from PowerBI, secrets
Zero Ops automatically detects errors in jobs and data quality with lineage analysis and proposes code fixes, while DABs now default to direct mode instead of Terraform with automatic state migration. Full-text search indexes deliver 400x faster queries on billion-row tables, Genie automatically converts PowerBI dashboards to Databricks metric views, and Unity Catalog secrets support granular read and reference-only permissions.
NewsDatabricks News: RT Lakehouse (Reyden), Lakebase, TTL
This video highlights recent Databricks updates, including the beta release of the high-performance "Raiden" real-time lakehouse engine and new lakeflow connectors. It also demonstrates administrative changes to user groups, new time data types, predictive optimization TTL deletes, user home volumes, and advanced search capabilities in Lakebase.
