Taking AUTO CDC to the next level: Solving the hardest real-world use cases
Summary
Spark Declarative Pipelines now supports Bitemporal AUTO CDC and Partial Updates, replacing hand-written MERGE logic to track business and system time independently and safely handle missing fields. These declarative change data capture capabilities are also expanding into open source Apache Spark 4.2 to deliver standardized, out-of-order processing to the broader ecosystem.
Summary generated by brickster.ai. For the full article, follow the source link above.
More from Databricks Blog
Your data, your storage, your rules: a 2026 guide to storing Unity Catalog managed tables
You can redirect where Unity Catalog managed tables land by using SET MANAGED LOCATION and moving existing data through external table conversion. Defining managed storage paths at the metastore, catalog, or schema level keeps you in control of your cloud storage to support compliance and accurate cost attribution.
How Databricks rolls out frontier models to 12,000 employees on Day 1
Databricks delivers Day 1 access to frontier AI models for all 12,000 employees to ensure cutting-edge capabilities are immediately available internally. See how the company prioritizes and operationalizes large-scale model rollouts across its entire workforce.
How Databricks rolls out frontier models to 12,000 employees on Day 1
Databricks makes workforce access to frontier AI capabilities a top priority by rolling out newly released models to 12,000 employees on Day 1. This organization-wide rollout demonstrates how the platform delivers immediate, day-one access to cutting-edge AI capabilities at scale.
Lakebase Search: State-of-the-art full text and vector search for Postgres
Lakebase Postgres now includes a built-in search engine, generally available on AWS and Azure, that enables native semantic, keyword, and hybrid search directly alongside operational data without separate ETL pipelines. Featuring a serverless architecture that scales to zero and decouples storage from compute, it delivers twice the throughput at a quarter of the cost of cloud Postgres with pgvector on 100-million-vector benchmarks.
Manufacturing data and AI: Connecting the product value chain
Databricks connects fragmented manufacturing systems across functions and plants by unifying data or querying it in place to answer cross-stage value chain questions. Governed semantics, natural-language analytics, and agentic applications empower teams to move from finding data to taking action without becoming data engineers.
How to roll out Genie One: A step-by-step enterprise playbook
Scaling Genie One from an initial demo to trusted enterprise adoption requires starting with a single, well-governed data domain. This step-by-step playbook outlines a phased plan to expand the data-smart AI coworker from a pilot team to org-wide adoption without losing control of your data.