Latest releases from the Databricks ecosystem.

Delta Lake 4.2.0

This release enhances Unity Catalog managed tables with support for REPLACE TABLE, RTAS, Dynamic Partition Overwrite, and improved streaming read options like `startingTimestamp` and `skipChangeCommits`. It also introduces general availability for Variant columns and adds support for Geospatial and Collations table features, alongside fixes for various data skipping and DML issues.

Delta Lake 4.1.0

Delta Lake 4.1.0 enhances Unity Catalog integration with improved support for catalog-managed tables, including atomic CTAS and conflict-free feature enablement for Deletion Vectors and Column Mapping. It also introduces a new Spark V2 connector based on Delta Kernel for streaming reads and server-side planning capabilities.

Delta Lake 4.0.1

The "managed table" feature is renamed to `catalogManaged` (breaking change) and now supports creating Unity Catalog-managed Delta tables with properties sent to the UC server. Unity Catalog OAuth authentication is now supported via catalog-scoped `auth.*` configurations, automatically acquiring and refreshing tokens.

Delta Lake 4.0.0

Delta Lake 4.0.0 introduces support for the Variant data type, enabling flexible semi-structured data storage, and enhances table management with instant feature dropping and improved type widening. This release also brings preview support for catalog-managed tables and Delta Connect for Spark Connect integration, while sunsetting Delta Standalone and its connectors in favor of Delta Kernel.

Delta Lake 3.3.2

This release fixes an issue where stale checksum files were not cleaned up during Delta table metadata maintenance. It also includes a fix for Flink's mapping of Delta's BinaryType to improve type compatibility.

Delta Lake 3.3.1

This release fixes an issue allowing user-specified schema on read if consistent with the table schema. It also includes a kernel fix for handling non-uniform value types in map[string, string] within Delta commit files.

Delta Lake 3.3.0

Delta Lake 3.3.0 introduces Identity Columns, faster VACUUM LITE, and the ability to enable Row Tracking on existing tables for row-level lineage. It also allows enabling UniForm Iceberg on existing tables without data rewrite and supports Type Widening in Delta Kernel.

Delta Lake 3.2.1

This release primarily delivers bug fixes across Delta Spark, UniForm, Delta Sharing Spark, Delta Kernel, and Delta Storage components. Key improvements include fixing MERGE operation logging, supporting RESTORE on clustered tables, and enabling UniForm Iceberg on existing tables via ALTER TABLE.

Delta Lake 4.0.0 Preview

Delta Lake 4.0.0rc1 introduces support for Spark Connect, Type Widening, the Variant data type, and Coordinated Commits for flexible multi-cloud writes. It also includes fixes for liquid clustering, improved CDF query filter pushdown, and various performance enhancements.

Delta Lake 3.2.0

This release introduces Liquid clustering for incremental table optimization and preview support for Type Widening, allowing column type changes without data rewrites. It also adds preview support for Apache Hudi in Delta UniForm tables and improves VACUUM operations with inventory tables and writer protocol checks.