v4.1.0·Delta Lake 4.1.0
Summary
Delta Lake 4.1.0 enhances Unity Catalog integration with improved support for catalog-managed tables, including atomic CTAS and conflict-free feature enablement for Deletion Vectors and Column Mapping. It also introduces a new Spark V2 connector based on Delta Kernel for streaming reads and server-side planning capabilities.
Summary generated by brickster.ai. For the full changelog and any code/binary attachments, follow the GitHub link above.
More from delta-io/delta
Delta Lake 4.2.0
This release enhances Unity Catalog managed tables with support for REPLACE TABLE, RTAS, Dynamic Partition Overwrite, and improved streaming read options like `startingTimestamp` and `skipChangeCommits`. It also introduces general availability for Variant columns and adds support for Geospatial and Collations table features, alongside fixes for various data skipping and DML issues.
Delta Lake 4.0.1
The "managed table" feature is renamed to `catalogManaged` (breaking change) and now supports creating Unity Catalog-managed Delta tables with properties sent to the UC server. Unity Catalog OAuth authentication is now supported via catalog-scoped `auth.*` configurations, automatically acquiring and refreshing tokens.
Delta Lake 4.0.0
Delta Lake 4.0.0 introduces support for the Variant data type, enabling flexible semi-structured data storage, and enhances table management with instant feature dropping and improved type widening. This release also brings preview support for catalog-managed tables and Delta Connect for Spark Connect integration, while sunsetting Delta Standalone and its connectors in favor of Delta Kernel.
Delta Lake 3.3.2
This release fixes an issue where stale checksum files were not cleaned up during Delta table metadata maintenance. It also includes a fix for Flink's mapping of Delta's BinaryType to improve type compatibility.
Delta Lake 3.3.1
This release fixes an issue allowing user-specified schema on read if consistent with the table schema. It also includes a kernel fix for handling non-uniform value types in map[string, string] within Delta commit files.
Delta Lake 3.3.0
Delta Lake 3.3.0 introduces Identity Columns, faster VACUUM LITE, and the ability to enable Row Tracking on existing tables for row-level lineage. It also allows enabling UniForm Iceberg on existing tables without data rewrite and supports Type Widening in Delta Kernel.