v4.0.0rc1·Delta Lake 4.0.0 Preview
Summary
Delta Lake 4.0.0rc1 introduces support for Spark Connect, Type Widening, the Variant data type, and Coordinated Commits for flexible multi-cloud writes. It also includes fixes for liquid clustering, improved CDF query filter pushdown, and various performance enhancements.
Summary generated by brickster.ai. For the full changelog and any code/binary attachments, follow the GitHub link above.
More from delta-io/delta
Delta Lake 4.2.0
This release enhances Unity Catalog managed tables with support for REPLACE TABLE, RTAS, Dynamic Partition Overwrite, and improved streaming read options like `startingTimestamp` and `skipChangeCommits`. It also introduces general availability for Variant columns and adds support for Geospatial and Collations table features, alongside fixes for various data skipping and DML issues.
Delta Lake 4.1.0
Delta Lake 4.1.0 enhances Unity Catalog integration with improved support for catalog-managed tables, including atomic CTAS and conflict-free feature enablement for Deletion Vectors and Column Mapping. It also introduces a new Spark V2 connector based on Delta Kernel for streaming reads and server-side planning capabilities.
Delta Lake 4.0.1
The "managed table" feature is renamed to `catalogManaged` (breaking change) and now supports creating Unity Catalog-managed Delta tables with properties sent to the UC server. Unity Catalog OAuth authentication is now supported via catalog-scoped `auth.*` configurations, automatically acquiring and refreshing tokens.
Delta Lake 4.0.0
Delta Lake 4.0.0 introduces support for the Variant data type, enabling flexible semi-structured data storage, and enhances table management with instant feature dropping and improved type widening. This release also brings preview support for catalog-managed tables and Delta Connect for Spark Connect integration, while sunsetting Delta Standalone and its connectors in favor of Delta Kernel.