v0.4.0·UnityCatalog 0.4.0
Summary
Unity Catalog 0.4.0 introduces AWS storage credentials and external locations enabling secure S3 access with automatic temporary credential vending, and makes CREATE TABLE AS SELECT atomic for Delta tables to prevent data corruption. Credential renewal defaults to enabled for long-running Spark jobs, and new DSPy integration allows UC AI functions to serve as tools in LLM agent workflows.
Summary generated by brickster.ai. For the full changelog and any code/binary attachments, follow the GitHub link above.
More from unitycatalog/unitycatalog
UnityCatalog 0.6.0
This release adds Apache Spark 4.2 support and introduces metric views and SQL views as governed catalog objects for semantic layers and query management. UC access tokens now expire after 24 hours by default, requiring periodic re-exchange instead of the previous indefinite validity.
UnityCatalog 0.5.1
0.5.1 improves credential caching in the Spark connector by keying credentials to resource scope instead of per-request identifiers, reducing latency for repeated access to the same UC tables while maintaining session isolation. The release also fixes permission GET endpoints to work correctly when server-side authorization is enabled, resolving HTTP 500 errors that occurred on secured deployments.
UnityCatalog 0.5.0
UC 0.5.0 introduces a dedicated UC Delta API for catalog-managed Delta table operations across Spark, Flink, Trino, DuckDB, and other engines, with standardized REST endpoints and server-side commit validation. The Spark connector now ships separate artifacts for Spark 4.0 and 4.1 and enables credential-scoped file systems by default, fixing out-of-memory issues in long-running sessions.
Unity Catalog AI 0.4.0
DatabricksFunctionClient now accepts an optional warehouse_id parameter to execute functions via the Statement Execution API when serverless compute is unavailable, while serverless remains the default execution path. The release requires Python 3.10 or later and fixes issues with Gemini toolkit, LangGraph integration, OSS client function creation, and dependency declarations.
UnityCatalog 0.4.1
Unity Catalog 0.4.1 adds atomic write guarantees for REPLACE TABLE AS SELECT and Dynamic Partition Overwrite operations on UC Managed Delta Tables, plus a credential-scoped file system to prevent out-of-memory errors in long-running Spark sessions. The release introduces VARIANT datatype support and fixes a critical JWT validation bypass that could allow user impersonation, requiring authorization-enabled deployments to add issuer and audience configuration before upgrading.
UnityCatalog 0.3.1
Unity Catalog 0.3.1 adds automatic credential renewal for S3, Azure, and GCS storage plus OAuth authentication support in the Spark connector to enable seamless long-running workloads without manual token refresh. The release also introduces experimental UC-managed Delta tables with server-coordinated commits and an improved Java client API with pluggable authentication and retry handling.