Latest releases from the Databricks ecosystem.
New versions from the official SDKs, CLI and Asset Bundles, Terraform provider, Unity Catalog, MLflow, Delta, dbt-databricks, and more. Summarized for scanning.
Last week
11 releasesThe `resource_id` field in `bundledeployments.Operation` is no longer required, which is a breaking change. Token caching for OIDC and lazy `dbutils` initialization improve performance and prevent crashes on Spark Connect clusters.
The Databricks SDK for Go now includes a TypeOverrides field for database.SyncedTableSpec and postgres.SyncedTableSyncedTableSpec. This allows practitioners to specify type overrides for synced tables within their Go applications.
The `direct` deployment engine is now Generally Available and the default for new deployments, with an option to revert to Terraform. New commands `databricks quickstart` and `databricks version --check` are added, alongside fixes for authentication and bundle deployments.
1.12.1
This release exposes Databricks Jobs IDs in dbt run results and adds support for SPOG vanity URLs. It also fixes issues with streaming table diffs, column-level constraints, and managed Iceberg incremental models losing clustering.
This release adds an `AcceleratedSync` field to `database.SyncedTableSpec` and `postgres.SyncedTableSyncedTableSpec`. These changes enable configuration of accelerated sync for synced tables within Databricks.
This release adds new services for AI Search and Bundle Deployments, along with numerous fields across existing services for managing catalogs, connections, schemas, MLflow, pipelines, and vector search. It also includes a breaking change by removing the `bundle` package and its associated service.
This release introduces new services for AI Search and Bundle Deployments, along with numerous new fields across existing services like Catalog, ML, and Vector Search. The `bundle` package and its associated workspace service have been removed.
This release introduces new services for AI Search and Bundle Deployments, along with several new fields across various services like Catalog, ML, and Vector Search. It also includes breaking changes by removing the old Bundle package and its associated workspace-level service.
* Canonicalize Bearer tokenType in Authorization headers
The SDK now detects the AI_AGENT environment variable for user agent reporting and passes unrecognized agent values through. New explicit factory methods for token and offset pagination have been added to the Paginator class. A bug was fixed where Paginator would silently drop results from token-paginated responses that returned an empty page with a next_page_token, ensuring list methods now page until the token is absent.
The SDK now better detects AI agents by honoring the Vercel AI_AGENT environment variable and passing through unrecognized agent names in the User-Agent header. This allows more specific agent versions to be visible instead of being generalized to "agent/unknown".
Week of Jun 1
13 releasesThis release is a backport from the 0.32.x line, which will receive voluntary support for a period. Consult the full changelog for specific user-facing changes, fixes, or breaking changes.
* direct: Fix updating the apps after the Go SDK upgrade ([#5444](https://github.com/databricks/cli/pull/5444))
Release: v2.11.0 (#1902)
This release introduces Unity Catalog and Workspace filesystem explorers, enhancing data and file navigation directly within VS Code. It also adds support for SPOG host URLs.
The `experimental open` command now supports opening a wider range of Databricks resource types directly in the workspace. Databricks Bundles gain a new `--select` flag for `plan` and `deploy` to target specific resources, along with improved retry logic for transient HTTP errors and support for `purge_on_delete` for Lakebase projects.
The Databricks SDK for Java now correctly handles OAuth token exchanges for browser-based flows by making the client ID optional in `DatabricksOAuthTokenSource`. This fixes a `NullPointerException` when no client ID is present in the IdP JWT, allowing account-wide token federation.
Creating an external location with `enable_file_events = false` now correctly sends this setting, preventing the server from defaulting it to true. Previously, this field was silently dropped, leading to file events being enabled despite the configuration.
Databricks SDK for Go now includes `DeploymentMode` fields for bundle deployments and versions. It also adds `CollaborationPlatformConnectivity` and `EffectiveCollaborationPlatformConnectivity` fields to settings.
This release adds new `deploymentMode` fields to bundle deployment and version objects. It also introduces `collaborationPlatformConnectivity` and `effectiveCollaborationPlatformConnectivity` fields to the settings service.
This release adds new fields for deployment mode in Databricks Asset Bundles and for collaboration platform connectivity in account settings. These changes expose additional configuration and status information through the SDK.
MLflow 3.13.0 introduces a new Role-Based Access Control system with an Admin UI for self-hosted MLflow, alongside trace retention and auto-archival to object storage. Breaking changes include the overhaul of the permission system, removal of MLServer as a pyfunc serving backend, and changes to Claude autologging and local file store behavior.
The SDK now supports a discovery flow that lands users on the account selector and includes a new method for workspace-level token management. Additional fields were added to job deployments, pipeline deployments, and token-related settings for enhanced configuration and information.
Workspace-scoped API calls now use `X-Databricks-Workspace-Id` instead of `X-Databricks-Org-Id`, accepting classic numeric IDs or other formats. New fields were added across several services, including token management, job and pipeline deployments, and OBO token requests.
This release adds comprehensive CRUD operations for feature engineering streams and a method to update token management settings. It also introduces several new fields across Jobs, Pipelines, and Token services, while removing `catalog_id` and `synced_table_id` from Postgres service objects.
Week of May 25
6 releasesYou can now manage Git credentials for service principals and permissions for Agent Bricks resources. Key fixes include proper updates for metastore external access, reliable destruction of UC objects, and configurable timeouts for vector search index creation.
This release adds new methods for managing feature engineering streams and introduces a `Parameters` field for various jobs and pipelines API calls. It also includes breaking changes by removing `CatalogId` and `SyncedTableId` fields from specific PostgreSQL service statuses.
This release adds comprehensive CRUD operations for feature engineering streams and introduces a `parameters` field across several Jobs and Pipelines API objects. It also includes breaking changes by removing `catalogId` and `syncedTableId` fields from specific Postgres service status objects.
Bundle users now receive a suggestion to set `bundle.engine: direct` in `databricks.yml` when using direct-only resources with the Terraform engine. The CLI also adds support for managing `vector_search_indexes` as a bundle resource, exclusively with the direct engine.
The config-file loader now correctly sets the profile name to "DEFAULT" when using the legacy fallback, resolving an issue where the profile was empty despite loading default settings. This fix ensures consistent profile identification, particularly for tools like the Databricks CLI that rely on the profile name for caching.