Latest releases from the Databricks ecosystem.
New versions from the official SDKs, CLI and Asset Bundles, Terraform provider, Unity Catalog, MLflow, Delta, dbt-databricks, and more. Summarized for scanning.
Last week
11 releasesThe `resource_id` field in `bundledeployments.Operation` is no longer required, which is a breaking change. Token caching for OIDC and lazy `dbutils` initialization improve performance and prevent crashes on Spark Connect clusters.
The Databricks SDK for Go now includes a TypeOverrides field for database.SyncedTableSpec and postgres.SyncedTableSyncedTableSpec. This allows practitioners to specify type overrides for synced tables within their Go applications.
The `direct` deployment engine is now Generally Available and the default for new deployments, with an option to revert to Terraform. New commands `databricks quickstart` and `databricks version --check` are added, alongside fixes for authentication and bundle deployments.
1.12.1
This release exposes Databricks Jobs IDs in dbt run results and adds support for SPOG vanity URLs. It also fixes issues with streaming table diffs, column-level constraints, and managed Iceberg incremental models losing clustering.
This release adds an `AcceleratedSync` field to `database.SyncedTableSpec` and `postgres.SyncedTableSyncedTableSpec`. These changes enable configuration of accelerated sync for synced tables within Databricks.
This release adds new services for AI Search and Bundle Deployments, along with numerous fields across existing services for managing catalogs, connections, schemas, MLflow, pipelines, and vector search. It also includes a breaking change by removing the `bundle` package and its associated service.
This release introduces new services for AI Search and Bundle Deployments, along with numerous new fields across existing services like Catalog, ML, and Vector Search. The `bundle` package and its associated workspace service have been removed.
This release introduces new services for AI Search and Bundle Deployments, along with several new fields across various services like Catalog, ML, and Vector Search. It also includes breaking changes by removing the old Bundle package and its associated workspace-level service.
* Canonicalize Bearer tokenType in Authorization headers
The SDK now detects the AI_AGENT environment variable for user agent reporting and passes unrecognized agent values through. New explicit factory methods for token and offset pagination have been added to the Paginator class. A bug was fixed where Paginator would silently drop results from token-paginated responses that returned an empty page with a next_page_token, ensuring list methods now page until the token is absent.
The SDK now better detects AI agents by honoring the Vercel AI_AGENT environment variable and passing through unrecognized agent names in the User-Agent header. This allows more specific agent versions to be visible instead of being generalized to "agent/unknown".
Week of Jun 1
13 releasesThis release is a backport from the 0.32.x line, which will receive voluntary support for a period. Consult the full changelog for specific user-facing changes, fixes, or breaking changes.
* direct: Fix updating the apps after the Go SDK upgrade ([#5444](https://github.com/databricks/cli/pull/5444))
Release: v2.11.0 (#1902)
This release introduces Unity Catalog and Workspace filesystem explorers, enhancing data and file navigation directly within VS Code. It also adds support for SPOG host URLs.
The `experimental open` command now supports opening a wider range of Databricks resource types directly in the workspace. Databricks Bundles gain a new `--select` flag for `plan` and `deploy` to target specific resources, along with improved retry logic for transient HTTP errors and support for `purge_on_delete` for Lakebase projects.
The Databricks SDK for Java now correctly handles OAuth token exchanges for browser-based flows by making the client ID optional in `DatabricksOAuthTokenSource`. This fixes a `NullPointerException` when no client ID is present in the IdP JWT, allowing account-wide token federation.
Creating an external location with `enable_file_events = false` now correctly sends this setting, preventing the server from defaulting it to true. Previously, this field was silently dropped, leading to file events being enabled despite the configuration.
Databricks SDK for Go now includes `DeploymentMode` fields for bundle deployments and versions. It also adds `CollaborationPlatformConnectivity` and `EffectiveCollaborationPlatformConnectivity` fields to settings.
This release adds new `deploymentMode` fields to bundle deployment and version objects. It also introduces `collaborationPlatformConnectivity` and `effectiveCollaborationPlatformConnectivity` fields to the settings service.
This release adds new fields for deployment mode in Databricks Asset Bundles and for collaboration platform connectivity in account settings. These changes expose additional configuration and status information through the SDK.
MLflow 3.13.0 introduces a new Role-Based Access Control system with an Admin UI for self-hosted MLflow, alongside trace retention and auto-archival to object storage. Breaking changes include the overhaul of the permission system, removal of MLServer as a pyfunc serving backend, and changes to Claude autologging and local file store behavior.
The SDK now supports a discovery flow that lands users on the account selector and includes a new method for workspace-level token management. Additional fields were added to job deployments, pipeline deployments, and token-related settings for enhanced configuration and information.
Workspace-scoped API calls now use `X-Databricks-Workspace-Id` instead of `X-Databricks-Org-Id`, accepting classic numeric IDs or other formats. New fields were added across several services, including token management, job and pipeline deployments, and OBO token requests.
This release adds comprehensive CRUD operations for feature engineering streams and a method to update token management settings. It also introduces several new fields across Jobs, Pipelines, and Token services, while removing `catalog_id` and `synced_table_id` from Postgres service objects.
Week of May 25
9 releasesYou can now manage Git credentials for service principals and permissions for Agent Bricks resources. Key fixes include proper updates for metastore external access, reliable destruction of UC objects, and configurable timeouts for vector search index creation.
This release adds new methods for managing feature engineering streams and introduces a `Parameters` field for various jobs and pipelines API calls. It also includes breaking changes by removing `CatalogId` and `SyncedTableId` fields from specific PostgreSQL service statuses.
This release adds comprehensive CRUD operations for feature engineering streams and introduces a `parameters` field across several Jobs and Pipelines API objects. It also includes breaking changes by removing `catalogId` and `syncedTableId` fields from specific Postgres service status objects.
Bundle users now receive a suggestion to set `bundle.engine: direct` in `databricks.yml` when using direct-only resources with the Terraform engine. The CLI also adds support for managing `vector_search_indexes` as a bundle resource, exclusively with the direct engine.
The config-file loader now correctly sets the profile name to "DEFAULT" when using the legacy fallback, resolving an issue where the profile was empty despite loading default settings. This fix ensures consistent profile identification, particularly for tools like the Databricks CLI that rely on the profile name for caching.
This release switches the SDK's internal formatter and linter to Ruff, aligning with Databricks' internal Python formatting guidelines. This change has no behavioral impact on the published SDK for users.
The SDK now supports more granular AI agent detection in User-Agent headers and passes unrecognized values as-is. Several API changes introduce new fields for dashboards, apps, ML materialized features, and synced table statuses, along with a `Revert` method for Lakeview dashboards.
This release introduces new methods for Lakeview and Postgres services, including `revert()` for Lakeview and `undeleteBranch()` for Postgres, alongside new fields for Jobs, IAM, and ML features. Several breaking changes require `actionType` and `resourceId` for bundle operations, `cliVersion` for bundle versions, and alter `tags` for marketplace listings and pagination for cluster events.
Week of May 18
15 releasesMLflow now features a major overhaul of Role-Based Access Control with a new Admin UI and unified permission APIs, alongside new coding-agent tracing plugins for AI Gateway. Databricks practitioners can also leverage end-to-end trace archival, Helm charts for Kubernetes deployment, and a new API for stress-testing GenAI agents.
This release introduces new fields for IAM user attributes, job trigger state, and enhanced branch management capabilities including undelete and purge options. It also includes several breaking changes for bundle operations, bundle versioning, marketplace listing tags, and cluster events pagination.
OAuth tokens for interactive logins are now stored in OS-native secure stores by default, requiring re-authentication after upgrading. A new `databricks aitools` command group is added for installing Databricks skills into coding agents.
Release: v2.10.8 (#1899)
- Bump Databricks JS SDK to 0.17.0
This release introduces a new bundle package and workspace-level service for managing Databricks Asset Bundles. It also adds an MtlsConfig field to ml.AuthConfig for mTLS authentication configurations.
This release adds support for the VARIANT type. No other user-facing features, fixes, or breaking changes are included.
This release fixes several regressions, including issues with MERGE operations, schema overwrites with predicates, and partition column changes. It also enables passing non-string datatypes in custom commit metadata and updates the minimum PyArrow version to 21.0.0 for variant type support.
Databricks SDK for Go now supports new fields for configuring pipeline refresh selections within jobs. Additionally, new fields for operational email custom recipients are available in settings.
New fields for pipeline refresh selection and checkpoint reset are available in `PipelineParams` and `PipelineTask` objects. Setting objects now include fields for operational email custom recipients.
Databricks SDK for Python now includes new fields for pipeline refresh and checkpoint reset selections within job parameters and tasks. Additionally, new fields for operational email custom recipients are available in settings.
* Add `CreateWorkspaceAssignmentDetail`, `DeleteWorkspaceAssignmentDetail`, `GetWorkspaceAssignmentDetail`, `ListWorkspaceAssignmentDetails` and `UpdateWorkspaceAssignmentDetail` methods for [a.AccountIamV2](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/iamv2#AccountIamV2API) account-level service.
* Add `createWorkspaceAssignmentDetail()`, `deleteWorkspaceAssignmentDetail()`, `getWorkspaceAssignmentDetail()`, `listWorkspaceAssignmentDetails()` and `updateWorkspaceAssignmentDetail()` methods for `accountClient.accountIamV2()` service.
This release introduces new methods for managing IAMv2 workspace assignments at both account and workspace levels, alongside new fields for ML features, disaster recovery stable URLs, and customer-facing ingress network policies. It also includes breaking changes to the `list_features()` method and `ListFeaturesRequest` by adding new required catalog and schema name fields.
Week of May 11
9 releasesThis release fixes a bug preventing partition column changes when overwriting tables and resolves a memory regression in Python MERGE operations. It also adds support for passing non-string datatypes in custom commit metadata and enables nanosecond timestamp support via a Cargo feature.
TypeScript SDK 0.2.0
Bump several RC TypeScript packages stable version.
The `vector_search_endpoints` configuration and commands now use `target_qps` instead of `min_qps`, requiring updates to `databricks.yml` and CLI invocations. Authentication commands received several usability improvements, including better keyring handling, clearer profile selection, and enhanced reporting of token storage.
* Add `catalogName`, `createdAt`, `createdBy`, `name` and `schemaName` fields for `com.databricks.sdk.service.ml.Feature`.
Jobs now support a `disabled` field for `RunTask` and `SubmitTask` configurations. The `ConnectionType` enum in Unity Catalog gained new values for HubSpot, GitHub, Outlook, and Smartsheet, while the `unspecified_resource_name` field was removed from `RequestedResource` in the Postgres service.
This release fixes state decoding errors for databricks_library, databricks_share, and databricks_quality_monitor after upgrading from v1.113.0 to v1.114.0. It also resolves issues where several account-level data sources and settings failed due to workspace ID resolution errors.
This release fixes an issue where the Databricks CLI `--profile` fallback was broken. It also introduces new API methods for `workspaceClient.supervisorAgents()` and `workspaceClient.vectorSearchEndpoints()`, along with several breaking changes related to `Example` and `Tool` fields, and `minQps` in Vector Search endpoints.
This release introduces new methods for managing supervisor agents and vector search endpoints, along with several new fields for connector options, ingestion sources, and customer-managed keys. Breaking changes include the removal of `min_qps` fields from vector search endpoint configurations and making `guidelines` and `description` fields optional in certain services.
The default query comment now includes the invocation ID. This release also fixes several issues, including enforcing the 255-character identifier limit for Databricks relations and suppressing various spurious warnings related to microbatch concurrency, insert overwrite, and Iceberg configurations.
Week of May 4
3 releases* Add `ConfluenceOptions` field for [pipelines.ConnectorOptions](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#ConnectorOptions).