Latest releases from the Databricks ecosystem.

This release fixes an issue where tag policy and assignment operations failed with 404 errors when tag keys contained forward slashes. It also introduces a breaking change by making the `ResourceId` field optional in `bundledeployments.Operation`.

The `resource_id` field in `bundledeployments.Operation` is no longer required, which is a breaking change. Several bug fixes improve performance and stability, including caching OIDC tokens, making `WorkspaceClient.dbutils` lazy to prevent crashes on Spark Connect, and better handling of remote runtime implementations.

The Databricks SDK for Go now includes a TypeOverrides field for database.SyncedTableSpec and postgres.SyncedTableSyncedTableSpec. This allows practitioners to specify type overrides when defining synced tables.

The `direct` deployment engine is now Generally Available and the default for new deployments, with existing deployments retaining their current engine. New CLI commands include `databricks quickstart` for interactive guides and `databricks version --check` to report available updates.

1.12.1

This release exposes Databricks Jobs IDs in dbt's adapter response for better correlation and adds support for SPOG (Single Point of Gateway) hosts. It also fixes issues with streaming tables, materialized views, snapshots, and ensures column-level constraints are only applied when `contract.enforced: true`.

This release adds an AcceleratedSync field to SyncedTableSpec for both generic database and PostgreSQL services. This new field enables configuring accelerated synchronization for synced tables.

This release adds new services for AI Search and Bundle Deployments, along with numerous new fields across various services like Catalog, ML, and Vector Search. It also includes a breaking change by removing the `bundle` package and its associated service.

This release introduces new services for AI Search and Bundle Deployments, along with numerous new fields across existing services like Catalog, ML, and Vector Search. The `bundle` package and its associated workspace service have been removed.

This release introduces new services for AI Search and Bundle Deployments, along with several new fields across existing services like Catalog, ML, and Pipelines. It also includes breaking changes by removing the old Bundle package and its associated workspace-level service.

* Canonicalize Bearer tokenType in Authorization headers

The SDK now detects the AI_AGENT environment variable for user agent reporting and passes unrecognized agent values through. Pagination factory methods were added to explicitly define strategy, and a bug was fixed where token-paginated results were silently dropped on empty pages with a next token.

The SDK now better detects AI agents by honoring the Vercel AI_AGENT environment variable and passing through unrecognized agent names in the User-Agent header. This allows more specific agent versions to be reported instead of a generic "unknown" agent.

This release is a backport from the 1.x branch to the 0.32.x line, which will receive continued support. Consult the full changelog for specific user-facing changes, fixes, or breaking changes.

* direct: Fix updating the apps after the Go SDK upgrade ([#5444](https://github.com/databricks/cli/pull/5444))

Release: v2.11.0 (#1902)

This release introduces a Unity Catalog explorer and a workspace filesystem explorer, enhancing navigation and management within Databricks. It also adds support for SPOG host URLs.

The experimental open command now supports opening various Databricks resource types directly from the CLI. Databricks Bundles gain a new --select flag for partial deployments and improved handling for Lakeflow Designer notebooks and Lakebase project deletion.

The Databricks SDK for Java now correctly handles OAuth token exchanges for browser-based flows by making the client ID optional in `DatabricksOAuthTokenSource`. This fixes a `NullPointerException` when no client ID is present, allowing account-wide token federation.

Creating an external location with `enable_file_events = false` now correctly sends this setting, preventing the server from defaulting it to true. Previously, this field was silently dropped, leading to file events being enabled despite the configuration.

Databricks SDK for Go now includes `DeploymentMode` fields for bundle deployments and versions. It also adds `CollaborationPlatformConnectivity` and `EffectiveCollaborationPlatformConnectivity` fields to settings.

This release adds new `deploymentMode` fields to bundle deployment and version objects. It also introduces `collaborationPlatformConnectivity` and `effectiveCollaborationPlatformConnectivity` fields for settings.

This release adds new fields for deployment mode in Databricks Asset Bundles and for collaboration platform connectivity in account settings. These changes expose additional configuration options and status information through the SDK.

MLflow 3.13.0 introduces a new Role-Based Access Control system with an Admin UI for managing users and permissions, alongside trace retention and auto-archival to object storage. This release also includes one-click observability for coding agents, new engines for MLflow Assistant, and an official Helm chart for Kubernetes deployments.

The SDK now supports directing users to the account selector during the discovery flow and includes a new method for workspace token management. Additional fields have been added to job deployments, pipeline deployments, and token-related settings for enhanced configuration and information.

Workspace-scoped API calls now use `X-Databricks-Workspace-Id` header, accepting classic numeric or other workspace identifiers. New fields were added across several services, including token management, job and pipeline deployments, and OBO/public token information.

This release adds comprehensive CRUD operations for feature engineering streams and a method to update token management settings. It also introduces breaking changes by removing `catalog_id` and `synced_table_id` fields from specific Postgres service objects.

You can now manage Git credentials for service principals and permissions for Agent Bricks resources. Key bug fixes include correctly updating metastore external access, destroying UC objects after workspace binding removal, and handling libraries removed outside Terraform.

This release adds new methods for managing feature engineering streams and introduces `Parameters` fields for Databricks Jobs and Pipelines. It also includes breaking changes by removing `CatalogId` and `SyncedTableId` fields from PostgreSQL catalog status objects.

This release adds comprehensive CRUD operations for feature engineering streams and introduces a `parameters` field across various Jobs and Pipelines API objects. It also includes breaking changes by removing `catalogId` and `syncedTableId` fields from specific Postgres service status objects.

Bundle users now receive a suggestion to set `bundle.engine: direct` in `databricks.yml` when using direct-only resources with the Terraform engine. Vector search indexes are now supported as a direct-engine bundle resource, including UC grants and destructive operation confirmations.

The config-file loader now correctly sets the profile name to "DEFAULT" when using the legacy fallback, which fixes issues for consumers that derive identifiers from the profile name. This ensures consistent profile identification across different operations, such as OAuth cache key lookups.

This release switches the SDK's internal formatter and linter to Ruff, aligning with Databricks' internal Python formatting guidelines. This change has no behavioral impact on the published SDK for users.

The Databricks SDK for Go now supports additional fields for dashboards, apps, and ML materialized features, including a new `Revert` method for Lakeview dashboards. Performance for `WorkspaceClient.CurrentWorkspaceID` has improved by excluding unnecessary entitlement data from SCIM requests.

This release introduces new methods for Lakeview and Postgres services, including `revert()` for Lakeview and `undeleteBranch()` for Postgres, alongside new fields for Jobs, IAM, and ML services. Several breaking changes require `actionType` and `resourceId` for bundle operations, `cliVersion` for bundle versions, and alter the `tags` field for marketplace listings and pagination for cluster events.

This release introduces a new Databricks Asset Bundles service and adds methods for reverting Lakeview dashboards and undeleting Postgres branches. It also includes breaking changes to the `marketplace.ListListingsRequest` tags field and pagination for `ClustersAPI.events`.

MLflow now features a major overhaul of Role-Based Access Control with a new Admin UI and unified permission APIs, alongside new capabilities for GenAI agent observability including coding-agent tracing plugins and automated stress-testing. Databricks practitioners can also leverage end-to-end trace archival, OpenTelemetry span links, and database replica routing for improved scalability.

This release introduces new fields for IAM user attributes, job trigger state, and enhanced branch management capabilities including undelete and purge options. It also includes several breaking changes for bundle operations, bundle versioning, marketplace listing tags, and cluster event pagination.

OAuth tokens for interactive logins are now stored in the OS-native secure store by default, requiring re-authentication after upgrading. A new `databricks aitools` command group is added for installing Databricks skills into coding agents.

Release: v2.10.8 (#1899)

- Bump Databricks JS SDK to 0.17.0

This release introduces a new bundle package and workspace-level service for managing Databricks Asset Bundles. It also adds an MtlsConfig field to ml.AuthConfig for mTLS authentication configurations.

* Add `com.databricks.sdk.service.bundle` package.

This release adds support for the VARIANT type.

This release fixes several regressions, including issues with MERGE operations, overwriting tables with partition changes, and schema mode behavior with mixed-case columns. It also enables passing non-string datatypes in custom commit metadata and updates the minimum PyArrow version to 21.0.0, which includes preliminary support for variant types.

Databricks SDK for Go now supports new fields for configuring pipeline refresh selections within jobs. Additionally, new fields for operational email custom recipients are available in settings.

New fields for pipeline refresh selection and checkpoint reset are available in `PipelineParams` and `PipelineTask` objects. The `Setting` object now includes fields for operational email custom recipients.

This release adds new fields for pipeline refresh and checkpoint reset selections within job parameters and tasks. It also introduces new fields for custom operational email recipients in settings.

* Add `CreateWorkspaceAssignmentDetail`, `DeleteWorkspaceAssignmentDetail`, `GetWorkspaceAssignmentDetail`, `ListWorkspaceAssignmentDetails` and `UpdateWorkspaceAssignmentDetail` methods for [a.AccountIamV2](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/iamv2#AccountIamV2API) account-level service.

* Add `createWorkspaceAssignmentDetail()`, `deleteWorkspaceAssignmentDetail()`, `getWorkspaceAssignmentDetail()`, `listWorkspaceAssignmentDetails()` and `updateWorkspaceAssignmentDetail()` methods for `accountClient.accountIamV2()` service.

This release introduces new methods for managing IAM v2 workspace assignments at both account and workspace levels. It also includes breaking changes to the `list_features()` method and `ListFeaturesRequest` in ML, requiring updates to existing code.

This release adds support for metric views, row filters, Python UDFs, and key-only Databricks tags. A breaking change alters how `databricks_tags` merge across hierarchy levels.

This release fixes a bug preventing partition column changes when overwriting tables and addresses a regression in MERGE operations that caused high memory usage. It also adds support for passing non-string datatypes in custom commit metadata from Python and includes nanosecond timestamp support behind a Cargo feature.

TypeScript SDK 0.2.0

Bump several RC TypeScript packages stable version.

The `vector_search_endpoints` configuration and commands now use `target_qps` instead of `min_qps`, requiring updates to `databricks.yml` and CLI invocations. Authentication commands received several usability improvements, including better keyring handling, clearer profile selection, and improved host input.

* Add `catalogName`, `createdAt`, `createdBy`, `name` and `schemaName` fields for `com.databricks.sdk.service.ml.Feature`.

Job tasks can now be marked as disabled in both run and submit requests. Catalog connection types gained new enum values for HubSpot, GitHub, Outlook, and Smartsheet, while the `unspecified_resource_name` field was removed from `RequestedResource` in the Postgres service.

This release fixes state decoding errors for `databricks_library`, `databricks_share`, and `databricks_quality_monitor` after upgrading from v1.113.0 to v1.114.0. It also resolves issues where several account-level data sources and settings failed due to workspace ID resolution errors, now supporting the `api` field or exempting them from workspace tracking.

This release fixes an issue where the Databricks CLI `--profile` fallback was broken. It also introduces new API methods for `workspaceClient.supervisorAgents()` and `workspaceClient.vectorSearchEndpoints()`, along with several breaking changes related to `Example` and `Tool` fields, and `minQps` in Vector Search endpoints.

This release introduces new methods for managing supervisor agents and vector search endpoints, along with several new fields for connector options, ingestion sources, and customer-managed keys. Breaking changes include the removal of `min_qps` fields from vector search endpoint configurations and making `guidelines` and `description` fields optional in certain services.

This release adds the dbt invocation ID to default query comments and enforces the 255-character identifier length limit for Databricks relations. It also resolves several issues, including spurious warnings for MicrobatchConcurrency and insert_overwrite, and improves materialized view refresh behavior with databricks_tags.

* Add `ConfluenceOptions` field for [pipelines.ConnectorOptions](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#ConnectorOptions).

Release: v2.10.7 (#1895)

This release adds initial compatibility for Remote Development and renames "Databricks Asset Bundles" to "Declarative Automation Bundles." It also fixes issues with profile management, including signing in with existing hosts and listing profiles with account IDs.

The `databricks api` command now supports unified hosts, allowing `--account` and `--workspace-id` for scoping, and recognizes `?o=<workspace-id>` in URLs. JSON output for single objects now uses standard spacing.

Nightly - unity-catalog

Nightly build from unity-catalog

MLflow 3.12.0 introduces multimodal tracing, allowing users to store and render PDFs, audio, and images as artifact attachments in tracing spans. It also adds AI Gateway guardrails to prevent unsafe model inputs/outputs and expands coding agent tracing support for Codex, Gemini, and Qwen.

Nightly - fix-new-profile-sign-in

Nightly build from fix-new-profile-sign-in

You can now adopt pre-existing Databricks Postgres branch and endpoint resources using the replace_existing argument. A change to databricks_external_location prevents Terraform drift from server-populated fields.

This release introduces new services for disaster recovery and temporary volume credentials, alongside extensive new methods for knowledge assistants. It also adds support for various new fields and enum values across several services, while making the `description` field optional for `SupervisorAgent` and removing the `connection` field from `Tool`.

* Fixed Databricks CLI `--profile` fallback by detecting the CLI version at init time. The previous error-based detection was broken because `--profile` is a global Cobra flag silently accepted by old CLIs.

The `--experimental-is-unified-host` flag and related configuration options are removed; unified hosts are now detected automatically. List commands now support interactive pagination, displaying 50 rows at a time with options to view more or quit. You can also opt-in to experimental OS-native secure token storage.

Rolls back to v1.113.0 as the stable release.

This release introduces new resources and data sources for managing disaster recovery failover groups, stable URLs, supervisor agents, and UC secrets. It also enables adopting pre-existing Postgres branch and endpoint resources using the `replace_existing = true` argument.

MLflow 3.12.0rc0 introduces enhanced AI agent development features, including automatic tracing for various coding assistants and OpenClaw, plus new AI Gateway guardrails for production safety. It also adds multimodal trace attachments for images and audio, and a new mlflow.diffusers flavor for saving and serving diffusion models.

This release introduces a new disaster recovery package and adds methods for managing knowledge assistant examples. It also includes breaking changes related to the `supervisoragents.Connection` and `supervisoragents.Tool` fields.

Unity Catalog AI 0.4.0

DatabricksFunctionClient now supports an optional warehouse_id for function execution, enabling use in workspaces without serverless compute. Python 3.10+ is now required, and several bug fixes address issues with Gemini toolkit, LangGraph, and OSS client function creation.

Unified host detection is now automatic, removing the `Experimental_IsUnifiedHost` field and enabling a single configuration profile for both account and workspace operations. The file-based OAuth token cache has been removed, defaulting to an in-memory cache unless a persistent cache is explicitly provided.

The SDK now automatically detects AI coding agents and appends agent information to HTTP request headers, while also removing the unused `experimentalIsUnifiedHost` field from `DatabricksConfig`. A bug fix addresses `X-Databricks-Org-Id` header issues for `SharesExtImpl.list()` on SPOG hosts, and several API method paths have changed, which are breaking changes.

This release introduces new workspace-level services for secrets_uc and supervisor_agents, along with an update method for tokens. Several breaking changes occurred, primarily due to method path changes for data classification, environments, knowledge assistants, postgres, and warehouses services.

WorkspaceExt upload/download and SharesExt list now include the X-Databricks-Org-Id header for SPOG host compatibility. WorkspaceClient.get_workspace_id avoids an API call when the workspace ID is already known, fixing a SPOG host failure.

The CLI now supports a --limit flag for paginated list commands and caches host metadata lookups for faster execution. The 'auth env' command is deprecated, and Bundles now support Vector Search Endpoints and prompt before destroying Lakebase resources.

This release enhances the new Datafusion TableProvider and improves log parsing performance. It also fixes numerous bugs, including issues with DeltaScan schema handling, streamed merge file pruning, and incorrect row counts for DELETE operations.

This release introduces new workspace-level services for supervisor agents and Unity Catalog secrets, along with an update method for tokens. Several existing API methods for data classification, environments, knowledge assistants, Postgres, and warehouses have breaking changes due to path modifications.

This release adds Azure MSI authentication support and improves `.databrickscfg` default profile resolution. It also fixes issues with non-JSON error responses and Databricks CLI token scope mismatches, alongside several API additions and two breaking changes.

The SDK now supports a default host metadata resolver for easier shared configuration and improved SPOG host compatibility for Workspace and Shares API calls. WorkspaceClient.CurrentWorkspaceID() directly uses the configured WorkspaceID, eliminating an API call and fixing SPOG host authentication issues.

Nightly - fix/replace-softprops-action

Nightly build from fix/replace-softprops-action

This release automatically detects unified hosts for account and workspace operations, and accepts DATABRICKS_OIDC_TOKEN_FILEPATH for OIDC token files. Python 3.8 and 3.9 are no longer supported, requiring Python 3.10 or newer.

This release adds support for Notebook-scoped packages when submitting commands or running notebook jobs. It also includes fixes for workflow job creation, preventing duplicate aliases in empty mode, and enabling insert-by-name for microbatch and replace_where strategies.

This release fixes an error where bundle commands failed due to an expired key when downloading Terraform. Users will no longer encounter the "unable to verify checksums signature: openpgp: key expired" error.

This release fixes an error that prevented bundle commands from running due to an expired key when downloading Terraform. Users can now execute bundle commands without encountering checksum signature verification failures.

This release fixes an error where bundle commands failed due to an expired key when downloading Terraform. Users can now run bundle commands without encountering checksum signature verification issues.

This release fixes an error where bundle commands failed due to an expired OpenPGP key during Terraform downloads. Users can now run bundle commands without encountering checksum verification errors.

This release fixes an error where bundle commands failed due to an expired key when downloading Terraform. Users will no longer encounter "unable to verify checksums signature: openpgp: key expired" errors.

This release fixes a critical error preventing bundle commands from downloading Terraform due to an expired key signature. Users previously encountering "unable to verify checksums signature: openpgp: key expired" will now find bundle commands functional.

This release fixes an error where bundle commands failed due to an expired key when downloading Terraform. Users can now run bundle commands without encountering checksum signature verification issues.

This release fixes a critical error preventing bundle commands from running due to an expired GPG key. Databricks practitioners can now use bundle commands without encountering checksum verification failures.

This release fixes a "key expired" error when running `databricks bundle deploy` by updating the Terraform binary installation process. It now uses a hardcoded ArmoredPublicKey to resolve checksum signature verification issues.

UnityCatalog 0.4.1

The Unity Catalog Spark connector now supports atomic REPLACE TABLE AS SELECT and Dynamic Partition Overwrite for UC Managed Delta Tables, and introduces a credential-scoped file system to prevent out-of-memory errors. A critical security fix addresses a JWT issuer validation bypass, requiring new server configurations for existing deployments with authorization enabled.

This release adds new resources for managing Postgres catalogs, synced tables, and workspace base environments. It also introduces an 'api' field for dual account/workspace resources to explicitly control API level, supporting unified hosts.

Delta Lake 4.2.0

This release enhances Unity Catalog managed tables with support for REPLACE TABLE, RTAS, Dynamic Partition Overwrite, and improved streaming reads. It also introduces general availability for Variant columns and new Geospatial and Collations table features.

This release raises the minimum Go version to 1.24 and introduces new features like a host metadata resolver hook and a lazy iteration utility with item limits. It also includes numerous bug fixes for token acquisition and caching, along with extensive API additions for various Databricks services.

TypeScript SDK 0.2.0 RC1

Release candidate for `@mlflow/vercel` TypeScript package with version 0.2.0: https://github.com/mlflow/mlflow/pull/22105

MLflow 3.11.1 introduces AI-powered issue detection for agent traces, budget alerts for AI Gateway spending, and a new interactive graph view for visualizing trace hierarchies. It also enhances security with pickle-free model serialization and improves dependency management with native UV support.

Model Catalog

Per-provider model catalog files. Updated weekly by CI.

Stripped third-party dependencies from evaluation and AI Gateway features, replacing external provider routing with built-in implementations.

This release adds a new method to force-refresh cached U2M OAuth tokens, returning an error on failure instead of falling back. It also introduces a new error type for when a refresh token is missing during a refresh request.

This release adds a new `disableGovTagCreation` field to the `RestrictWorkspaceAdminsMessage` for both settings and settingsv2 services. This field allows configuring restrictions on the creation of governance tags by workspace administrators.

The SDK now automatically detects AI coding agents and appends `agent/<name>` to HTTP request headers. Two new `DisableGovTagCreation` fields were added to `RestrictWorkspaceAdminsMessage` in both `settings` and `settingsv2`.

The SDK now automatically detects AI coding agents and appends agent information to HTTP request headers. Two new `disable_gov_tag_creation` fields were added to restrict workspace admin settings.

* Add resource and data sources for `databricks_postgres_role`.

* Add `parentPath` field for `com.databricks.sdk.service.dashboards.GenieSpace`.

OAuth token refreshing is now proactive for tokens expiring within five minutes, improving token validity for callers. The `dashboards.GenieSpace` struct gains a new `ParentPath` field.

This release introduces a new `environments` service for managing Databricks environments and adds a `parent_path` field to `GenieSpace` dashboards. It also includes a new `can_create_app` permission level for IAM.

The Databricks SDK for Go now supports specifying a `default_profile` within the `[__settings__]` section of your `.databrickscfg` file. This allows for easier configuration management when working with multiple profiles.

This release introduces AI-powered issue identification for agent traces, budget alerts for AI Gateway spending, and an interactive graph view for trace hierarchies. It also includes pickle-free model serialization for enhanced security and support for OpenTelemetry GenAI conventions.

Release: v2.10.6 (#1858)

This release fixes an issue where the VS Code extension would not correctly handle 404 errors from the Databricks SDK. This improves stability when interacting with Databricks resources.

Databricks Jobs now support new alert-related fields in run outputs and task definitions, enabling more granular notification configurations. A new `environments` package and service were introduced, along with a `CAN_CREATE_APP` permission level for IAM.

This release adds new fields for job alert configurations across various job-related structs, including `RunOutput`, `RunTask`, `SubmitTask`, and `Task`. It also introduces a new `environments` package and service for managing workspace environments, alongside a `CanCreateApp` permission level for IAM.

You can now specify a `default_profile` in your `.databrickscfg` file's `[__settings__]` section for consistent profile resolution. New `alert_output` and `alert_task` fields have been added to several job-related API objects.

python-v1.5.0: faster writes, log compaction, spil config in MERGE

This release introduces faster Delta table writes through parallel partition writers and adds log compaction for improved performance. The `get_add_actions` method now returns an ArrowTable instead of an ArrowRecordBatch, which is a breaking change.

This release adds new fields to the Ingestion Pipeline Definition and Origin objects, enhancing configuration options for data ingestion pipelines. It also introduces a subDomain field for External Function Requests in the serving API.

This release adds new fields and methods across several services, including updates for ML features, pipelines, and Postgres roles. It also introduces breaking changes by making previously required fields optional in ML-related DeltaTableSource, Feature, Function, and KafkaSource configurations.

Databricks SDK for Python now supports new fields for defining ingestion pipelines, including connector type, data staging options, and detailed ingestion source information. External function requests in model serving can now specify a sub-domain.

This release enables concurrent microbatch execution and adds an optimize() call to snapshot materialization. Fixes include quoting catalog names, adding a cluster clause to streaming table alter SQL, and applying column-level tags for V1 table materialization.

This release adds new fields and methods for managing ML features, functions, and Kafka sources, including a new updateRole method for workspace Postgres services. Several fields related to ML DeltaTableSource, Feature, Function, and KafkaSource are now optional, which is a breaking change.

This release introduces new methods for the Genie workspace service and an update_role method for the Postgres service. Several fields across ML and other services are now optional, including breaking changes for `entity_columns`, `inputs`, `function_type`, `entity_column_identifiers`, and `timeseries_column_identifier` in various ML-related services.

This release introduces new resources for managing Postgres databases, data classification catalogs, and knowledge assistant configurations. It also renames the `databricks_apps_space` resource to `databricks_app_space`.

This release introduces new services for Data Classification and Knowledge Assistants, accessible via `workspaceClient`. It also adds several new methods for managing Genie evaluation runs and results.

Databricks SDK for Java now allows fine-grained control over HTTP request timeouts through a new `withRequestConfig` method on `CommonsHttpClient.Builder`. This enables practitioners to configure specific timeout settings for their API calls.

Databricks CLI authentication now correctly errors on token scope mismatches, prompting re-authentication instead of silently using incorrect permissions. New `dataclassification` and `knowledgeassistants` services and corresponding workspace-level APIs have been added.

This release adds a "try-it" page for Gateway usage examples and filters gateway experiments from the experiment list in the UI. It also fixes numerous UI issues, artifact download problems, and tracing errors, including issues with model copying across workspaces.

The SDK now dynamically adjusts authentication token stale periods, extending them up to 20 minutes for standard OAuth. Users can override this behavior by manually setting the stale period in the CachedTokenSource builder.

The SDK now dynamically adjusts authentication token refresh periods based on the token's initial lifetime, up to 20 minutes for standard OAuth. Users can override this behavior by specifying a fixed stale_duration in the Refreshable class constructor.

This release introduces row filter functionality and support for metric views.

Delta Lake 4.1.0

Delta Lake 4.1.0 enhances Unity Catalog integration with improved support for catalog-managed tables, including atomic CTAS and conflict-free feature enablement for Deletion Vectors and Column Mapping. It also introduces a new Spark V2 connector based on Delta Kernel API for streaming reads and server-side planning capabilities.

Utility clusters created by resources like `databricks_aws_s3_mount` now default to `SPOT_WITH_FALLBACK` for improved reliability. Plaintext credential fields in `databricks_model_serving` and `databricks_git_credential` are now marked sensitive to prevent display in plan/apply output.

MLflow 3.10.0 introduces multi-workspace support for organizing experiments and models, alongside new GenAI features like multi-turn evaluation, LLM cost tracking, and AI Gateway usage analytics. The UI has been redesigned for improved navigation, and in-UI trace evaluation is now available.

* Mark `effective_enable_file_events` as read-only in `databricks_external_location` to prevent Terraform drift.

This release fixes an issue where multiple foreign keys between tables were not retained after an incremental run. It also resolves a bug where changes to materialized view partition_by clauses failed, now using DROP and CREATE for updates.

You can now manage Lakebase database project permissions using `database_project_name` in `databricks_permissions` and configure instance pool node type flexibility with a new block in `databricks_instance_pool`. A bug was fixed that previously caused errors during WorkspaceClient() creation in `databricks_grant` and `databricks_grants` resources.

This release adds new resources and data sources for managing Databricks Apps Space and Endpoints. It also updates the underlying Go SDK to version 0.108.0.

This release fixes a specific issue within the Rust core of delta-rs, which impacts Python users. It backports a previous pull request to resolve the problem.

UnityCatalog 0.4.0

Unity Catalog 0.4.0 introduces full support for AWS Storage Credentials and External Locations, centralizing cloud storage authentication and access management. It also enables atomic CTAS for Delta tables, defaults credential renewal in the Spark connector, and adds DSPy integration for AI functions.

The `databricks_workspace_file` resource now supports payloads larger than 10MB, and `databricks_mws_storage_configurations` includes a `role_arn` field for S3 bucket sharing with Unity Catalog. Several bug fixes address issues with `databricks_mws_ncc_private_endpoint_rule` updates, `databricks_secret_acl` management, `databricks_app` resource reading, and `databricks_users` data source `extra_attributes` parameter.

MLflow now supports multi-workspace environments for organizing experiments and resources, alongside a new top-level navigation split for GenAI and Classical ML workflows. Key new features include multi-turn conversation simulation, automatic LLM trace cost tracking, AI Gateway usage analytics, and a CLI command to generate a demo environment.

This release introduces a session-first DataFusion integration and exposes Delta Lake Vacuum metadata as Arrow streams. It also fixes issues with schema merge appends for generated columns and improves parquet predicate pushdown.

This release fixes an issue where DML predicates were not correctly resolved against the execution scan schema. It also updates the asserted nullability in the DataValidation output schema.

SQL warehouses now support "5X-Large" cluster sizes and a higher maximum of 40 clusters. This release also fixes permanent drift for external model credentials in databricks_model_serving and improves dashboard file content change detection.

Release: v2.10.5 (#1834)

- Update Databricks CLI to v0.286.0

v.3.9.0

MLflow 3.9.0 introduces an in-product MLflow Assistant chatbot and a Trace Overview Dashboard for GenAI experiments, enhancing debugging and performance insights. The AI Gateway is revamped for direct tracking server integration, alongside new LLM judge features for online monitoring and custom prompt building.

This release enables deletion vector features for Delta tables and improves logical planning for update and delete operations. It also fixes issues with reporting failed data in data checks and supporting user names in Azure URLs.

This release adds new resources for account user settings, default warehouse overrides, and fixes issues with importing databricks_share and creating databricks_dashboard resources. The exporter now supports additional network policy resources and rewrites cloud-specific attributes in cluster policies.

This release updates the dbt-core pin for the 1.10.latest version. No user-facing features, fixes, or breaking changes are included.

MLflow 3.9.0rc0 introduces an in-product AI Assistant for debugging and a new Trace Overview Dashboard for GenAI experiments. The AI Gateway is now integrated into the tracking server, and users can configure LLM judges for online monitoring and build custom judges directly in the UI.

Delta Lake 4.0.1

The "managed table" feature is renamed to `catalogManaged` and its associated Unity Catalog table ID property is updated, which is a breaking change. Unity Catalog now supports OAuth authentication for catalogs and allows creating UC-managed Delta tables where table properties are sent to the UC server as the source of truth.

This release introduces a new table provider in the query builder and DataSink, along with migrating table scans. It also fixes an issue where deleting from an empty table failed and addresses data validation mask exhaustion.

This release updates internal dbt-common and dbt-adapter dependency pins for the 1.10.x series. No user-facing features, fixes, or breaking changes are included.

This release updates the dbt-core dependency pin. No user-facing features, fixes, or breaking changes are included.

* feat: Add query-id to SQLQueryStatus by @colin-rogers-dbt in https://github.com/databricks/dbt-databricks/pull/1280

python-v1.3.1: read support deletion vectors, column mapping

This release adds read support for Delta Lake tables utilizing deletion vectors and column mapping. It also includes performance improvements for table scans and predicate pushdown, alongside better error messages for Unity Catalog and LakeFS.

This release adds support for multiple constraints at once, generates Symlink Manifests for external engines, and introduces GCS auto-registration. It also includes fixes for schema evolution in merge operations, improved error reporting, and enhanced handling of empty tables.

This release introduces several API changes and integrates `delta_kernel` for improved stats parsing performance. It also fixes issues with schema evolution during merge operations and null handling in scalar extraction.

This release updates the dbt-core upper bound, enabling compatibility with dbt-core version 1.10.16. This allows Databricks practitioners to use dbt-databricks with the latest 1.10.x dbt-core releases.

Release: v2.10.4 (#1821)

- Update Databricks CLI to v0.280.0

UnityCatalog 0.3.1

The Unity Catalog Spark connector now supports automatic credential renewal for S3, Azure, and GCS, and introduces OAuth authentication for seamless token management. This release also adds experimental support for UC-managed Delta tables, allowing Unity Catalog to coordinate storage and commits.

This release fixes an issue where default query tag values were not properly escaped or truncated. Databricks practitioners will now see correct handling of these values in their dbt projects.

Assessment export now uses zip compression to handle large files and prevent failures. Workflow assessment is now a standalone task, offering more control and manual triggering options.

You can now skip workflow assessments during installation and export assessment results to Excel. Federated catalog creation and group fetching have improved robustness with better error handling and retries.

Release: v2.10.3 (#1772)

This release updates the Databricks CLI to v0.266.0. Be aware that this CLI version introduces breaking changes.

You can now force a rerun of an assessment to get new results, even if it was previously run. Dashboard management is more robust, automatically creating a new dashboard if a PermissionDenied exception occurs or if the existing one is invalid.

UnityCatalog 0.3.0

Unity Catalog now supports Spark 4.0 and Delta Lake 4.0, enhancing compatibility with the latest Databricks runtime components. New API surfaces for credentials and external locations provide more flexible handling of external storage services.

Release: v2.10.2 (#1738)

- Update Databricks CLI to v0.259.0

The workflow assessment functionality now includes an experimental task that analyzes recent workflows for migration problems, providing recommendations and documentation links. UCX documentation has been significantly enhanced with a revamped main page, a new Getting Started section, and updated contributor guidance.

Release: v2.10.1 (#1704)

This release updates the Databricks CLI to v0.253.0 and improves virtual environment management by using UV. It also adds support for complex variables in the UI and prevents `sys.exit` calls in Jupyter initialization scripts.

Delta Lake 4.0.0

Delta Lake 4.0 introduces preview support for catalog-managed tables, the Variant data type for semi-structured data, and Delta Connect for Spark Connect integration. It also enables instant dropping of table features without history truncation and deprecates Delta Standalone and its connectors in favor of Delta Kernel.

Delta Lake 3.3.2

This release fixes an issue where stale checksum files were not cleaned up during Delta table metadata maintenance. It also includes a fix for Delta Flink to correctly map BinaryType to Flink's data types.

Unity Catalog AI 0.3.1

The Unity Catalog AI client now automatically configures itself in Databricks environments, improves handling of SQL NULL default parameters, and offers more robust connection recovery. Error messages are clearer, Spark sessions are created on-demand, and function execution returns native Python types.

UCX now requires matching account groups to exist before assessment and clarifies Service Principal setup for account-level installations. It also fixes table migration when a default catalog is set and pauses the migration progress workflow schedule by default.

Delta Lake 3.3.1

This release fixes an issue allowing user-specified schemas on read if consistent with the table schema. It also includes a kernel fix for handling non-uniform value types in map[string, string] within Delta commit files.

Release: v2.9.4 (#1670)

- Rollback Databricks CLI to v0.245.0 to fix auth problems

Release: v2.9.3 (#1665)

- Update Databricks CLI to v0.248.0

You can now create account-level groups from nested workspace-local groups and install UCX in offline environments. The UI for tables and compute summaries now includes clickable hyperlinks for easier navigation.

Release: v2.9.2 (#1635)

Windows users can now run notebooks locally using the %run magic, fixing a previous bug. The Databricks CLI has been updated to version 0.245.0.