Databricks SDK
Recent items mentioning Databricks SDK across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
The Databricks SDK for Python and Go now supports the Statement Execution V2 API, deployment security scan management for Databricks Apps, and custom HTTP transport adapters such as mutual TLS 45. Both SDKs also added workspace parser management for LakeWatch alongside the workspace-level AssumeGroup IAM service and Google Ads pipeline ingestion 2367.
Generated daily from the 10 most recent items mentioning Databricks SDK. Click any [N] to jump to the source.
Release: v3.1.0
This release reorganizes the sidebar into dedicated Connection, Project, and Models views, adding support for Unity Gateway models directly in VS Code Chat. Bundle runs and Databricks Connect sessions now default to serverless compute when unconfigured, and execution environments no longer inherit cloud provider credentials from VS Code.
This release adds parser management methods for the Lakewatch service alongside new support for Delta Sharing skills, Google Ads pipeline ingestion, and AI and vector search index statuses. Breaking changes include removing the MigrationTypeOnline enum for Postgres migrations and making the SqlQueryId field optional in job SQL condition configurations.
This release adds LakeWatch workspace parser management methods alongside support for Google Ads pipeline ingestion and Delta Sharing skills. Breaking changes make sql_query_id optional in job SQL condition configurations and remove the online migration type for PostgreSQL.
The SDK introduces the Statement Execution V2 API service alongside new methods to inspect and manage security scan findings for Databricks Apps deployments. It also adds new testing parameters for pipelines, Iceberg table support for machine learning data sources, and custom container image configuration for compute environments.
The SDK now supports the Statement Execution V2 workspace service and introduces an http_adapter configuration for custom HTTP transports like mutual TLS. It also adds deployment security scan management methods for Databricks Apps alongside new testing parameters for pipelines.
The SDK adds the workspace-level AssumeGroup IAM service and an EffectiveResources field for apps. Breaking changes make the OutputRef field required and remove the Expired status in AI Functions batches, alongside dropping the Continuous schedule field in Lakewatch.
The release introduces the workspace-level assume_group service, adds an effective_resources field for Apps, and ensures SDK User-Agent headers are sent on all Files API requests. Breaking changes make output_ref required for AI Functions batches, remove the continuous schedule in LakeWatch, and drop the expired status enum value.
Databricks Asset Bundles adds support for feature spec resources and resource grant dependencies, while preventing redeployments from resetting manually configured app compute sizes. Additionally, the AI Runtime accelerates execution through parallel snapshot compression for large Git repositories and workspace connection reuse.
The serving package adds a PtTerm field to the PtServedModel configuration. In a breaking change, the CustomContentPackPath and CustomPresetsPath fields have been removed from lakewatch CommonConfig.
The NotebookContent field for CleanRoomAssetNotebook in the Clean Rooms service is no longer required. This release also adds the OutputTypeText field to ML features, alongside ServicePrincipal and GroupName fields for Postgres configurations.
Databricks Asset Bundles now deploy registered model aliases to Unity Catalog and resolve bugs that repeatedly reapplied permissions and grants to recreated resources. The CLI also restores Databricks-specific pip magic execution in Jupyter notebooks over SSH and removes the CRITICAL compute priority class from AI Runtime.
The Go SDK introduces new workspace services for serverless compute, software artifacts, and Postgres migrations, along with materialization controls for feature engineering. It also adds endpoint scaling status fields for AI and Vector Search, while making the warehouse ID field optional for LakeWatch data sources.
Transfer endpoint discovery is now enabled by default for WorkspaceClient files, alongside new workspace services for serverless compute, software artifacts, and Postgres migrations. POST requests are no longer retried following read timeouts to prevent duplicate execution, and the warehouse ID field is now optional for lakewatch datasources.
Release v0.195.0 of the Databricks Go SDK adds an UndropMetastoreById method to the workspace-level Metastores service, which restores a dropped metastore by its ID. The release notes list no other features, fixes, or breaking changes.
Databricks SDK for Python v0.162.0 adds an undrop_metastore_by_id() method to the workspace-level w.metastores service, which lets practitioners restore a dropped metastore by its ID. This is the release's only listed change, an API addition with no fixes or breaking changes noted.
Variables used as secret values are now sensitive, so referencing them from any other field is an error, and sensitive: true extends the same protection to other secret-holding variables. Other changes include new data_quality_monitors and materialization_sets resources, stricter host and profile-name checks in databricks auth login, and a fix so dotted profile names no longer overwrite their parent profile.
v0.194.0 adds a new lakewatch package with a workspace-level w.Lakewatch service, batch methods on w.AiFunctions (CreateBatch, GetBatch, ListBatches, CancelBatch), and feature spec methods on w.FeatureEngineering (CreateFeatureSpec, GetFeatureSpec, DeleteFeatureSpec). It is a breaking change that removes the Critical value from jobs.AiRuntimeTaskPriorityClass, so code that references that constant will fail to compile.
v0.161.0 changes WorkspaceClient.files so failed uploads and downloads fall back to the next transfer endpoint (presigned URLs, then the Files API) only if no data was committed, and download_to writes through a temp file so a failed download leaves no partial file. It also adds the experimental experimental_files_ext_enable_endpoints_api option to follow the workspace's listed endpoints, deprecates experimental_files_ext_enable_storage_proxy, adds the lakewatch, ai_functions batch, and feature_engineering feature-spec APIs, and removes the critical value from AiRuntimeTaskPriorityClass (a breaking change).
Release v0.193.0 of the Databricks Go SDK adds a ViewName field to ml.IngestionDestination and a Notifications field to ml.MaterializationSet. The release contains only these API additions, with no fixes or breaking changes.
Release v0.160.0 of the Databricks SDK for Python adds a view_name field to IngestionDestination and a notifications field to MaterializationSet in the ML service. The release contains only these two API additions, with no fixes or breaking changes.
v0.192.0 adds new ComputeMigrations and GenieAgents workspace services, Sandbox file upload and download methods, and job-completion triggers and serverless compute options for Jobs. It also breaks the iamv2 API: many request types drop their account, group, user, and principal ID fields in favor of Name and Parent fields, and the AccountIamV2 and WorkspaceIamV2 methods take arguments in a new order and have new paths.
v0.159.0 adds the compute_migrations and genie_agents workspace services, sandbox file download and upload, job-completion triggers, AI function tasks, and serverless compute for job clusters. It also makes breaking changes to the IAM v2 APIs: account_id, group_id, and similar path fields are replaced by name and parent resource fields, and many method signatures have new required argument orders.
Databricks CLI v1.23.0 makes databricks ssh connect provide claude and codex on dedicated clusters and in serverless IDE sessions and non-interactive commands, and adds GPU_1xRTX6000 workload support plus workspace.sync_mode (overwrite, in-place, immutable), which replaces the deprecated presets.source_linked_deployment and experimental.immutable_folder settings. The release also reverts bundle file uploads to the legacy /workspace-files/import-file endpoint to fix notebooks being uploaded as plain files, adds lifecycle.depends_on and bundle deploy --select for deletions, and fixes bundle validate errors when a schedule, trigger, or continuous block is referenced as a whole.
Databricks SDK for Java v0.172.0 fixes CLI authentication so that small wrapper scripts, such as launcher scripts or version manager shims, are no longer mistaken for the legacy Python CLI, which now only applies to the pip entry point. The release also adds budget spend status methods to the account budgets service and an aiFunctionTask field across the jobs task types, with matching aiFunctionTaskOutput on run output.
Adds get_budget_spend_status() and list_budget_spend_statuses() to the account-level budgets service. Fixes files.download_to with use_parallel=True to download byte ranges from presigned URLs or the storage proxy instead of always using the Files API, and to request the correct path for names containing #, ? or %.
Release v0.191.0 adds SDK methods for AgentKit memory and session store permissions, budget spend status queries, feature engineering materialization sets, and starting or stopping vector search endpoints. It also adds external spend inclusion for budgets, unhealthy-table notifications for data quality, relative error options for approximate count-distinct and percentile functions, and table trigger conditions and timing controls.
Databricks SDK for Java v0.171.0 adds AgentKit permission methods for memory and session stores, Feature Engineering materialization set methods, and new fields across data quality, jobs, ML, pipelines, and sandbox APIs. The release also adds refresh-by-path selection fields to pipeline updates and a fileOutput option on jobs SQL outputs.
v0.157.0 adds Agent Kit methods for granting, revoking, listing, and getting memory and session store permissions, plus create, get, update, and delete methods for Feature Engineering materialization sets, along with new fields for data quality notifications, SQL task file output, pipeline refresh-by-path selection, and sandbox metadata. It also requires certifi 2026.10.19 or newer to pick up an updated CA trust bundle.
Release: v2.22.0 (#2272)
Release v2.22.0 stops bundle sub-project discovery from blocking the extension host and has VS Code handle proxies and certificates for Databricks SDK calls. It also fixes uv workspace member environment selection and constraint conflicts, and exposes the bundle Configuration view and Bundle Resource Explorer in Remote SSH mode.
Release v1.22.0 adds support for the features resource type, lets bundle deployment bind and unbind accept type.key references, and removes the 50-workspace limit in auth login. It also adds DATABRICKS_BUNDLE_STRICT_PROFILE_MATCH for exact profile matching, and release binaries now require macOS 13 Ventura or later.
This release adds start and stop methods for workspace-level vector search endpoints, along with new starting, stopping, and stopped status values. It also adds OneDrive as a pipeline ingestion source, an include_external_spend option for budgets, regex filtering for identity visibility, trigger conditions and timing controls for table triggers, and a relative_error parameter for approximate count-distinct and percentile functions.
Adds SdpSource to ml.DataSource and CustomerVersionLabel, MaxDbu, and ReydenClusterSizeMultiplier to SQL warehouse requests and responses. Breaking: removes ReplicationConfig from sharing.CreateShare and sharing.ShareInfo.
The Go SDK introduces methods for ML experiment issue tracking and sandbox file operations, along with support for Realtime SQL warehouses and job duration targets. Breaking changes remove the EffectiveServerlessComputeId field from job tasks and replace ReadMask with View across Agentkit memory requests.
The SDK adds file and directory management methods to the workspace sandbox service alongside duration target support across Jobs APIs. Unity Catalog connections now support a secrets field, policies include governed tags, and disaster recovery failover groups gain replication tracking fields.
The SDK adds issue tracking methods for ML experiments, a realtime warehouse type for SQL endpoints, and replication configuration support for Delta Sharing. As a breaking change, the effective_serverless_compute_id field was removed from job run tasks.
The SDK adds support for Slack and Microsoft Teams notification destinations, as well as ModelService securable types in Databricks Apps. Model provider configurations in Unity Catalog now include pricing fields and support for Bedrock Mantle.
The SDK adds account-level management for identity visibility filters and introduces statement timeout configuration for SQL warehouses. It also adds parent path support for pipelines, custom template formats for SQL alerts, and streaming shuffle partition settings for machine learning features.
This release adds the Agent Kit workspace service, account IAM identity visibility filters, and statement timeout configuration for SQL warehouses. It also introduces a breaking change by removing the Mason workspace service and package.
The Terraform deployment engine for Databricks Asset Bundles has been removed, requiring users to migrate to the direct engine or remain on CLI v1.19.x. Additional updates include a fix for large integer corruption in API commands, group-level run-as support in bundles, and expanded authentication options for Databricks Apps.
The SDK now includes the AgentKit package and workspace service, as well as a HealthCheck field for app deployments. In a breaking change, the Mason package and its workspace service have been removed.
MLflow 3.17.0
MLflow 3.17.0 introduces fine-grained access control for runs and traces, links evaluation metrics directly to managed datasets, and adds support for Databricks Unity Catalog model service resources. Self-hosted SQL tracking servers require a coordinated schema migration via mlflow db upgrade with all writers stopped, as rolling deployments are not supported.
A new account-level service is now available to manage private network gateways. Additionally, the SDK adds a project environment field for compute environments and a dataframe schema field for machine learning request sources.
The Go SDK now includes the PrivateNetworkGateways account-level service for network management. Additionally, a new DataframeSchema field is available for machine learning request sources.
The SDK introduces environment variable configurations and maintenance window notifications across Jobs and Tasks, along with new skill management APIs in AI Gateway. It also adds the Mason workspace service and makes CommandPath in job deployments and ApiSecretRef in ML schema registries optional.
The Databricks Python SDK introduces a new list_commands method to the workspace sandbox service. This update allows practitioners to programmatically list and inspect commands within Databricks sandbox environments.
The AI functions service now supports an ai_decide method, and multiple Jobs and Tasks models now accept environment variable configurations. Additionally, command_path in DeploymentSpec and api_secret_ref in SchemaRegistryConfig are no longer required fields.
The Databricks SDK for Go now supports Config.Headers to configure custom HTTP headers on all client requests in addition to normal authentication. Practitioners can use the new StaticHeaders helper to easily apply a fixed set of headers.
The Databricks Python SDK introduces skill management operations to the AI Gateway workspace client. Users can now create, retrieve, list, update, finalize, and delete skills programmatically using new methods on the AI Gateway service.
AI Runtime commands have moved from experimental to databricks air, and SSH commands now support keeping detached background processes running after tunnel disconnect. Databricks Asset Bundles direct engine resolved multiple issues around unnecessary resource recreations, Unity Catalog grant convergence, and Git-sourced Python tasks.
MLflow 2.11.5
Users can now opt in to route Unity Catalog model registry artifact uploads and downloads through the Databricks SDK Files API. This update provides an alternative transfer mechanism for managing model artifacts within Databricks environments.
This release adds support for the new Mason workspace-level service via the w.mason client. It also introduces service_credential and secret_reference fields to Unity Catalog model provider configurations.
This release adds AI Gateway credential management APIs for MCP services and expands data pipeline connectors to include TikTok Ads and Smartsheet as ingestion sources. It also introduces Avro and Protobuf transformer format support for pipelines, timezone configuration for ML job scheduling, and new securable types for agent services and skills.
The SDK adds new credential management methods for MCP services in AI Gateway and expands job notifications to include maintenance-related events. Support for additional data formats (Avro, Protobuf) in pipelines and new connection types (TikTok Ads, Smartsheet) extend integration capabilities.
Direct deployment state version 3 is a breaking change requiring Databricks CLI v1.8.0 or later. New capabilities include Docker credential helpers for Databricks Artifact Registry, AI Gateway service support in bundles, file-change job run triggers, and bundle improvements for resource reference handling, deployment reporting, and state management.
The SDK adds support for new fields across multiple services: AI Runtime jobs can now specify image paths and priority classes, ML models support budget policies and tags during publishing, and serving configurations gain workspace credential options including new GPU accelerator types. Pipelines add development mode configuration and workspace settings gain ingress network policy controls.
SSH sessions now automatically reattach and replay missed bytes when the tunnel connection drops temporarily, preventing session interruptions and eliminating manual reconnect workarounds. The CLI adds PyDABs secrets support, JSON output for aitools install with error categorization for CI automation, and displays bundle sync progress by default.
Release introduces three new workspace services (AI Functions, Domains, Sandbox), extends Feature Engineering with new backfill and operation management methods, and adds AWS Secrets Manager and Azure Key Vault connection support. Multiple breaking changes remove fields from model serving and catalog configurations that may require code updates.
The SDK adds a purge_feature_entities() method for feature engineering and introduces new deny policy type support with a deny field in PolicyInfo. New budget_policy_id and tags fields are available for IngestionConfig and MaterializedFeature to enable budget policy tracking and resource tagging.
Get Tuesday's version of this
Tracking Databricks SDK? The Tuesday email carries what moved across the whole ecosystem, not just this topic. Free, one-click unsubscribe.