Databricks SDK
Recent items mentioning Databricks SDK across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
The Databricks SDK for Go introduced breaking changes across IAM v2 and Bundle Deployments APIs, replacing InternalId with dedicated identity fields 1 while updating CreateDeployment signatures and removing DeploymentId 2. These releases added workspace DirectoryInfo metadata 1, job trigger PauseStatus fields 1, and Unity Catalog destination owner enums 2, while broader ecosystem updates demonstrated Python client streaming via Lakeflow Connect Zerobus Ingest 3.
Generated daily from the 3 most recent items mentioning Databricks SDK. Click any [N] to jump to the source.
Go version requirement raised to 1.25, and IAM list methods now include pagination with breaking signature changes, while CreateOperation is removed from BundleDeployments. Fixes concurrent OAuth token cache handling and Spark runtime selection for major-only versions.
The action_type and status fields in the bundledeployments.Operation model are no longer required. Additionally, the create_operation method has been removed from the workspace-level bundle_deployments service.
The SDK now supports LinkedIn Ads and Marketo as pipeline connectors and expands PostgreSQL synced table support with halfvec and varchar types. The list_workspace_assignments methods now require pagination handling, which is a breaking change for existing code calling these APIs.
This release adds pagination to four IAM methods in AccountIamV2API and WorkspaceIamV2API (list_direct_group_members and list_workspace_assignment_details), which is a breaking change requiring code updates if you use these methods. Several new fields are added across services including agent_type for Genie conversations, group_id for direct group members, effective_serverless_compute_id for job runs, and pipeline_channel for Postgres pipelines.
The SDK adds new methods for managing groups, users, service principals, and workspace assignments at both account and workspace levels, providing expanded IAM capabilities. The NewCluster field in JobCluster is now optional, a breaking change that affects existing cluster configuration code.
The SDK adds comprehensive IAM v2 methods for account and workspace-level management of users, groups, service principals, and workspace assignments. The `new_cluster` field for `JobCluster` is now optional, which is a breaking change.
* Bump `github.com/databricks/databricks-sdk-go` from v0.166.0 to v0.170.0 ([#6251](https://github.com/databricks/cli/pull/6251)).
The CLI adds `databricks environments setup-local` to provision matched Python environments for Databricks compute targets and extends `aitools install` to support Gemini CLI and Pi. Bundles fix the ignored `bundle.deployment.lock.force` setting, add pipeline `cascade_on_destroy` control, improve experimental `job_runs` with idempotency tokens and completion waiting, and add UC secrets resource support.
The SDK adds job triggers for scheduling, API source connectors for pipelines, secret value retrieval in requests, new GpuXlarge8 workload types, Netsuite connection support, and token forwarding for apps. Breaking changes remove browse-related fields from model and MCP service APIs and make several user and service principal fields required.
The Terraform deployment engine is now deprecated; migrate to the direct deployment engine for continued support. AI runtime tasks now automatically package local directories into tarballs during deployment, and the direct engine fixes several convergence issues with empty grants, webhook notification ordering, pipeline configuration, and vector search index creation.
The Databricks SDK adds principal-level overrides and resource-type tracking to billing configurations, plus a new block_usage billing action type. New features include NetSuite connection support in the catalog, grant-level policy control, and input/output column mapping for pipeline transformers.
The Go SDK adds fields for granular billing control: `PrincipalOverrides`, `ScopeType`, and `ResourceType` for alerts and budgets, plus a new `BlockUsage` action type for alert configuration. The catalog API now supports grant management for policies via a new `Grant` field and `PolicyTypeGrant` enum type.
Breaking changes include removal of direct AWS and Azure credential fields from model provider configurations and State field type changes in bundle deployment operations. New features add KinesisStreamConfig support for ML streaming sources, a Mode field for pipeline updates, and Pi coding agent attribution in the User-Agent header.
The SDK adds new fields to job triggers (PauseStatus), workspace objects (DirectoryInfo), and IAM entities (GroupId, ServicePrincipalId, UserId) for enhanced API access. A breaking change removes the InternalId field from IAM Group, ServicePrincipal, and User types, requiring code updates for any existing consumers.
The SDK introduces a new AI Gateway workspace service and adds new methods and fields across bundle deployments, compute, jobs, serving endpoints, and other APIs. Breaking changes include a modified create_deployment() method signature and removed deployment_id and lifetime fields.
The Go SDK adds AwsAccessKey authentication for Amazon Bedrock model providers and EntraServicePrincipal authentication for Azure OpenAI and Microsoft Foundry providers. These new configuration options enable additional credential management methods when integrating external AI models through Databricks.
SSH connect now supports specifying serverless usage policies via --usage-policy-id, and bundle deploy/destroy are more robust against transient app states and full workspaces. Bundle validation is stricter for grants while empty field values are now dropped, fixing deployment failures and spurious permission drift.
The release introduces a new workspace-level AI Gateway service for managing AI catalog operations. Bundle Deployments now support update operations and include additional fields for tracking version history, dashboard metadata, and modification details.
The Go SDK adds a PatchTelemetryConfig method for managing serving endpoint telemetry configuration. New fields expand pipeline schema specs with FanoutOptions and SQL alerts with Parameters support.
New special destination enum values enable more granular catalog ownership assignments at the schema, table, volume, function, and model levels. Breaking changes to BundleDeployments require updating your deployment creation calls due to argument order changes and removal of the DeploymentId field.
The SDK adds DependencyMode field support across cluster management APIs along with new fields for AI runtime tasks, schema registry configuration, and synced table management. A breaking change removes the Lifetime field from ml.TimeWindow that will require code updates.
The `databricks genie ask` command is now stable (promoted from experimental), enabling practitioners to ask natural-language questions about their data directly from the CLI. Multiple reliability and usability improvements ship, including auth profile validation timeouts, fixes for GCS-backed UC Volumes deletion, bundle validation for sql_warehouse configurations, opt-in spark_python_task file downloads in bundle generation, and instance_pools resource support in Declarative Automation Bundles.
The Databricks Go SDK now exposes effective entitlements data for workspace assignments and adds support for serverless compute IDs in job cluster configurations. These additions enable better management of workspace access controls and serverless compute resources through the SDK.
The SDK adds new fields for job performance targets, ML stream and window configuration, VPC endpoint information, and compliance standards across multiple services. Breaking changes affect IAM v2 API method paths and ML window configuration, requiring code updates to remove WindowDuration as a required field and handle the removal of LongRolling support.
The SDK adds support for new clean room task run management and PostgreSQL CDF configuration, plus numerous new fields across compute, ML, pipelines, and serving services. Breaking changes include IAM v2 method endpoint updates, type changes for internal IDs, and removal of deprecated fields in jobs and ML services.
The SDK adds workspace-level grants listing methods and new schema fields for jobs, ML experiments, serving endpoints, and workspace configuration. Breaking changes include removal of fields from secrets and ML services, and making the postgres role field required.
This release adds SDK support for JAR analysis assets, review workflows, and task run listing in Databricks Clean Rooms, alongside a new JDBC connection type for catalog integrations. Additional updates introduce Azure capacity reservation group settings for compute instance pools, initial parent paths for bundle deployments, and lifetime attributes for ML time windows.
This release introduces a breaking change by removing the CodeSourcePath field from AI runtime job tasks. It also adds the EffectiveWorkspaceId field for disaster recovery stable URLs and the SourceMetadataColumn field for Delta Live Tables pipeline configurations.
Clean Rooms API support in the Go SDK now includes fields for shared output configuration, notebook asset metadata, and collaborator aliases. Pipeline options for Google Ads, Meta Marketing, and TikTok Ads connectors now support custom
* Add `CreateCdfConfig`, `DeleteCdfConfig`, `GetCdfConfig`, `GetCdfStatus`, `ListCdfConfigs` and `ListCdfStatuses` methods for [w.Postgres](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/postgres#PostgresAPI) workspace-level service.
NewsLearn about Zerobus in 15 min!
Databricks Lakeflow Connect Zerobus Ingest is a high-performance, multi-cloud ingestion service that allows users to stream event data directly into their lakehouse without the cost and complexity of a traditional message bus. The video explains the architecture of Zerobus Ingest, announces upcoming API integrations for Kafka and MQTT, and demonstrates how to configure and run a Python client to write data directly into a Delta table.
Introduces catalogs.yml v2 support, a `skip_optimize` config for opting out of post-materialization OPTIMIZE, and Rust kernel backend for SQL warehouses. Fixes numerous incremental model bugs around constraints and tags, but now requires `--full-refresh` to apply changes to primary/foreign key expressions.
Profile-based authentication now takes precedence over environment variables, and deployment bugs affecting PyDABs-generated permissions and resource grants with --select have been fixed. New experimental job_runs resources are available, along with UC volume path support and options to hard-delete Lakebase branches or assume control of existing postgres databases on bundle deploy.
The Databricks Go SDK updated job run structures to include deployment and version ID fields. MLflow experiment objects now support trace locations, and disaster recovery stable URLs include a stable workspace ID field.
This release adds an UpdateTime field to bundle deployment resources. As a breaking change, the Role field is now required in PostgreSQL database specifications.
The SDK now supports specifying a parent path for jobs and associating a Git credential ID with workspace repositories. Model serving configurations now include CpuLarge and CpuMedium options for workload types.
Breaking changes in this release remove browse-related secret fields from Unity Catalog APIs and replace the Name field with FullName in IAM users. New functionality includes List and ListEffective methods for workspace grants, SQL condition support for Jobs triggers, and added fields across ML and Postgres services.
The SDK adds SQL conditions for job triggers, Postgres endpoint pooled hosts and autoscaling controls, Genie visualization features, and telemetry configuration for serving endpoints. The `name` field was removed from the User object, requiring code updates for any references to this field.
SSH connect adds `--base-environment` for custom base environments, and aitools install now uses plugins instead of raw skills. Bundle deployments fix drift on model serving endpoints and failed migrations on permissioned resources.
* Add `Spec` field for [environments.WorkspaceBaseEnvironment](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/environments#WorkspaceBaseEnvironment).
The SDK adds cluster policy enforcement capabilities, including a new cancel_pending_cluster_enforcement() method and related compliance tracking fields, plus AI Runtime Task support across job submission and execution APIs. The replicate_workspace_assets field in WorkspaceSet is no longer required, a breaking change that may impact workspace replication configurations.
Workspace exports now handle illegal filenames gracefully, and SSH sessions default to bash with workspace-home startup. Bundle deployments fix job URL access for non-admin users and prevent postgres roles from being recreated on each deploy.
This release introduces new methods, fields, and event types to support pending cluster policy compliance enforcement, while making the ReplicateWorkspaceAssets field optional for disaster recovery workspace sets. Additionally, it fixes a bug in the user agent builder to prevent the User-Agent header from growing indefinitely when identical key-value pairs are repeatedly injected.
This release introduces AI runtime task support to the jobs service by adding new fields to job tasks, submitted tasks, run tasks, resolved values, and run outputs. Additionally, a new bundle root path field has been added to the workspace info struct within the bundle deployments service.
This release adds a new Xlarge compute size option to the Databricks Apps service. It also introduces a meta-harness user-agent dimension that automatically detects and reports the omnigent meta-harness environment variable.
The SDK now provides Data API methods for PostgreSQL in Databricks workspaces. Other additions include capacity reservation support for Azure compute, serverless compute options for pipelines, endpoint ID fields for vector indexes, and Dynamics 365 as a connection type.
This release adds a new ServerlessComputeId field to Delta Live Tables pipeline configurations in the Go SDK. Practitioners can now use this field when creating, editing, cloning, or specifying the configuration of serverless pipelines.
This release adds new methods for managing Postgres data APIs at the workspace level. Compute clusters can now specify an Azure Capacity Reservation Group in their attributes.
This release fixes an issue where tag policy and assignment operations failed for hierarchical tag keys containing slashes. It also introduces a breaking change by making the `ResourceId` field optional in `bundledeployments.Operation` and adds support for Dynamics365 connection types and `EndpointId` in vector search indexes.
The Databricks SDK for Go now includes a TypeOverrides field for database.SyncedTableSpec. This field is also added to postgres.SyncedTableSyncedTableSpec.
1.12.1
dbt-databricks 1.12.1 exposes job run IDs in adapter responses for correlating dbt runs with Databricks workflow executions and adds support for SPOG account-level vanity URLs. The release fixes critical issues including Iceberg incremental models losing clustering, column constraint enforcement now gated to require explicit `contract.enforced: true`, and various SQL warehouse compatibility problems.
This release adds an `AcceleratedSync` field to `database.SyncedTableSpec` and `postgres.SyncedTableSyncedTableSpec`. This new field enables configuration of accelerated sync for synced tables within Databricks.
The Databricks Python SDK v0.116.0 adds new workspace services for AI Search and Bundle Deployments, along with configuration options for Vector Search facets, Delta Live Tables auto-clustering, and custom catalog retention hours. This release also introduces a breaking change by removing the legacy bundle workspace service and package.
This release introduces new services for AI Search and Bundle Deployments, along with several new fields across existing services like Catalog, ML, and Pipelines. A breaking change removes the old Bundle package and its associated workspace-level service.
The SDK now detects the AI_AGENT environment variable for user agent reporting and passes unrecognized agent values through. Pagination factory methods have been added to explicitly define token or offset strategies, deprecating the old constructor. A bug was fixed where token-paginated results were silently dropped if an empty page with a next_page_token was encountered, ensuring list methods now page until the token is absent.
Databricks Asset Bundles now features a select flag for plan and deploy commands to target specific resources alongside their transitive dependencies. The experimental open command now supports all bundle resource types with workspace URLs
The Databricks SDK for Java now correctly handles OAuth token exchanges for browser-based flows where the client ID is not present in the IdP JWT. It no longer throws a NullPointerException and omits the client_id parameter for account-wide token federation.
Databricks Asset Bundles now support a DeploymentMode field for both Deployment and Version objects. Workspace settings include new fields for CollaborationPlatformConnectivity and EffectiveCollaborationPlatformConnectivity.
The SDK now allows specifying an account target for the discovery flow, directing users to the account selector. New fields were added for job and pipeline deployments, token management, and bundle operations, alongside a new `UpdateTokenManagement` method.
