Databricks AI Search
Recent items mentioning Databricks AI Search across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Streaming Tables, Materialized Views, and Iceberg v3+ managed tables are now natively supported in Databricks AI Search without requiring separate synchronization pipelines 9. Complementing this data source expansion, recent Go and Java SDK releases added endpoint scaling status fields and detailed status tracking for AI and vector search indexes 1234.
Generated daily from the 10 most recent items mentioning Databricks AI Search. Click any [N] to jump to the source.
This release adds parser management methods for the Lakewatch service alongside new support for Delta Sharing skills, Google Ads pipeline ingestion, and AI and vector search index statuses. Breaking changes include removing the MigrationTypeOnline enum for Postgres migrations and making the SqlQueryId field optional in job SQL condition configurations.
The SDK introduces parser management methods for the Lakewatch service and makes sqlQueryId optional in job SQL condition configurations. It also adds Google Ads pipeline ingestion support, skill sharing via Delta Sharing, and detailed status tracking for Vector Search and AI Search indexes.
New services for software artifacts and serverless compute are now available, alongside client methods for Postgres migrations and feature engineering materializations. The warehouse ID field is no longer required for Lakewatch datasources, and new scaling status fields have been added to AI and vector search endpoints.
The Go SDK introduces new workspace services for serverless compute, software artifacts, and Postgres migrations, along with materialization controls for feature engineering. It also adds endpoint scaling status fields for AI and Vector Search, while making the warehouse ID field optional for LakeWatch data sources.
Release v0.191.0 adds SDK methods for AgentKit memory and session store permissions, budget spend status queries, feature engineering materialization sets, and starting or stopping vector search endpoints. It also adds external spend inclusion for budgets, unhealthy-table notifications for data quality, relative error options for approximate count-distinct and percentile functions, and table trigger conditions and timing controls.
Release v0.170.0 adds startEndpoint() and stopEndpoint() methods for vector search endpoints, along with new STARTING, STOPPING, and STOPPED status values. It also adds OneDrive ingestion support for pipelines, an includeExternalSpend option for budgets, and new relativeError, TableTrigger, and regex fields.
This release adds start and stop methods for workspace-level vector search endpoints, along with new starting, stopping, and stopped status values. It also adds OneDrive as a pipeline ingestion source, an include_external_spend option for budgets, regex filtering for identity visibility, trigger conditions and timing controls for table triggers, and a relative_error parameter for approximate count-distinct and percentile functions.
Direct deployment state version 3 is a breaking change requiring Databricks CLI v1.8.0 or later. New capabilities include Docker credential helpers for Databricks Artifact Registry, AI Gateway service support in bundles, file-change job run triggers, and bundle improvements for resource reference handling, deployment reporting, and state management.
The Terraform deployment engine is now deprecated; migrate to the direct deployment engine for continued support. AI runtime tasks now automatically package local directories into tarballs during deployment, and the direct engine fixes several convergence issues with empty grants, webhook notification ordering, pipeline configuration, and vector search index creation.
SSH connect now supports specifying serverless usage policies via --usage-policy-id, and bundle deploy/destroy are more robust against transient app states and full workspaces. Bundle validation is stricter for grants while empty field values are now dropped, fixing deployment failures and spurious permission drift.
The CLI improves SSH error reporting with server logs, extends timeouts to 45 minutes for GPU accelerators, fixes environment variable handling to prevent default profile fallback, and corrects Cursor integration hanging. Bundle deployments prevent WAL corruption after failed deploys, fix resource references used multiple times within the same field, add postgres_roles and postgres_databases resources for Lakebase branches, and eliminate spurious recreates when names are backend-normalized.
This release adds new Postgres data API methods to the workspace client, introduces Dynamics 365 as a catalog connection type, and adds capacity reservation and sync configuration fields across compute, database, and vector search services. Additionally, the resource ID field in bundle deployment operations is no longer required, which represents a breaking change.
This release fixes an issue where tag policy and assignment operations failed for hierarchical tag keys containing slashes. It also introduces a breaking change by making the ResourceId field optional in bundledeployments.Operation and adds support for Dynamics365 connection types and EndpointId in vector search indexes.
The direct deployment engine is now generally available and the default for new bundle deployments, while existing deployments continue using their current engine. The CLI adds a quickstart command, version checking, authentication verification fixes for account console profiles, and a new genie_spaces bundle resource for managing Databricks Genie spaces.
This release introduces the AI Search and Bundle Deployments services to the workspace client, while removing the legacy bundle service and package. Additionally, it adds new configuration fields across multiple services, including custom retention hours for catalogs and schemas, auto-clustering options for pipelines, and facet filtering for vector search queries.
The Databricks Python SDK v0.116.0 adds new workspace services for AI Search and Bundle Deployments, along with configuration options for Vector Search facets, Delta Live Tables auto-clustering, and custom catalog retention hours. This release also introduces a breaking change by removing the legacy bundle workspace service and package.
Databricks Asset Bundles now includes a select flag in the direct deployment engine to plan and deploy specific resource subsets, along with support for Terraform references and automatic transient HTTP retries. The experimental open command expands to support additional workspace resources such as volumes and vector search endpoints, while notebook task paths now correctly preserve Lakeflow Designer files.
The provider adds service principal Git credential management via principal_id on databricks_git_credential and enables permission management for Agent Bricks resources. Key fixes include metastore external_access_enabled now properly sent in PATCH requests, vector search index timeout increased to 75 minutes and made configurable, and workspace_id now accepting connection IDs alongside numeric workspace IDs.
The error reported when a direct-only resource (catalogs, external locations, vector search endpoints) is used with the terraform engine now also suggests setting bundle.engine: direct in databricks.yml, in addition to the DATABRICKS_BUNDLE_ENGINE environment variable (#5295).
This release introduces a breaking change that renames the min_qps configuration and CLI flag to target_qps for vector search endpoints, and it stops applying bundle name prefixes to these endpoints. Additionally, it improves the interactive authentication profile pickers, enhances OS keyring token storage behavior, and fixes bundle bugs related to nested notebook directory generation and multi-profile authentication propagation.
This release fixes an issue where the Databricks CLI --profile fallback was broken. It also introduces new API methods for workspaceClient.supervisorAgents() and workspaceClient.vectorSearchEndpoints(), along with several breaking changes related to Example and Tool fields, and minQps in Vector Search endpoints.
This release introduces new methods for managing supervisor agents and vector search endpoints, along with several new fields for connector options, ingestion sources, and customer-managed keys. Breaking changes include the removal of min_qps fields from vector search endpoint configurations and making guidelines and description fields optional in certain services.
Get Tuesday's version of this
Tracking Databricks AI Search? The Tuesday email carries what moved across the whole ecosystem, not just this topic. Free, one-click unsubscribe.