dbt-databricks

Metric views now replace existing tables or views using a backup-and-create strategy rather than direct create-or-replace syntax. Lazily formatted Databricks SQL log records are also now properly interpolated when mirrored into dbt logs.

v1.12.4 fixes three bugs: case-insensitive SQL credential redaction, composite unique key deletion on DBR <17.1, and quote escaping in relation comments. databricks-sql-connector is updated to 4.4.0 to fix three CVEs including TLS hostname bypass and DoS vulnerabilities.

Adds catalog_database in v2 catalogs.yml to route Unity catalog models to independent physical catalogs (requires dbt-core 1.12+). Resolves bugs affecting Iceberg Python models, dbt clone operations, non-ASCII column incremental strategies, streaming table tags, and governed tag column drops.

Introduces catalogs.yml v2 support, a skip_optimize config for opting out of post-materialization OPTIMIZE, and Rust kernel backend for SQL warehouses. Fixes numerous incremental model bugs around constraints and tags, but now requires --full-refresh to apply changes to primary/foreign key expressions.

The SDK now caches OIDC tokens to reduce latency and rate-limiting issues, and fixes crashes on Spark Connect clusters with lazy dbutils initialization and improved runtime fallback handling. API updates include type_overrides support for synced table specs and making resource_id optional for bundle operations.

1.12.1

dbt-databricks 1.12.1 exposes job run IDs in adapter responses for correlating dbt runs with Databricks workflow executions and adds support for SPOG account-level vanity URLs. The release fixes critical issues including Iceberg incremental models losing clustering, column constraint enforcement now gated to require explicit contract.enforced: true, and various SQL warehouse compatibility problems.

v1.12.0 adds metric views, row filters, and Python UDFs as new materializations, along with SCHEDULE EVERY and TRIGGER ON UPDATE refresh modes for materialized views and streaming tables. databricks_tags now merge additively across hierarchy levels instead of child replacing parent (breaking change), and the release includes fixes for metric view validation, pydantic v1 compatibility, and streaming table refresh scheduling.

This release adds invocation_id to query comments and enforces a 255-character identifier length limit for Databricks relations. It fixes multiple issues including suppression of spurious warnings that appeared on every run and improves handling of materialized views, Iceberg, and Hudi metadata.

Using dbt with Databricks: Architecture decisions that determine success

Databricks users who skip dbt incur compounding costs. A solution architect explains key architecture decisions and when to act to ensure success.

Release v1.11.7 adds notebook-scoped packages for command and notebook job runs, and fixes workflow job handling including duplicate alias prevention and insert-by-name support for microbatch and replace_where strategies. It also improves named compute capability detection and updates dependencies for dbt-core and databricks-sql-connector compatibility.

This release enables concurrent microbatch execution and adds optimize() calls to snapshot and table v2 materializations for better performance. It also fixes issues with catalog name quoting, streaming table cluster clauses, column-level tag application, and constraint configuration handling.

dbt-databricks v1.12.0a1 introduces row filter functionality for implementing row-level access controls and adds support for metric views. These features expand SQL modeling capabilities within the Databricks environment.

Fixed a bug where multiple foreign keys from one table were lost during incremental runs. Fixed materialized view updates to use drop and recreate when partition_by configuration changes.

Version 1.10.19 updates the underlying dbt-core dependency to the latest 1.10.x release. No user-facing changes or new features are included in this maintenance release.

chore: Updating dbt-common and dbt-adapter pins for 1.10.x by @benc-db in https://github.com/databricks/dbt-databricks/pull/1300

This release updates the dbt-core dependency pin. No user-facing features, fixes, or breaking changes are included.

feat: Add query-id to SQLQueryStatus by @colin-rogers-dbt in https://github.com/databricks/dbt-databricks/pull/1280

This release updates the dbt-core upper bound, enabling compatibility with dbt-core version 1.10.16. This allows Databricks practitioners to use dbt-databricks with the latest 1.10.x dbt-core releases.

This release fixes an issue where default query tag values were not properly escaped or truncated. Databricks practitioners will now see correct handling of these values in their dbt projects.