How dbt Labs and Fivetran are moving dbt workloads onto the data lake, model by model
Summary
Teams can move dbt workloads onto an open Iceberg-based data lake incrementally, using Fivetran's managed data lake, an in-place "Move" of Snowflake-native tables to Iceberg that keeps dependent dbt models and BI dashboards working, and Lake Compute, which shifts simple dbt models onto cheaper lake-native compute one at a time. Built around an Iceberg REST catalog (Apache Polaris, with Glue or Unity Catalog integration) so Snowflake, Databricks, and BigQuery can share the same data, the approach cut roughly 15 to 20 percent from one Snowflake compute bill and targets ingestion and transformation, which the speakers estimated at around 75 percent of stack spend.
Summary generated by brickster.ai. For the full article, follow the source link above.
More from dbt Labs Blog
How to put generative AI directly into a dbt pipeline with plain SQL
Google Cloud shows how to call BigQuery's AI SQL functions from inside dbt models, so generative AI steps like sentiment scoring run as ordinary SQL transformations in the pipeline. It extends the pattern to LLM-generated product briefings, all written in plain SQL without separate orchestration code.
How Sigma runs a 2,883-model semantic layer on dbt and Dagster
Sigma manages its semantic layer as version-controlled code, running 2,883 dbt models orchestrated with Dagster. The post walks through how that code-based approach works at scale, giving Databricks practitioners a concrete reference for treating a semantic layer like any other software project.
ClickHouse + dbt: Building a GitOps workflow at Twilio
Twilio consolidated its Kafka, Spark, and Elasticsearch stack into a single ClickHouse pipeline, with dbt models and a GitOps workflow governing how changes are made. The post walks through how that setup works, which is useful for Databricks practitioners weighing how to manage analytics pipelines through version-controlled, declarative workflows.
How KOHO built a Claude-powered self-serve analytics tool on years of dbt governance
KOHO built a Claude-powered self-serve analytics tool on top of years of dbt governance, and argues that AI analytics only works when a governed dbt foundation and a strong data culture are already in place. The post walks through why that groundwork, rather than the model itself, is what makes self-serve analytics trustworthy.
Why an AI agent's revenue query was $3.5 million wrong, and how AWS fixed it with dbt Semantic Layer
AWS shows how a plausible-looking AI agent revenue query came out $3.5 million wrong, and how grounding the agent in the dbt Semantic Layer prevents that kind of error. The post argues that supplying business context, such as governed metric definitions, is what keeps AI-generated numbers trustworthy.
How DoorDash scaled merchant-facing analytics to thousands of users in weeks, on dbt and ThoughtSpot
DoorDash used a dbt-defined semantic layer to take merchant-facing analytics from pilot to thousands of users in just weeks. Learn how pairing dbt with ThoughtSpot let the team scale access without rebuilding metrics for each new audience.