Cost Optimization
Recent items mentioning Cost Optimization across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Recent discussions highlight significant cost savings when migrating to or optimizing on Databricks. One user reported a 77% cost reduction per run after migrating from Snowflake to Databricks in 12 weeks 1, while Octopus Energy achieved a 50x cost reduction in their data engineering pipelines by leveraging Databricks Serverless and Delta Lake Change Data Feed 2. For those looking to optimize, two recent ebooks offer guidance on Databricks cost optimization 34.
Generated daily from the 4 most recent items mentioning Cost Optimization. Click any [N] to jump to the source.
NBCUniversal’s Seamless Migration: Unlocking Scalable Analytics with Databricks
NBCUniversal cut costs by 30% by moving from a slot-based reservation model to Databricks' dedicated job compute, letting parallel pipelines scale independently and consistently hit SLA targets. Partner EXL executed the migration in phases with custom accelerators for code conversion, data migration, and automated validation, giving NBCUniversal a unified platform for ML development, real-time analytics, and collaborative data engineering.
The Open-Weight Revolution: A Game Changer for Our LLM Cost Optimization Odyssey
Scaling for MHHS: how Octopus Energy achieved a 50x cost reduction in margin data engineering
Octopus Energy achieved a 50x cost reduction in their margin data engineering pipelines by re-architecting on Databricks for UK MHHS regulation. They leveraged Delta Lake Change Data Feed and Databricks Serverless to process 48x more data at a fraction of the original cost, improving freshness from weekly to daily.
NewsDatabricks Apps vs Model Serving: Authentication, Cost, and Performance Compared
Databricks Apps are now the recommended first choice for deploying agents due to their flexibility in handling full-stack applications with multiple components, offering faster iteration and local testing compared to Model Serving. Model Serving remains suitable for use cases prioritizing high QPS, governance features like AI Gateway, inference tables, and guardrails, or when scaling to zero is acceptable for cost optimization.
NewsGPU Accelerated Spark Connect
This video demonstrates how to accelerate Spark Connect using GPUs for both Spark SQL and ML workloads. It details the architecture, deployment, and benchmark results showing significant speedups and cost savings compared to CPU-only execution.
