Fast, fault-tolerant PyTorch training on AI Runtime
Summary
Fast, fault-tolerant PyTorch training on AI Runtime treats GPU failures as the expected case, using torch's distributed asynchronous checkpoint saves to make frequent checkpointing nearly free and cut recovery cost. It also argues that checkpointing the data pipeline alongside the model is essential, since model-only checkpoints can silently corrupt training data on resume.
Summary generated by brickster.ai. For the full article, follow the source link above.
Topics
More from Databricks Blog
Building for the AI Era: Lakebase, Streaming, and Lakehouse Innovations at VLDB 2026
Databricks is bringing Lakebase, Structured Streaming, and Lakehouse optimizations to VLDB 2026, spotlighting Lakebase as a third-generation cloud database that decouples transactional compute from storage to support agentic workflows. The company's Engineering and Recruiting teams will also be on-site at the conference.
What QSR reports miss about the decisions matter the most
The Lovelytics QSR Executive Performance Control Tower on Databricks links a missed-promotion metric to its likely root cause—guest demand, franchise participation, ingredient availability, or restaurant execution—rather than just flagging that plan wasn't hit. That shared view of cause and value at stake lets corporate and franchisees act on restaurant-level margin before the next planning cycle closes.
Enhancing Agent Retrieval with Structured Chart Extraction
The MotivationMore and more enterprises are now asking agents to work with their...
Vertical Advantage: Transforming Industries with Lakebase and Agentic AI
Databricks Lakebase partners are shipping production-ready, industry-specific solutions—from fraud and regulatory-change agents to real-time claims, prior authorization, dynamic pricing, and grid intelligence—built on a single governed foundation for operational data, analytics, and AI. Powered by Lakebase's serverless Postgres, sub-10ms operational serving, zero-copy branching, and agent-native memory, these solutions are deployable today and designed to turn platform capability into measurable business value.
Object Storage + WAL: Lakebase Postgres for the agentic era
Lakebase Postgres now decouples storage from compute by pairing object storage with a write-ahead log, targeting the I/O bottlenecks agentic workloads create in traditional OLTP setups. The architecture is built for new deployments handling agent-driven database interactions rather than retrofitted onto existing ones.
Introducing Governance Hub: Intelligent, account-level governance over your Databricks estate
Governance Hub gives Databricks admins a single, account-level view of data governance, AI usage, and cost across their entire estate, with each vertical surfacing KPIs and drill-downs so teams can assess governance health without building custom dashboards. A built-in Genie layer adds agentic insights, surfacing anomalies and governance gaps enriched with context about the underlying assets and workloads.