Transforming Data Pipeline Management With a Targeted Proof of Concept
Summary
Capital One's credit line increase program required processing trillions of records across 160 customer behavior and financial features using data from multiple sources, but their existing compute infrastructure was timing out. By migrating to Databricks' Photon processing engine, they achieved an 80% reduction in time and cost per job, completing operations 60 times faster.
Summary generated by brickster.ai from the video transcript.
Topics
More from Databricks
NewsHow HSS Offloads Provider Administrative Burden with the Databricks Data + AI Platform
NewsHow NOL Universe Scales Global Travel Operations for 1,000+ Users With Databricks
NewsHow adidas Uses Databricks to Build Better Products
Adidas uses Databricks' lakehouse platform to centralize all its data—from product to football-related insights—enabling faster analytics across the organization. The company's Genie analytics tool helps analysts spend less time processing data and more time on strategic questions, ultimately supporting better product development.
NewsDatabricks for Good x MapAid: Creating a Searchable Database for Groundwater Discovery
MapAid is using Databricks to build an AI groundwater mapping system called Well Mapper that processes water documents to identify accurate well-drilling locations in Ethiopia, improving on the current 30% success rate. The system analyzes 400 well logs in half a second instead of four weeks, with potential to double Ethiopia's food supply through improved irrigation.