From Days to Seconds — Reducing Query Times on Large Geospatial Datasets by 99%
Summary
The Global Water Security Center reduced geospatial query times from days to seconds by implementing Databricks medallion architecture with H3 spatial indexing and autoloader data ingestion on 30-billion-row datasets. This achieved a 99% query time reduction, saving approximately $1 million annually in labor costs and enabling analysts to self-serve analysis workflows through automated notebook orchestration.
Summary generated by brickster.ai from the video transcript.
More from Databricks
NewsHow Jio Platforms move insights into production with Databricks | Data + AI Executive Series: Fast 5
Jio Platforms relies on Databricks Lakehouse to unify data and AI governance and achieve end to end lineage across its businesses including telecom, retail, media, and finance. The company uses Databricks Genie and Lakehouse to accelerate insights to production and allow business users to define their own KPIs and metrics.
TutorialsMigrate Databricks classic workloads to serverless with Genie Migrate | Databricks Demo
Genie Migrate is a Databricks tool designed to help users migrate workloads, including notebooks and jobs, from classic clusters to serverless. The tool automatically analyzes code and cluster infrastructure, identifies compatibility issues, and provides suggestions or applies fixes to facilitate the transition to serverless environments.
ReleasesMulti-Statement Transactions in Unity Catalog: Atomic Writes on Delta and Iceberg
Multi statement transactions in Unity Catalog allow atomic writes across multiple Delta or Iceberg tables, ensuring that updates either succeed completely or are aborted together. This capability eliminates the need for complex reconciliation logic, separate governance models, or data copying between systems.
ReleasesAutomate Data Warehouse Migration with AI - Genie Migrate for Lakehouse Intro
Genie Migrate for Lakehouse is an AI-powered migration solution that automates the movement of data and code from legacy data warehouses to the Databricks Lakehouse. It converts SQL from seven major data warehouses into open ANSI SQL and PySpark, reducing delivery timelines and ensuring high conversion accuracy.
TutorialsHow To Schedule Data Refresh in Excel | Databricks Add-in Demo
The Databricks Excel add-in allows users to connect Databricks workspaces to Excel spreadsheets, enabling data imports. Users can create scheduled refreshes to automatically update their spreadsheets with the latest data from their lakehouse.
