How Superhuman and Databricks built a 200K QPS inference platform together
Summary
Superhuman migrated their 200K QPS custom LLM inference to Databricks FMAPI Provisioned Throughput, achieving sub-second P99 latency and offloading infrastructure management. Joint engineering delivered 60% per-GPU throughput gains and reduced serving costs through FP8 quantization and Hopper architecture optimizations.
Summary generated by brickster.ai. For the full article, follow the source link above.
More from Databricks Blog
Automate Data & KPI Monitoring with SQL Alerts
In many organizations, data monitoring is still a manual, repetitive routine: open...
How to Build Real-Time Fraud Detection using Spark Real-Time Mode and Lakebase
Build real-time fraud detection with sub-second intervention using Spark Real-Time Mode and Lakebase. This unified platform processes high-throughput data streams, executes low-latency ML models, and serves explainable fraud scores to reduce detection lag and operational complexity.
How Databricks Genie improves supply chain visibility with real-time AI analytics
Databricks Genie now offers a conversational AI layer for real-time production intelligence, giving manufacturing VPs of Operations direct access to unified operational data. This eliminates data access bottlenecks, allowing leaders to ask context-aware questions and accelerate decision-making without complex analyst requests or BI tool training.
A CFO’s guide to managing value-based care financial performance
Databricks Genie for Value-Based Financial Intelligence helps CFOs bridge the VBC Financial Intelligence Gap by enabling conversational querying of integrated clinical-financial data. This allows healthcare finance leaders to quickly monitor patient utilization, track per-member-per-month costs, and identify clinical outliers essential for managing value-based care financial performance.
Stop rogue AI: How Unity Catalog secures your agent actions
The risks of agentic AI are no longer theoretical. Agents connected to external tools...
Why AI Security Infrastructure is Now a CMO Priority
Databricks launched Lakewatch, an open, agentic SIEM built on the Lakehouse, to bring security detection to where enterprise data already lives. CMOs must now prioritize AI security infrastructure and engage in enterprise data and platform decisions to scale AI safely, as marketing teams are increasingly targeted by agentic cyberattacks.