Evaluating AI Agents Live at the Grounded Reasoning Cup
Summary
Stanford's team won the Grounded Reasoning Cup with 63.3% accuracy on OfficeQA Pro V2, a new 120,000-page U.S. Treasury document benchmark, using an end-to-end agent optimization approach combining reusable skills, document-representation fallbacks, and adaptive verification. Across all 11 academic teams, out-of-the-box frontier agents averaged under 30% accuracy, showing that agent performance tuned on one benchmark doesn't reliably generalize to a new corpus.
Summary generated by brickster.ai. For the full article, follow the source link above.
More from Databricks Blog
How Databricks Feature Store serves features with sub-second freshness
Databricks Feature Store now delivers streaming feature updates from Kafka to the online store at 200ms p99 latency, cutting feature staleness from minutes or hours down to milliseconds. The gains come from Spark Real-Time Mode's continuous per-event processing and amortized checkpointing paired with Lakebase's compute-storage separation, which together enable low-latency, high-throughput writes for real-time model inference.
The prototyping tax is killing your AI roadmap
Platform-native agents grounded in business semantics—like Databricks Genie plus Unity Catalog—eliminate the "prototyping tax" of fragmented context and siloed domain knowledge, hitting 77% accuracy versus 56–72% for general coding agents at roughly half the cost. Abacus Insights proved the approach even in regulated settings, cutting new-client onboarding time by about 50% and manual data-mapping effort by 40% inside a HIPAA-grade environment.
Using AI_Functions in Your Data Warehouse: Top Use Cases
In most organizations, data warehouses hold structured data, while unstructured data...
How Scottish Water Made Its Capital Investment Data Conversational With Databricks Genie
Scottish Water replaced fragmented, specialist-dependent reporting with Databricks Genie and governed data layers, letting teams query capital investment data in natural language and get trusted answers on demand. By surfacing this through Microsoft Teams via Copilot, the utility put governed insights directly into the tools business users already work in. Client.listTools() called but server does not advertise tools capability - returning empty list
What are AI Hallucinations?
What are AI hallucinations, why does it matter, and what can enterprises do about it? Newer reasoning models from OpenAI and DeepSeek are actually hallucinating more than their predecessors, not less, making detection and prevention a must for any production deployment. Enterprises can curb the risk with retrieval-augmented generation, domain-specific fine-tuning, systematic evaluation frameworks, and strong data governance.
Smart Routing in Unity AI Gateway: Match frontier quality with 30%+ lower cost per task
Smart Routing in Databricks Unity AI Gateway automatically directs coding tasks to the optimal model and harness across the price/performance frontier, matching frontier-level quality while cutting cost per task by more than 30%. The feature addresses the growing diversity of models and harnesses practitioners face by handling that selection for them rather than requiring manual tuning.