Pushing the Frontier for Data Agents with Genie
Summary
Databricks introduces specialized knowledge search, parallel thinking, and multi-LLM architectures in Genie to tackle the unique asset discovery and reasoning challenges of enterprise lakehouses. On real-world data analysis benchmarks, these techniques boosted Genie's accuracy from 32% to over 90% compared to leading coding agents while significantly reducing costs and latency.
Summary generated by brickster.ai. For the full article, follow the source link above.
More from Databricks Blog
Read Restrictions and Catalog Labels: Unifying governance across engines and catalogs
Apache Iceberg has introduced read restrictions and catalog labels, two new specifications designed to standardize policy enforcement across different engines and catalogs. These additions eliminate fragmented enforcement by delegating access decisions to trusted engines and making governance metadata portable across federated catalogs.
Lakebase Postgres branch-based restores for fast recovery at scale
Lakebase Postgres introduces branch-based restores, a metadata operation that instantly recovers databases by pointing to immutable history in decoupled storage rather than copying data. This capability cuts recovery times for 100 TB databases down to seconds, eliminating prolonged downtime and enabling seamless branching and undo workflows for AI agents.
Connecting customer context to measurable ROI with agentic marketing
Agentic marketing relies on AI agents grounded in real-time, governed customer context and identity resolution to recommend next-best actions within defined guardrails. Moving incrementality measurement directly into this decision loop proves what changed because marketing acted, providing a shared basis for marketing and finance investment decisions.
IP Functions are Generally Available, bringing high-performance network analytics to the Lakehouse
Native IP functions are now generally available on Databricks Runtime 18.3+, providing built-in support to parse, validate, canonicalize, and join IPv4 and IPv6 addresses and CIDR blocks without UDFs or regex. Optimized in Photon across SQL, PySpark, and Scala, these native functions run demanding network workloads in seconds and complete IP CIDR joins up to 3.1x faster and 6.4x cheaper than another leading cloud data warehouse.
Introducing ai_decide: make fast decisions on your governed data
Databricks has introduced ai_decide, a new AI Function that processes unstructured text to return structured decisions faster and at lower cost than traditional LLMs. Optimized for decision-making rather than text generation, it runs directly on governed data at batch scale in SQL or in real time over REST for tasks like prompt routing, metadata tagging, and agent quality evaluation.
A practical guide to cost optimization with Lakebase Postgres
Lakebase Postgres delivers low, predictable costs when you sync only the working subset of Lakehouse data, match sync modes to actual freshness needs, and right-size compute so hot data fits in cache. Combining these optimization practices with serverless scale-to-zero, tuned PITR, and shared storage ensures strong performance and availability without paying for unused resources.