Enhancing Agent Retrieval with Structured Chart Extraction
Summary
Extracting charts as structured JSON with ai_parse_document improves document retrieval and agent answer accuracy on chart-based questions while outperforming large multimodal embedding models. By indexing these extractions with lightweight 300M-parameter text embeddings via Databricks AI functions, practitioners can enable tools like Databricks Genie to reliably resolve fine-grained numerical queries from visual documents.
Summary generated by brickster.ai. For the full article, follow the source link above.
More from Databricks Blog
How California built its Behavioral Health Public County Profile on Databricks
California launched the Behavioral Health Public County Profile, a public-facing dashboard powered by Databricks that unifies fragmented behavioral health data across all 58 counties. Built to support the Behavioral Health Services Act, the platform establishes standardized metrics, robust governance, and end-to-end data lineage across disparate county systems to transparently track local funding, service delivery, and homelessness initiatives.
Solving defense supply chain visibility through governed data sharing
Governed data sharing enables defense supply chain visibility across 200,000+ suppliers by allowing every tier to publish validated evidence directly from existing records without exposing source systems. Powered by open protocols and existing reporting obligations, this federated approach eliminates manual data calls while avoiding vendor lock-in and costly platform overhauls.
Announcing Workday Data Connect federation in Unity Catalog
The new Workday Data Connect connector (Beta) brings zero-copy federation to Unity Catalog, allowing teams to query Workday's shared Iceberg tables directly from cloud storage without ingestion pipelines or data duplication. Queries run on Databricks compute under Unity Catalog governance, enabling you to combine live HR and financial data with existing Databricks datasets for Genie-powered exploration, workforce analytics, and AI.
Introducing Funke: Native HL7v2 Parsing on Databricks
Funke is an open-source Python and PySpark library that parses HL7v2 electronic health record messages directly into native Spark types on the Databricks Lakehouse while preserving their complete message hierarchy. Rebuilt around Unity Catalog, Declarative Automation Bundles, and Spark Declarative Pipelines as the successor to Smolder, it includes a runnable demo to help you stand up an end-to-end HL7 ingestion pipeline in minutes.
Lakebase and Agentic SDLC: Branching Databases for Coding Agents
Lakebase resolves the database bottleneck for parallel coding agents by providing sub-second, scale-to-zero copy-on-write database branching for each agent. Learn how to implement an end-to-end workflow pairing Claude Code, Git worktrees, and GitHub Actions to run Drizzle migrations, deploy preview environments on Databricks Apps, and test against Unity Catalog-masked data.
Biomedical Imaging's Real Bottleneck Is the Data, Not the Model
The primary bottleneck in medical imaging AI is fragmented data infrastructure rather than model architecture. Scaling clinical and R&D impact requires a governed lakehouse foundation to centralize imaging assets, de-identify scans at scale, and link them directly with EHR, omics, and trial data.