Description
Stepping into this brave new digital world we are certain that data will be a central product for many organizations. The way to convey their knowledge and their assets will be through data and analytics. Delta Sharing was the world's first open protocol for secure and scalable real-time data sharing. Through our customer conversations, there is a lot of anticipation of how Delta Sharing can be extended to non-tabular assets, such as machine learning experiments and models. In this session, we will cover how we extended the Delta Sharing protocol to other sharing workflows, enabling sharing of ML models, arbitrary files and more. The development resulted in Arcuate, a Databricks Labs project with a data sharing flavor. The session will start with the high-level approach and how it can be extended to cover other similar use cases. It will then move to our implementation and how it integrates seamlessly with Databricks-managed Delta Sharing server and notebooks. We finally conclude with lessons learned, and our visions for a future of data sharing and beyond Talk by: Vuong Nguyen and Milos Colic Connect with us: Website: https://databricks.com Twitter: https://twitter.com/databric…
Description from YouTube. Full content on the video page.
Topics
More from Databricks
ReleasesDatabricks launches across the Data + AI stack in 90 seconds
Databricks announced LTAP to unify lakebased and lakehouse data, eliminating ETL and enabling a single copy of data for analytical and operational needs. They also introduced Unity AI Gateway for governance, Genie Ontology for enterprise knowledge graphs, and open-sourced Omniant for managing multiple coding agents.
ReleasesIntroducing Omnigent: The Ultimate Meta-Harness for AI Agents
Omnigent is a new open-source meta-harness for AI agents that provides a unified interface for composition, control, and collaboration across multiple models and agent workflows. It enables stateful, data-centric policies for guardrails and allows real-time sharing and steering of live agent sessions with teammates.
NewsHow DEFRA and Natural England Accelerate Peatland Restoration with AI and Databricks
DEFRA and Natural England utilize AI and Databricks to accelerate peatland restoration by automating the mapping of peatland features and peat dams across England. This technology significantly reduces the time required for mapping, enabling faster identification and restoration of these crucial carbon-storing habitats.
NewsAI Stack Explained in 3 Layers (LLM, Agent Harness, Omnigent)
The AI stack now includes a third layer, the meta harness, which sits above individual agent harnesses. This meta harness, exemplified by Databricks' open-sourced Omnigent, allows for routing queries to appropriate agents and orchestrating tasks across multiple agents, enabling seamless interaction and context sharing between them.
NewsWhat’s coming next to Free Edition
Databricks announces the availability of Genie, GPUs, Agent Hooks, Lakehouse, and Lake Flow Designer on its Free Edition. This update provides virtually all of Databricks' production platform features for free, enabling users to learn and build data and AI projects.