No one needs to care about table formats with Databricks' Ryan Blue, creator of Apache Iceberg
Summary
Databricks announced the GA release of Iceberg v3, which unifies data layers so files can be shared across Delta and Iceberg tables without rewriting. The company is also working towards a unified metadata layer in Delta 5 and Iceberg v4, aiming for a full unification vision later this year.
Summary generated by brickster.ai from the video transcript.
More from Databricks
NewsWhat’s coming next to Free Edition
Databricks announces the availability of Genie, GPUs, Agent Hooks, Lakehouse, and Lake Flow Designer on its Free Edition. This update provides virtually all of Databricks' production platform features for free, enabling users to learn and build data and AI projects.
NewsDatabricks + Panther: advancing the security lakehouse
Databricks acquired Panther, a cybersecurity company that developed an AI SOC platform to make security teams smarter and faster. Panther's platform helps security teams detect and respond to threats and manage compliance by leveraging large datasets without requiring deep data infrastructure expertise.
CommunityDefending against a tidal wave of AI attacks with Lakewatch, the agentic security Lakehouse
Databricks introduces LakeWatch, an agentic security lakehouse designed to defend against AI-powered attacks by enabling security and data teams to work together. It allows for the ingestion and analysis of all security data, automates detection rule creation and incident investigation using AI agents, and breaks down proprietary data silos.
ReleasesIntroducing CustomerLake – the agentic CDP built in Databricks
Databricks introduces CustomerLake, an agentic Customer Data Platform (CDP) built within the Databricks Lakehouse, designed to power "infinity campaigns" for real-time, one-to-one customer personalization. CustomerLake features Profile Agents for automated customer 360 data creation and Campaign Agents for generating and managing continuously adapting, personalized marketing campaigns.
NewsAgentic machine learning with Genie Code (includes demo)
Databricks announces Genie Code for machine learning, an enhanced coding agent that leverages an ontology of an organization's ML projects to build, evaluate, and deploy models following team best practices. They also introduce Genie Zero Ops for machine learning, an agent that autonomously monitors production models, identifies root causes of issues, and proposes solutions for human approval.
EventsInside Lakebase: fully-managed serverless Postgres – Nikita Shamgunov, VP, Engineering, Databricks
Lakebase is a fully-managed serverless Postgres database that runs on a data lake, offering familiar, nimble, and mission-critical features. It achieves high scalability, low latency, and cross-cloud disaster recovery by decoupling compute and storage, re-architecting storage with safekeepers and page servers, and integrating with the lake.