Lakebase branching for CI/CD: lessons from 100 PR databases
Summary
Snorkel AI's CI/CD pipeline now gives every open pull request its own Lakebase Postgres branch, a full read/write copy of production data that's ready in under a minute rather than the 30+ minutes that previously made per-PR copies impractical. Because branches are copy-on-write, 100 concurrent PR databases on a ~2.5 TB production dataset add storage only for what each one changes, with TTL expiration cleaning up automatically and private connectivity plus Okta SSO keeping the setup governed.
Summary generated by brickster.ai. For the full article, follow the source link above.
Topics
More from Databricks Blog
Databricks and SAP unleash end-to-end data & AI capabilities to level up enterprise AI transformation
Databricks and SAP are expanding their partnership to give customers broader access to SAP application data across cloud and on-premises, speeding time to value for analytics and AI. Genie, Lakeflow, Unity Gateway, and Lakebase now come to SAP Databricks fully integrated with SAP Business Data Cloud, so teams can use the full Data + AI Platform on all their SAP data.
Wealth Management Client Onboarding Needs More Than AI
Task-level AI speeds up document processing and KYC screening, but wealth management client onboarding still stalls when evidence, case state, and decisions are fragmented across systems. The fix is governed case context, a shared, versioned view of each case's evidence, exceptions, and decision history, which supervised AI agents on the Databricks Data + AI Platform use with human reviewers to coordinate the full journey with traceability, and whose impact you can measure through cycle time, manual touches, and approval-to-funding time.
Deliberate decisions and enterprise AI value
Enterprise AI value depends on deliberate, grounded decisions, especially when organizational readiness, evidence and expectations vary widely. The post draws on what senior leaders are learning about investment, ownership, human judgment and strategic flexibility, and lays out 5 leadership disciplines for turning AI ambition into sustained value.
Announcing General Availability of Real-Time Mode for Spark Declarative Pipelines on Databricks
Real-Time Mode is now generally available on Spark Declarative Pipelines, delivering end-to-end processing latencies in the millisecond range for operational streaming workloads. It replaces the need for a separate engine like Apache Flink by unifying batch, ETL, and low-latency streaming in one framework, and ForEach sinks write instantly to operational stores for use cases like fraud detection and live personalization.
Smarter cost controls for serverless
Serverless cost controls in Databricks now extend beyond infrastructure settings to cover who can run workloads, how costs are attributed, and how much teams can spend, and you can already apply them today with usage-policy tags, billing system tables, access controls, and budget alerts. Rate limits are now available to help prevent runaway workloads, with shared budgets coming soon to keep team spending within bounds.
How Procore is powering construction analytics with OpenSharing
Procore moved its construction analytics product from an API-driven, once-a-day refresh to OpenSharing on Databricks, so customers can now pull data into their downstream systems and business processes hourly. OpenSharing distributes hundreds of curated tables with partitioned per-customer access across hundreds of thousands of data shares, doubling data availability for end customers and internal product teams.