A builder’s guide to scaling Databricks Apps
Summary
Scaling Databricks Apps starts with agreeing on target response times and peak user counts, then routing heavy work to the right service while app compute handles the UI and coordinates calls. Keep code responsive and state recoverable by caching per user and storing hard state in Lakebase or Unity Catalog tables, then size instances, workers and Lakebase connections from load tests and telemetry.
Summary generated by brickster.ai. For the full article, follow the source link above.
More from Databricks Blog
Which Data Collaboration Tool: OpenSharing, Clean Rooms, Marketplace
Databricks offers three ways to share data and AI assets, and the right one depends on your relationship with the other party. Use OpenSharing for live, read-only access with a known recipient, Clean Rooms when partners want to combine data without exposing all of it, and Marketplace to reach or find audiences you don't know yet.
From Delta Sharing to OpenSharing: what changed, what's next
OpenSharing evolves Delta Sharing into an open, Linux Foundation-governed protocol that extends sharing beyond tables to AI assets such as agent skills and models, so teams can share across platforms, Iceberg engines, and on-premises storage without copying data or building custom pipelines per partner. It's additive: existing Delta Sharing setups keep working, and you enable the new asset types on a Unity Catalog metastore using the same share, recipient, and grant workflow.
Push-based CDC into Unity Catalog with Zerobus Ingest and Debezium
Debezium Server and Databricks Zerobus Ingest now stream CDC events straight into Unity Catalog Delta tables using a push-based architecture, with no intermediate Kafka brokers or pull-based connectors required. Because data is pushed outward, locked-down private networks that block inbound connections can still replicate to the lakehouse, which simplifies the infrastructure needed to build current-state tables for downstream analytics.
Introducing Apache Spark® 4.3 - Now available in Databricks Runtime 19
Apache Spark 4.3, now available in Databricks Runtime 19, brings stateful processing to Real-Time Mode with streaming aggregation, deduplication, and custom stateful logic, hitting sub-150 ms P99 end-to-end latency across three benchmarks using familiar Structured Streaming APIs. It also adds SCD Type 2 support to Auto CDC in Python, SQL, and Spark Connect for history-preserving pipelines, plus Project Feather to cut execution overhead on eligible small-data queries and improvements to Spark Connect, Python, and SQL.
The Enterprise Case for Open-Weight Models on Databricks
Open-weight models now approach frontier-class intelligence at lower cost, and the post argues this gives enterprises data sovereignty and operational autonomy without a large capability tradeoff. Unity AI Gateway serves open-weight, frontier, and external models through one governed API, with fast inference, smart routing, fallbacks, and budget controls.
Declare and optimize with UNIQUE constraints on Databricks
Databricks UNIQUE constraints let you declare multiple nullable or composite uniqueness guarantees beyond a table's single PRIMARY KEY and reference them from foreign keys. Once you've validated and continuously maintained these informational constraints, marking them RELY lets the optimizer eliminate redundant joins and DISTINCT aggregations, removing over 99.99% of shuffle and speeding up queries by up to 12× in 10 TB TPC-DS benchmarks.