Model Serving
Recent items mentioning Model Serving across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Databricks' SDKs are expanding model serving's infrastructure surface: the Python SDK adds a gpu_xlarge_8 workload type for serving endpoints 3, while the Java SDK extends serving telemetry configs with table-name and telemetry-profile-ID fields for better usage tracking 5. Model serving is also solidifying as the inference layer in reference architectures, sitting alongside Lakebase and AI gateways to power both ML and LLM workloads in Databricks' energy-theft detection demo 4.
Generated daily from the 5 most recent items mentioning Model Serving. Click any [N] to jump to the source.
NewsHow AI and Data Keep 2.3 Million Lawns Healthy | TruGreen & Databricks
TruGreen uses Databricks Genie and Lakehouse to manage 2.3 million lawns with AI that optimizes service timing and predicts customer churn using weather, soil, and service data. The system enables non-technical branch managers to take daily actions through customized reports without requiring data expertise.
The CLI adds `databricks environments setup-local` to provision matched Python environments for Databricks compute targets and extends `aitools install` to support Gemini CLI and Pi. Bundles fix the ignored `bundle.deployment.lock.force` setting, add pipeline `cascade_on_destroy` control, improve experimental `job_runs` with idempotency tokens and completion waiting, and add UC secrets resource support.
NewsHow FOX Sports Uses AI to Power Search
Fox Sports rebuilt their search system on Databricks to handle rapidly changing sports information by continuously streaming player, team, and content data into the index while computing real-time trends. The system uses semantic vector search with time-weighted ranking to surface fresh content higher, doubling the rate at which users find what they're looking for.
TutorialsDetect Energy Theft Faster with Genie
Databricks demonstrates an end-to-end AI application that detects energy theft, automates investigations, and generates executive reports using Unity Catalog and Genie. The video walks through an architecture featuring Lakebase for transactional storage, model serving for machine learning and LLMs, and AI gateways for governance and cost control.
This release adds effective entitlements to workspace assignment details and serverless compute ID support for job clusters. It also updates model serving telemetry configurations to include fields for table names and telemetry profile IDs.
What happens in the milliseconds after you tap pay
This sample Databricks App demonstrates how to achieve low-latency real-time fraud scoring by pairing route-optimized Model Serving with Lakebase Postgres for online feature lookups. Under load testing of 5,000 requests, this architecture achieved end-to-end latencies of 27 ms at p50 and 37 ms at p95 while maintaining a 100% success rate.
The SDK now supports specifying a parent path for jobs and associating a Git credential ID with workspace repositories. Model serving configurations now include CpuLarge and CpuMedium options for workload types.
The Java SDK adds a parent path field for job creation and settings, along with git credential ID support for workspace repositories. It also introduces CPU_MEDIUM and CPU_LARGE workload type enums for Model Serving.
SSH connect adds `--base-environment` for custom base environments, and aitools install now uses plugins instead of raw skills. Bundle deployments fix drift on model serving endpoints and failed migrations on permissioned resources.
Databricks and NVIDIA: Building for the Agentic Era
Databricks and NVIDIA are expanding their collaboration to deliver an end-to-end AI platform, accelerating model training, inference, and agentic AI development on governed enterprise data. This includes multinode training in AI Runtime, GPU support in Databricks Free Edition, Model Serving Enhancements, and support for NVIDIA Agent Toolkit and industry-specific AI frameworks.
What’s New in the AI Platform: Agents for ML Engineering, Our Deep Learning Platform, and New Capabilities for Real-Time ML
Databricks shipped Genie Code, a coding agent for ML engineering, and AI Runtime, a serverless GPU platform for deep learning. Power real-time ML at scale with new Feature Store and Model Serving capabilities, including streaming features and high-QPS serving.
How ERGO Hestia reduced time-to-market with Lakebase and Mosaic AI Model Serving
ERGO Hestia modernized its real-time pricing engine with Databricks Lakebase and Mosaic AI Model Serving, reducing time-to-market by unifying data, features, and decisions for millisecond pricing. This eliminated extraction overhead and fragmented governance from their previous multi-hop architecture, enabling faster model deployment and instant market response.
NewsBanks' Secret Weapon Against Money Laundering: Multi-Agent AI
Databricks demonstrates a multi-agent AI solution for Anti-Money Laundering (AML) operations, significantly reducing false positives and accelerating investigation cycles from hours to minutes. The platform unifies siloed systems, employs specialized AI agents for analysis and recommendations, and offers AI-assisted SAR generation and executive-level reporting with natural language chat.
NewsDatabricks Apps vs Model Serving: Authentication, Cost, and Performance Compared
Databricks Apps are now the recommended first choice for deploying agents due to their flexibility in handling full-stack applications with multiple components, offering faster iteration and local testing compared to Model Serving. Model Serving remains suitable for use cases prioritizing high QPS, governance features like AI Gateway, inference tables, and guardrails, or when scaling to zero is acceptable for cost optimization.
MLflow 3.11.1 introduces AI-powered issue detection in traces, AI Gateway budget alerts and spending controls, trace graph visualization, native Databricks gateway provider, and pickle-free model serialization. TypeScript SDK packages are now @mlflow-scoped and LiteLLM is no longer required for GenAI evaluation.