Model Serving
Recent items mentioning Model Serving across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.
Databricks App manifests can now reference Model Serving endpoints as Unity Catalog securable resources via updates to the Python and Java SDKs 12, while the Databricks Terraform provider expanded its resource exporter to support serving endpoint dependencies 4. In parallel, new workflows demonstrate querying serverless GPU-managed endpoints directly from SQL via ai_query 6, with MLflow 3.16.1 resolving trace location resolution issues within Model Serving 8.
Generated daily from the 8 most recent items mentioning Model Serving. Click any [N] to jump to the source.
The Databricks Java SDK now includes the MODEL_SERVICE enum value for Unity Catalog securable types in Databricks Apps. This update allows developers to configure and reference model serving endpoints as securable resources within app manifests.
The Python SDK now includes a new model_service enum value for Unity Catalog securable types under Databricks Apps. This update allows practitioners to configure Model Serving endpoints as securable resources within Databricks App manifests.
From post-training to production: How TinyFish runs its web-native model on Databricks AI Runtime
TinyFish established a continuous path from post-training to deployment by using Databricks AI Runtime on managed GPUs, MLflow for evaluation, and Databricks Model Serving for production inference. Tested across 2,178 web-agent tasks, the resulting web-native model achieved a 77.6% task-success rate in internal go/no-go benchmarking.
Use Genie Code outside the workspace
Hey! I’m a Databricks product manager working on third-party development experiences. We just shipped Genie Code CLI , a new coding agent specialised for data and AI work. We’d love for you to try it out and share what you want to see next! Why we built this: Many of you prefer working locally with agents, but these agents might not always know the best way to work with Databricks. We wanted to provide an agent preconfigured to understand your Databricks data, pipelines, models, and apps. Use this when you want: Minimal setup and maintenance to use an agent that works effectively with Databricks. Governance provided through Unity Gateway. Access to agents outside of the Databricks workspace without model subscriptions or contracts. Getting started: (1) Install the latest version of the Databricks CLI . (2) Install Genie Code CLI : macOS, Linux, WSL: curl -fsSL https://github.com/databricks/genie-code-cli/releases/latest/download/install.sh | bash Windows powershell: powershell -ExecutionPolicy Bypass -c "irm https://github.com/databricks/genie-code-cli/releases/latest/download/install.ps1 | iex" (3) Start an interactive session by running genie in your project directory. Note that usage is billed through Unity Gateway’s model serving capabilities . For full details, check out the Genie Code CLI docs . Share your feedback: We’re releasing this fast to get your input asap. Please leave feedback and questions in the comments or reach out directly with a Reddit message! submitted by /u/BricksTrixTwix [link] [comments]
This release adds resources and data sources for private network gateways alongside Mason sessions and managed memory components. The resource exporter expands dependency support to include Lakeflow pipeline settings, model serving endpoints, Databricks Apps, cross-cloud cluster policies, and several new job task types.
Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks
A complete reference architecture is now available for building a real-time, multi-stage e-commerce recommendation engine on Databricks using AI Search for candidate retrieval, Lakebase for online feature serving, and Model Serving for low-latency inference. By unifying batch precomputation and real-time scoring on a single governed platform, this design replaces fragmented ML stacks to eliminate glue code and deliver end-to-end lineage from raw clickstream to production predictions.
Running open-Jev in SQL on Databricks
Run models like SemIf-OpenJev directly on governed Databricks data using serverless GPUs and managed Model Serving. Query custom endpoints from SQL or Lakeflow jobs using ai_query to return structured decisions and probability scores at scale, with an importable notebook to deploy and adapt the workflow.
NewsHow adidas Uses Databricks to Build Better Products
Adidas uses Databricks' lakehouse platform to centralize all its data—from product to football-related insights—enabling faster analytics across the organization. The company's Genie analytics tool helps analysts spend less time processing data and more time on strategic questions, ultimately supporting better product development.
Multi-region model serving on Databricks with OpenSharing
MLflow 3.16.1
MLflow 3.16.1 removes the default basic-auth admin password from basic_auth.ini as a security fix (GHSA-gq3w-7jj3-x7gr) and adds support for span links in Unity Catalog traces. The release also fixes trace location resolution in Databricks Model Serving, includes a timeout option for the @scorer decorator, and improves web UI access control for non-admin users.
Release introduces three new workspace services (AI Functions, Domains, Sandbox), extends Feature Engineering with new backfill and operation management methods, and adds AWS Secrets Manager and Azure Key Vault connection support. Multiple breaking changes remove fields from model serving and catalog configurations that may require code updates.
MLflow 3.16.0
MLflow 3.16.0 makes the redesigned trace explorer the default interface, introducing natural-language custom trace views via the MLflow Assistant, session grouping for multi-turn conversations, and span links. The update also adds Unity Catalog model service support for built-in evaluation judges, per-user AI Gateway budget policies, and fail-closed authorization by default.
The SDK adds an execute_command_sync() method for sandbox and new fields for ML features (mode, time_window, full_feature_name). Breaking changes remove routing and rate limiting configuration fields from model serving APIs.
Model Serving An internal error occurred during feature store lookup all deploys failing
NewsHow AI and Data Keep 2.3 Million Lawns Healthy | TruGreen & Databricks
TruGreen uses Databricks Genie and Lakehouse to manage 2.3 million lawns with AI that optimizes service timing and predicts customer churn using weather, soil, and service data. The system enables non-technical branch managers to take daily actions through customized reports without requiring data expertise.
The CLI adds databricks environments setup-local to provision matched Python environments for Databricks compute targets and extends aitools install to support Gemini CLI and Pi. Bundles fix the ignored bundle.deployment.lock.force setting, add pipeline cascade_on_destroy control, improve experimental job_runs with idempotency tokens and completion waiting, and add UC secrets resource support.
NewsHow FOX Sports Uses AI to Power Search
Fox Sports rebuilt their search system on Databricks to handle rapidly changing sports information by continuously streaming player, team, and content data into the index while computing real-time trends. The system uses semantic vector search with time-weighted ranking to surface fresh content higher, doubling the rate at which users find what they're looking for.
TutorialsDetect Energy Theft Faster with Genie
Databricks demonstrates an end-to-end AI application that detects energy theft, automates investigations, and generates executive reports using Unity Catalog and Genie. The video walks through an architecture featuring Lakebase for transactional storage, model serving for machine learning and LLMs, and AI gateways for governance and cost control.
This release adds effective entitlements to workspace assignment details and serverless compute ID support for job clusters. It also updates model serving telemetry configurations to include fields for table names and telemetry profile IDs.
What happens in the milliseconds after you tap pay
This sample Databricks App demonstrates how to achieve low-latency real-time fraud scoring by pairing route-optimized Model Serving with Lakebase Postgres for online feature lookups. Under load testing of 5,000 requests, this architecture achieved end-to-end latencies of 27 ms at p50 and 37 ms at p95 while maintaining a 100% success rate.
The SDK now supports specifying a parent path for jobs and associating a Git credential ID with workspace repositories. Model serving configurations now include CpuLarge and CpuMedium options for workload types.
The Java SDK adds a parent path field for job creation and settings, along with git credential ID support for workspace repositories. It also introduces CPU_MEDIUM and CPU_LARGE workload type enums for Model Serving.
Databricks AI Model Serving in production: scaling, cost, and latency lessons
SSH connect adds --base-environment for custom base environments, and aitools install now uses plugins instead of raw skills. Bundle deployments fix drift on model serving endpoints and failed migrations on permissioned resources.
A breaking fix ensures query parameters specified in ForceSendFields are now explicitly sent over the wire even when holding zero values. Additionally, new API updates add support for Genie conversation visualizations, telemetry configuration on model serving endpoints, and expanded Postgres service options.
Databricks and NVIDIA: Building for the Agentic Era
Databricks and NVIDIA are expanding their collaboration to deliver an end-to-end AI platform, accelerating model training, inference, and agentic AI development on governed enterprise data. This includes multinode training in AI Runtime, GPU support in Databricks Free Edition, Model Serving Enhancements, and support for NVIDIA Agent Toolkit and industry-specific AI frameworks.
What’s New in the AI Platform: Agents for ML Engineering, Our Deep Learning Platform, and New Capabilities for Real-Time ML
Databricks shipped Genie Code, a coding agent for ML engineering, and AI Runtime, a serverless GPU platform for deep learning. Power real-time ML at scale with new Feature Store and Model Serving capabilities, including streaming features and high-QPS serving.
How ERGO Hestia reduced time-to-market with Lakebase and Mosaic AI Model Serving
ERGO Hestia modernized its real-time pricing engine with Databricks Lakebase and Mosaic AI Model Serving, reducing time-to-market by unifying data, features, and decisions for millisecond pricing. This eliminated extraction overhead and fragmented governance from their previous multi-hop architecture, enabling faster model deployment and instant market response.
Azure OpenAI v1 API support for External Model Serving / Mosaic AI Gateway?
Databricks Model Serving: The Complete Guide to Production ML at Scale
NewsBanks' Secret Weapon Against Money Laundering: Multi-Agent AI
Databricks demonstrates a multi-agent AI solution for Anti-Money Laundering (AML) operations, significantly reducing false positives and accelerating investigation cycles from hours to minutes. The platform unifies siloed systems, employs specialized AI agents for analysis and recommendations, and offers AI-assisted SAR generation and executive-level reporting with natural language chat.
Unable to make fresh deployments to an agent model serving endpoint due to permission issues
NewsDatabricks Apps vs Model Serving: Authentication, Cost, and Performance Compared
Databricks Apps are now the recommended first choice for deploying agents due to their flexibility in handling full-stack applications with multiple components, offering faster iteration and local testing compared to Model Serving. Model Serving remains suitable for use cases prioritizing high QPS, governance features like AI Gateway, inference tables, and guardrails, or when scaling to zero is acceptable for cost optimization.
MLflow 3.11.1 introduces AI-powered issue detection in traces, AI Gateway budget alerts and spending controls, trace graph visualization, native Databricks gateway provider, and pickle-free model serialization. TypeScript SDK packages are now @mlflow-scoped and LiteLLM is no longer required for GenAI evaluation.
Databricks SDK for Python now supports new fields for defining ingestion pipelines, including connector type, data staging options, and detailed ingestion source information. External function requests in model serving can now specify a sub-domain.
NewsLakehouses: The Best Start to Your Graph Data and Analytics Journey
The video demonstrates how to build and deploy graph neural networks on a lake house architecture using Databricks instead of relying solely on graph databases. It showcases an end to end data science workflow including data ingestion, graph machine learning training with DGL and PyTorch, hyperparameter tuning with MLflow, and model serving via Unity Catalog.
NewsLLMOps: Everything You Need to Know to Manage LLMs
Databricks demonstrated how to build, evaluate, and productionize a retrieval-augmented generation question-and-answer bot using its Lake House AI platform. The presentation walked through setting up vector search, prompt engineering, model serving, and lake house monitoring.
Get Tuesday's version of this
Tracking Model Serving? The Tuesday email carries what moved across the whole ecosystem, not just this topic. Free, one-click unsubscribe.