Monitor LLM Usage & Prevent Data Leaks with Traces
Summary
Databricks Unity AI gateway's traces feature lets admins monitor all LLM usage and verify that sensitive data hasn't leaked. Traces provide a comprehensive view of all LLM activities to ensure security and compliance across the platform.
Summary generated by brickster.ai from the video transcript.
More from Databricks Skill Builder
NewsConfigure Budget Thresholds for AI Models
Databricks allows you to configure shared budget thresholds (with a $10,000 cap and usage blocking) to control overall AI model spending. You can also set per-user budget limits (such as $5,000) with alert notifications to monitor individual consumption when certain users dominate budget usage.
NewsAzure Pipelines: Triggering Deployments Based on Branches
Azure Pipelines YAML uses branch-specific triggers to automatically deploy code to different environments based on which branch receives the push. Dev branch deployments go to a dev workspace while main branch deployments go to prod workspace after PR approval.
NewsAI Runtime CLI | Serverless GPU LLM Training
Databricks AI Runtime is a CLI tool that enables distributed LLM training on serverless GPUs using YAML configuration files, supporting deployments up to 256 H100 GPUs with built-in MLflow experiment tracking and hardware metrics. Users develop training code locally in their preferred environment and submit jobs via CLI commands that automatically handle infrastructure provisioning, log streaming, and performance monitoring.
NewsSet AI Budgets & Control Spending in Databricks!
Databricks admins can configure spending budgets for individual users, workspaces, or groups through the AI Gateway's govern section to prevent overspending. The demo shows creating a named budget and selecting resource types like Unity AI Gateway to define which models and resources are subject to the budget limits.
NewsDatabricks AI: PII Blocking for Data Privacy!
Databricks PII blocking creates input guardrails that prevent personally identifiable information from reaching AI models before processing. The feature allows users to configure and apply PII blocking policies across their account to protect sensitive data.
TutorialsHow to Connect Databricks and Fabric (No Copying Required)
Databricks catalog federation and Fabric mirroring enable zero-copy data access between platforms, allowing Fabric compute to read Databricks data or Databricks compute to read OneLake data without duplicating storage or managing ETL pipelines. The demonstration covers practical setup steps, required prerequisites like storage credentials and external data access permissions, and current limitations including read-only access and support for table objects only.