Was die Community fragt.
Aktuelle Threads aus r/databricks und dem [databricks]-Tag von Stack Overflow (konkrete Schmerzpunkte, Integrationsfragen und Sonderfälle, die man kennen sollte).
This week
84 questionsDatabricks Connect + Serverless: Why "Module Not Found" Shows Up Only in CI, Not Interactively
Anyone else gotten a rough surprise with Databricks costs once things hit production?
This keeps coming up in conversations with clients and I feel like it's worth its own thread. The pattern is almost always the same. A pipeline gets built to bring in data for analytics or ML, works fine in testing, then goes to production and the compute bill is way higher than expected. Nobody budgeted for it because on paper it looked like a simple ingestion job. Usually the real issue isn't Databricks itself, it's the ingestion design. The repeat offenders I keep seeing: Full reloads instead of proper CDC, so you're paying to process data that hasn't even changed. Serverless SQL running more often than needed, because someone assumed near real time was required when batch every few hours would've worked fine. No plan for schema evolution, so jobs fail or reprocess more than they should every time something shifts upstream. Cluster sizing set for peak load "just in case" instead of actual daily volume. Most of the fix comes down to being honest about the freshness you actually need. A solid CDC layer feeding into something like Kafka before it hits Databricks tends to cut a lot of the unnecessary compute, since you're only moving what changed. Curious what caused it for others, ingestion design or job scheduling? submitted by /u/Only-Dragonfruit4130 [link] [comments]
Power BI dashboards using Direct Lake
Databricks Advanced Learning Festival: September 16 - October 14 2026
FAQ for Advanced Learning Festival September 16 - October 14 2026
Genie Agents dude architecture
submitted by /u/Chrisesc0 [link] [comments]
Genie Agents dude architecture
Hi, I have a question now that I’m planning to start a Genie Agents project and am organizing my architecture. I’m working from SAP BDC, and I plan to create two workspaces in Databricks—one for DEV and one for PRD. But I’m not sure where it makes the most sense to create the Genie Agent. Technically, the queries and tests would be better suited for the PRD workspace; running them in DEV doesn’t make much sense because the Genie Agent DEV is connected to DEV tables. But then, what role does CI/CD play here for Genie Agents? Or am I misunderstanding the architecture, and should I create two Genie Agents (DEV and PRD) in the PRD workspace? Thank you very much if you can help me with this question. submitted by /u/Chrisesc0 [link] [comments]
Databricks zerobus vs Fabric open mirroring (2026)
Has anyone seen any comparison between the generalized ingestion mechanism (zerobus) with Fabric's offering (open mirroring)? Seems like there should be a blog or youtube video comparing the two by now. But I haven't seen any. Unfortunately it sounds like they both rely on proprietary middleware. Ideally there would be a similar type of software which that we could just run on-premise to land data into cloud blobs (like a gateway of some kind). Not sure why that would be so hard for someone to do as a github library or something. Maybe it would need to be done in a performant language like rust or .net, but it doesn't seem like it would be rocket science. Both those technologies are relatively recent: Fabric open mirroring : May 2025 Zerobus : Feb 2026 Personally I wouldn't want to pick either one of these technologies until a comparison could be made. Microsoft's open mirroring claims that they can land data in their lakehouses for free. After that point, the raw deltalake tables would be accessible to both platforms. If open mirroring is truly free then it seems odd that any databricks customers would be using zerobus. They should just purchase the smallest possible capacity from Microsoft like an F2, and use that for moving all their data to raw/bronze in adls gen2 containers. Whatever happens after that can take place in either of these two saas'es, databricks or fabric. submitted by /u/SmallAd3697 [link] [comments]
Databricks Dashboard - Scheduling Questions
Hello fellow developers, I recently developed and published a Databricks dashboard for a personal project, and I want to learn how the scheduling feature works. For context, I have read the Databricks documentation, Manage scheduled dashboard updates and subscriptions | Databricks on AWS , but I couldn't find answers to the questions below. Databricks Dashboard — Question Set 1 I published the Databricks dashboard using Individual Data Permissions . I then added another person as a participant, created a schedule, and subscribed to the schedule from both accounts. Let's call them Person A (publisher + subscriber) and Person B (subscriber). Questions a) Assume Person B works in the Sales department and should only be able to see Sales data. Since the schedule was created by Person A, when the scheduled run occurs and Person B receives the email notification they subscribed to, will the PDF attachment contain all departments' data , or will it contain only the data Person B is permitted to see? b) After the scheduled run, if Person B opens the dashboard using the link in the email notification, or accesses the dashboard directly through Databricks, what data will they see? Will they see all department data , or only the data they are permitted to see based on their individual data permissions? My understanding is that when Person B visits the dashboard, it does not automatically refresh just because a scheduled dashboard update has occurred. I may be misunderstanding how this works, so I'd appreciate some clarification. submitted by /u/SquareLong2523 [link] [comments]
Recording | BrickTalk: Mastering Databricks Genie Capabilities
Endpoint Error
Databricks SQL Alert - Scheduling Questions
Databricks SQL Alert — Question Set Help needed, I created a SQL alert in databricks to notify me when a new entry is added to a dataset/table within the last 24 hours. SQL SELECT COUNT(*) AS new_ FROM catalog.schema.table WHERE xyz IN ('a', 'b', 'c', 'd') AND datetime >= CURRENT_TIMESTAMP() - INTERVAL 24 HOURS; Within the Alerts panel, I set the condition as follows: First row Column name: new_ Operator: > Static value: 0 I then added my email address in the Notifications section and set "When alerting, notify" to Always . I then selected View Alert , where I created a schedule — let's say every day at 9:00 AM . Questions a) How exactly does the schedule work here? I am asking because I selected "Always" under the notification settings, but I have also created a schedule to run every day at 9:00 AM. What is the relationship between these two settings? b) Let's suppose I build this alert and create the schedule as described above. Person B (sales dept rep) also subscribes to this alert. I do not want Person B to receive an alert unless the new record belongs to the Sales department. For example, if a new record is added that belongs to a department other than Sales, and the scheduled alert runs at 9:00 AM, will Person B still receive an email notification? Thanks in advance for any clarification! submitted by /u/SquareLong2523 [link] [comments]
Lesson learned from serving real-time data app
submitted by /u/Responsible_Status49 [link] [comments]
Open Data Lakehouse: Build Like Google
submitted by /u/codingdecently [link] [comments]
Databricks Certification Exam Suspended - Databricks Certified Data Engineer Associate
Hospino : Hospital Management Software
Confused new user: where to find the notebook for free course?
Databricks OpenSharing Explained: Delta Sharing for the AI Era (w/ Databricks Product Leader)
Hey everyone! In this video, a Databricks product leader joins me to explain what OpenSharing, and what happened to Delta Sharing! Delta Sharing is one of the features I used often at a previous role to provide data to multiple customers, so it will be interesting to see how the journey with the expanded focus (data + AI sharing) goes. submitted by /u/JosueBogran [link] [comments]
Questions about "Store OpenTelemetry traces in Unity Catalog"
Using autoloader with multiple object types in load path
Databricks Dashboard - Scheduling Questions
( FYI, I did write the initial draft of this message by myself, and then later asked AI to proof read it and later pasted the same here, Apologies in advance!) I recently developed and published a Databricks dashboard for a personal project, and I want to learn how the scheduling feature works. For context, I have read the Databricks documentation, Manage scheduled dashboard updates and subscriptions | Databricks on AWS , but I couldn't find answers to the questions below. Databricks Dashboard — Question Set 1 I published the Databricks dashboard using Individual Data Permissions . I then added another person as a participant, created a schedule, and subscribed to the schedule from both accounts. Let's call them Person A (publisher + subscriber) and Person B (subscriber). Questions a) Assume Person B works in the Sales department and should only be able to see Sales data. Since the schedule was created by Person A, when the scheduled run occurs and Person B receives the email notification they subscribed to, will the PDF attachment contain all departments' data , or will it contain only the data Person B is permitted to see? b) After the scheduled run, if Person B opens the dashboard using the link in the email notification, or accesses the dashboard directly through Databricks, what data will they see? Will they see all department data , or only the data they are permitted to see based on their individual data permissions? My understanding is that when Person B visits the dashboard, it does not automatically refresh just because a scheduled dashboard update has occurred. I may be misunderstanding how this works, so I'd appreciate some clarification. Databricks SQL Alert — Question Set 2 I also created a SQL alert to notify me when a new entry is added to a dataset within the last 24 hours. Please note: The screenshot below is not from my actual example. I am only including it so that new developers like myself can relate to what I am referring to. enter image descr […truncated]
transformWithState might be causing 'Module not found'
Tagging individual ai_query() calls in SQL?
Essentially the title, but for background we’re using ai_query() in Databricks SQL Warehouse and our pipeline makes 3 LLM calls in parallel for different steps. We can see token usage/cost in system.ai_gateway.usage, but we can’t tell which call came from which step. From what I can tell, request tags are possible via the Python/rest sdk, but not through ai_query() in SQL. Has anyone found a workaround, or is moving the calls out of SQL currently the only option? Edit for clarity: the main thing we’re trying to do is join each pipeline step back to system.ai_gateway.usage so we can attribute token usage/cost to steps a/b/c etc submitted by /u/hulioshort [link] [comments]
You can select a level of effort in Genie Code
Another good news for Genie Code 🔴. You asked for it, Databricks built it. You can now select the level of effort and change it for a conversation: Auto (default, recommended): This level provides the highest quality for any task. Low : This level lowers the cost for simpler tasks. submitted by /u/Youssef_Mrini [link] [comments]
Streaming Doesn't Mean Your Compute Needs to Run Forever
Evaluating AI Agents Live at the Grounded Reasoning Cup
Announcement | How Databricks Genie Code Automated 90% of Data Ingestion for a Major Railroad
DQX Forge - Extension for DQ in Databricks
DQX Forge - Extension for DQX in Databricks
Hey guys, i'm just launched a VsCode extension for Data Quality proccess using DQX. The idea is simplify the process using AI (you can do it manually to). The extension construct data contracts, jobs and dashboards, all of that with few clicks and with less than 10 minutes. If you can, please, test and send me a feedback, so i can improve that. You can download directly in VsCode Extensions Marketplace, or https://marketplace.visualstudio.com/items?itemName=arthurfr23.dqx-forge https://preview.redd.it/peasj2p0ajkh1.png?width=3418&format=png&auto=webp&s=4b2b9c9dd3ca68c65a3c45fd08a6e793396d5dff submitted by /u/Significant-Side-578 [link] [comments]
Genie Ontology Benchmark
request for Databricks Certified Associate Developer for Apache Spark 3.0 free voucher
☕ Qué es un CLUSTER en Databricks | Tipos, DBR, Serverless, Costes, DBUs y Spot Instances 🇪🇸 🚀
Open Data Lakehouse: A Practical Guide
submitted by /u/codingdecently [link] [comments]
Databricks App Access Issue - Free Tier Platform
Serverless Access Control is here!
We finally have a simple way to control access to serverless in the workspace. Two built-in objects have been introduced: - Default Interactive Compute - Default Automated Compute The first one supports notebooks and databricks connect. Second one jobs and SDP pipelines. To limit who can use serverless: Click Compute in the workspace sidebar. In the Serverless tab, click the kebab menu next to Default Automated Compute , then click Edit permissions . Remove the All Users group, or the group that includes all workspace users. Add only the specific users, groups, or service principals that you want to authorize. https://preview.redd.it/sg59jole9hkh1.jpg?width=957&format=pjpg&auto=webp&s=0c33d19d04c57856c51f57f33c44719317fe8347 submitted by /u/szymon_dybczak [link] [comments]
[Announce] Apache Iceberg Virtual Meetup Series
submitted by /u/PrideDense2206 [link] [comments]
Estimating Databricks Annual Implementation and Licensing Costs in 2026
Unity Catalog Secrets - storage location for catalog.schema.secret
Read the announcement regarding Databricks Unity Catalog and how secrets can now be stored under a schema as opposed to using secret scopes backed by a secrets manager e.g. Azure Key Vault. Does anyone know if the secrets are stored in plaintext in bundle state if using the direct deployment engine mode for DABs? submitted by /u/RazzmatazzLiving1323 [link] [comments]
Using identity columns? Check out these best practices!
submitted by /u/minibrickster [link] [comments]
Vector Store update stale in Syncing status even the actual sync task is done.
Experience Using Databricks AI Gateway's "Policies" Feature for Adding Safeguards
I am personally happy to see how we are moving away from the AI wild-west and into governed AI with proper safeguards, access, and cost controls. Here I am using Omnigent & Databricks' Unity AI Gateway, which allows you to add several layers of protection against risky/non-compliant AI usage. Note: "Policies" is still in beta, and during my testing, I did hit some cases where the guardrails should have been more rigorous. I passed along the feedback to the Databricks team, and I expect that things will be addressed sooner rather than later. PS: Trying out this different style of animated GIFs to explain different concepts after some feedback from the community here. Love to hear your feedback! submitted by /u/JosueBogran [link] [comments]
Bedrock and Genie
Learn Databricks Agent Bricks | Build Enterprise RAG Agents
Building for the Databricks Genie App Challenge — what pain points do you actually want solved?
Hey r/databricks , I’m entering the Genie-Powered App Challenge and want to build something genuinely useful rather than another generic dashboard or basic search bar. Any ideas?? 🧐 submitted by /u/Basheer_Ahmed [link] [comments]
Agentic AI Projects?
Hi, Curious. What are some agentic AI projects you guys have worked on or implemented and how have they added value? I’m a newbie working in this space and trying to spar for some interesting project ideas :). One idea I’m thinking of already is an agentic conversational data reporting agent that also generates PowerPoints, emails, etc. suited to some pre-defined templates in addition to just being a conversational data explorer. submitted by /u/BearPros2920 [link] [comments]
Attach files in Genie Code
Genie Code is getting a new feature 🥳 Good news you can attach a document to Genie Code when you need it as temporary context for the current conversation. What's a good use case? 🔴 Explain or summarize a document: “Summarize this PDF and list the key decisions.” 🔴 Extract information from PDFs, scans, or images. 🔴 Generate code based on a specification: attach a requirements document and ask Genie Code to create Python or SQL. 🔴 Analyze data files such as CSV, Excel, JSON, or JSONL. 🔴 Convert documentation into code, tests, or a notebook. 🔴 Interpret diagrams, charts, architecture sketches, or screenshots. 🔴 Migrate Tableau or Power BI files into AI/BI dashboards using /importBI. What are the supported file types ? .pdf, .csv, .xlsx, .xls, .json, .jsonl, and common image formats. submitted by /u/Youssef_Mrini [link] [comments]
CUSTOMER STORY | From high costs to near real-time insights with Zerobus Ingest
Microsoft Fabric One and Databricks Unity Catalog — Bidirectional Access Without Data Movement
Possible Incorrect Answer Marking in Databricks Academy Assessment - Agent Evaluation on Databricks
Minilake: a free, local Databricks API emulator — a single-developer tool for testing databricks-sdk/Terraform code against real SQL, real Delta Lake, and real Job execution
submitted by /u/agentdero [link] [comments]
Databricks delta tables and Iceberg
We just shipped Session Restore for Serverless Jobs (beta): debug a job run without rerunning it
👋 I'm a PM working on Serverless Notebooks at Databricks. We just shipped Session Restore for Serverless Jobs in beta, and I'd love your feedback. The problem: A notebook job runs for hours, fails or produces weird output, and by the time you debug it, the state is gone. Rerunning a 16-hour job is painful. Reproducing an issue that happens once every few months can be nearly impossible. What we shipped: You can now restore the Python variables and Spark session from a serverless job run into a new interactive notebook, without rerunning the job . From the job run details, click "Clone into new notebook" or "Debug in new notebook" for a failed run. You can then: Inspect state from a failed run Investigate weird output without rerunning expensive queries Run expensive setup as a job, then pick up interactively from that state Try it: A workspace admin can enable "Session restore for serverless jobs" from the Previews page. Docs: https://docs.databricks.com/aws/en/notebooks/schedule-notebook-jobs#session-restore-for-serverless-jobs This was also demo'd at DAIS this year: https://www.databricks.com/dataaisummit/session/modern-databricks-notebook-tips-ai-workflows-and-best-practices-every If you try it, feel free to leave feedback or respond here. 🙏 submitted by /u/ThreeBricksWish [link] [comments]
DATABRICKS.SQL and DATABRICKS.TABLE Functions Debuts in Microsoft Excel
Solution Accelerator Series | Overall Equipment Effectiveness
🌟 Community Pulse: Your Weekly Roundup! August 10 – 16, 2026
how to enable column level lineage in databricks
Auto SCD API Tombstone Garbage Collection
Not Able to Remove Courses
Announcement | Taking AUTO CDC to the Next Level in Databricks
Open Sharing protocol of Databricks Apps and SaaS sharing
I want to understand the open sharing protocol, how it is sharing the apps, what artefact is shared when app is shared, is it sharing the app files from Workspace path directly? Secondly, how the SaaS is shared and what artefact is shared and how? Could not find documentation. Thanks
Identity Columns Best Practices for Databricks Lakehouse
Cannot work with workspace
DABs Migration Guide: Terraform to the Direct Deployment Engine
Not Able To Log In - Free Edition
Azure Databricks- Unable to enable datatbricks admin role to user to have access to system.serving catalog
This looks like a strange situation that i am facing here i have created a databricks with existing user in my personal account the user already have global administrator privilege however after i signed into datbricks, this user(personal id) appeared as worksapce admin rather a account admin. i realized this issue while attempting to access 'system.serving' catalog/table where the databricks not allowing logined user to access/grant permission to the system catalog. As per the Microsoft documentation, when we create a databricks workspace / login to workspace, it initially provide the privilege as Account admin, but it seems account admin privilage is not granted. Is there any solution to fix this? #Expected: my currnent user should have databricks admin rather workspace admin access
Genie Agent Volume Attachment (Beta) Docs mentions multiple formats but images aren't supported
Account Console environment for demonstrating workspace management capabilities to clients ?
Child-Parent Partnership
S3 LIST costs on high commit rate Delta tables: is there a start-after option on Databricks runtime?
ML Training low File I/O and Throughout
Databricks Community Contest | Genie-Powered App Challenge
where to find the lab to practice
Azure Databricks Serverless Compute Unable to Connect to Azure SQL MI Using Failover Group FQDN via
🇪🇸 ☕ ¿Cómo organiza Databricks tus datos? | De Unity Catalog a Delta Lake ⚡
CUSTOMER STORY | Albert Einstein Hospital cuts clinical query time with Genie
Beyond Prompting: Production-Ready Image Generation and Visual Asset Management on Databricks
How to Block Databricks Genie Usage When a Budget Limit Is Reached
Configure Databricks Access to Keyvault with Azure Role Based Access Control (RBAC)
Generative AI Engineering Pathway/Agent Evaluation on Databricks/Quiz
I want to help me with registration key
Databricks Genie Cost Control: How to Set Budgets and Block Usage
If statement in DAB YAML file support
Unity Catalog Lineage APIs – Are these REST APIs publicly supported?
Is the Semantic Layer the Real Challenge With Databricks Genie?
Last week
94 questionsOracle CDC Ingestion Pipeline - Schema Exploration finds 0 tables when DB_DOMAIN
Databricks as a Semantic Engine: Why the Semantic Layer Was Never Enough
test azure upload file
Exploring Databricks Instructed-Retriever-1 Through a Data Engineering Use Case
Exam issues due to Webassessor maintenance
Issues with Custom Agents on Free Edition
Are We Entering the Context Engineering Era?
Request to Merge Two Databricks Community Accounts
Visual, step-by-step curriculum for mastering DABs and Infrastructure as Code
SQL Query billing
Billing
API Get Metadata Registerd Models
How to use AI for photo filter during registration like OwnMates?
Omnigent Meta-Harness
facing issue in llm models
Excel Add-in Regression - Sign-in opens external browser window instead of embedding in task pane, r
Databricks Raises $5B at a $190B Valuation
why micro-batching matters so much in Databricks Auto Loader and Structured Streaming
End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and Databricks
How to extract data from SAP to Databricks?
End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and Databricks
Will Databricks Ever IPO?
--- top comments --- [jethronethro] Does Databricks need to IPO? If so, why? [eyehurtsme] They have 0 reason to IPO at the moment
Lakebase synced table doesn’t recognize Auto CDF on a SDP materialized view
Getting ready for Genie Ontology
Are you getting ready for Genie Ontology? You can leverage PAGES that sit in the Discover page and are organized by domain and subdomain. Each domain and subdomain has its own set of Pages and users with access to a domain can create and govern them. 🛑 But first, what do you mean by Pages? Pages are part of UC semantics; it's the business context that you define and govern explicitly, forming the human-modeled layer of the Genie Ontology. 🛑 Why is it useful? When Genie One answers a question about a concept you've defined in a Page, it prioritizes the Page's definition over context it infers automatically, and cites the Page so users can confirm the source. 🛑 Any tips to build pages? You can create Pages from those documents instead of writing each one by hand. Genie Code reads the documents you attach, extracts the terms it finds, and returns a set of proposed Pages. You review and edit the proposed Pages before any of them are created. 🛑 Is it a collaborative environment? You can Comment : Ask a follow-up question or flag context for the owner. You can Suggest edits : Suggest changes to a published Page's body. Each time you click Suggestion , edit the body, and click Save , your edits are grouped into a single batch. The owner or curator accepts or rejects the entire batch at once. Accepting a batch clears all other pending batches on the Page, including those from other users, and this can't be undone. You can React : Upvote or downvote a Page to signal whether it answered your question. The owner or curator can also edit a published Page's content directly, bypassing the suggestion workflow. 🛑What's next? Create domains, Subdomains, leverage UC metric views, and connect your external tools to Databricks submitted by /u/Youssef_Mrini [link] [comments]
Oh no, not another one! Databricks buys Electric
Gear Up: The Next Databricks Community Challenge is Almost Here!
Can I use a corporate voucher on my personal Databricks account? (Plus: discount type & expiry)
Solution Accelerator Series | R&D Optimization With Knowledge Graphs
Data quality Lineage Root cause analysis
How to call a SQL Server stored procedure using pymssql from Databricks Serverless Compute?
I have a Databricks notebook that currently uses pyodbc to connect to SQL Server and execute stored procedures. I need to migrate this notebook to Databricks Serverless Compute , so I am looking for an alternative to pyodbc . I am considering using pymssql instead. What is the correct way to connect to SQL Server and execute a stored procedure using pymssql from a Databricks Serverless Compute environment? As of now this is the code we're using: def exec_stored_procedure(stored_procedure, json_data): try: conn = pyodbc.connect(connection_string) cursor = conn.cursor() cursor.execute(f"OPEN SYMMETRIC KEY {Symmetric_name} DECRYPTION BY PASSWORD = '{Symmetric_key}'") cursor.execute("{CALL " + stored_procedure + "}", json_data) conn.commit() except pyodbc.Error as e: print("PyODBC error:", e) except Exception as e: print('An error occured: ', e) finally: try: cursor.execute(f"CLOSE SYMMETRIC KEY {Symmetric_name}") except pyodbc.Error as e: pass except Exception as e: print("An error occurred while closing symmetric key:", e) try: cursor.close() except pyodbc.Error as e: pass except Exception as e: print("An error occurred while closing cursor:", e) try: conn.close() except pyodbc.Error as e: print("PyODBC error while closing connection:", e) except Exception as e: print("An error occurred while closing connection:", e) What would the equivalent implementation using pymssql look like, and are there any additional requirements or limitations when using pymssql with Databricks Serverless Compute? My goal is to replace pyodbc while keeping the existing SQL Server stored procedure logic unchanged.
NetSuite JDBC Driver 8.10.190.0 - Databricks support
Difference in exam content between the PT-BR and English versions - Databricks Certified Data Engine
Object metadata
X_NHC_CONTROL_PLANE_UNREACHABLE
🌟 Community Pulse: Your Weekly Roundup! August 03 – 09, 2026
Skipping malformed records when reading Avro-files
Learn Databricks Lakebase: managed Postgres for apps, agents & real-time data
From 90 Minutes to 3: We Turned Our Genie Space Into an Employee
Open-sourcing Metals v2: Databricks' Java and Scala language server
Databricks streamlit app with write back capability and audit trail display
Learn Databricks Genie: 5 courses to go from curious to certified
LLMOps for Data Scientists and AI Builders: A Quickstart on Databricks
Databricks Community Fellows – July 2026 Recap
Unity Catalog Metric Views to be accessible to Custom Apps outside of DBX environment
Data Drift Metrics
Integrate genie workspaces automatically based on connection.
Migrate your Dashboards to AI/BI with Genie Code
Electric is joining team Neon at Databricks
Change Data Feed in Databricks Delta – How to Process It the Most Efficient Way
Databricks App architecture with AppKit - agentic app with governed write-back functionality
Electric Is Joining Databricks
Error DELTA_CATALOG_MANAGED_TABLE_UPGRADE_WITH_OTHER_PROPERTIES during catalog commit upgrade
Declarative Until It Isn't: Four Sharp Edges of Lakeflow Declarative Pipelines
How to remove/delete a Relationship Graph (Semantic Model) from an AI/BI Dashboard?
Electric is joining team Neon at Databricks
Neon acquires ElectricSQL to build better data syncing for agents
Track Secrets Access
Request for Name Correction on Databricks Certification
Why is the default auto-termination for serverless interactive notebook compute 60 minutes?
Electric(SQL) Joins Databricks
Multiplex Streaming, Delta Sinks, and Iceberg Reads with Databricks
Databricks Watermark-Based Incremental Ingestion
I benchmarked the new Databricks Lakehouse RT for billion-record tables
Electric is joining team Neon at Databricks
--- top comments --- [randombetch] Sick. Neon rocks. [randombetch] Sick
Electric Is Joining Databricks
Synced Tables - Partitioned Tables
Announcement | Databricks Completes Acquisition of Panther: Accelerating the Security Lakehouse Era
Databricks EMEA Learning Festival | September 29-30
Unable to Enable Unity Catalog – Azure Managed Identity Credential Not Found
Unity Catalog service credential get_token rejects api:// scope format — "not a valid URI"
AI/BI Dashboard pivot export: Excel
Multi-fact Star Schema patterns in Databricks
Unexpected behavior of Delta VACUUM – need explanation
Upcoming Community BrickTalk: Mastering Databricks Genie Capabilities
Disabling Change Tracking and enabling Change Data Capture in SQL Server Lakeflow
Is VARIANT supported in Databricks-to-Open sharing?
How to manage SQL queries for business data extraction in Databricks?
Azure Databricks Default Package Repository with Azure Key Vault-backed Secret Scope
From 40 Minutes to 8 minutes: Why We Dropped MERGE in Our SAP BW to Databricks Gold Layer
Solution: Simplify Genie Agent Instruction Updates Across Multiple Spaces
Creating Databricks agent
Triggered vs. Continuous Mode: A Deep Dive into Serverless Lakeflow Spark Declarative Pipelines
Google Drive ingestion pipeline failing – “Google Drive file system is not enabled”
How to Build, Test, and Ship SQL Pipelines on Databricks
Wrong Exam Selection – Request to Change from Data Analyst Associate to Data Engineer Associate
Lakehouse Monitoring Solution
How should schema evolution be handled across silver and gold layers in a medallion architecture?
Building a Production LangGraph Agent on Databricks - NorthStar Brand Copilot
Databricks Community Champion - July 2026 - Emma Stowell
Databricks Cost Optimizer: Audit Spend with Codex or Claude Code
Databricks Data Mesh Best Practices: A Practical Implementation Guide
Databricks Data Mesh Best Practices: Practical Implementation Guide
Learning Series | SQL Programming and Procedural Logic
Databricks Support #00984257 - Password Reset Email is not working for me
Week of Aug 3
22 questionsRT Lakehouse - impossible?
Change Data Feed on Materialized Views Why I Think This Is More Than an Incremental Processing
Request for Databricks Associate Engineering Exam Voucher
From Spreadsheets to Insights: How Genie One Transforms Excel
Getting on-premises SQL Server data into Databricks: the networking is the hard part
Feedback on Deploy Workloads with Lakeflow Jobs SPL
Is it safe to expose JWT in Databricks Job?
Databricks Knowledge Assistant
From Agent-ready to Audit-ready: The Evidence Layer of AI Governance
Monorepo vs Multi-repo for Databricks Asset Bundles: A Decision Framework
Building Custom Apps on Lakehouse
Managing AI Coding Costs at Scale
--- top comments --- [extr] I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like: - Spend most time prioritizing/discussing what to do. - Once that's agreed, use Fable 5 High + 5.6 Sol XHigh come up with a design + plan. Agree on the high level plan. (Usually this just comes down to choosing where the change belongs on the spectrum between minimal patch <-> full redesign) - Use Opus 5 or Sol Med to execute - Auto-fix bugs and CI until green + thermonuclear review skill x3. - Manual interrogation of change/nits - Come up with QA plan and have Codex Computer Use execute on it - Manually spot check the final result (usually a sizable diff, thousands of lines, complete feature E2E, etc) I probably spend like $80 a day at least but I produce the output of 3 or 4 2022 engineers and probably at better quality. So it's easily worth it. Would I save money by switching to GLM 5.2 and such...perhaps? IDK. At our scale it's not worth the time spent building the eval harness to actually understand the performance tradeoff. [lbriner] There are a surprising number of articles like this along the lines of, "we started using AI tools and ended up spending millions per year". On what planet do people start paying for things without keeping an eye on the costs and no-one notices until you have spent a crazy amount? I don't understand. You are either paying a fixed amount which you are happy about in-advance or you are PAYG in which case you would ballpark how much it costs. Otherwise it reads a bit like a fake problem, because it didn't really happen, you just foresaw it (as you should) and added a few guide rails. [sashank_1509] I suspect that when it comes to hard complex software products, you’re better off ignoring agents and doing “trad coding”. What you lose in short term speed you gain in manageable complex codebases. If you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term. Now of course, there are products that just involve lots of code but are not actually complex. This is generally the project with like hundreds or thousands of features but most of the features are separate and don’t actually interact in complex ways. Think a task management app with hundreds of features like calendar, email integration etc. there I think agents gives you more bang for the buck. Just my thought, using agents at work. [platinumrad] Careful. If you admit to using models that weren't trained by OpenAI or Anthropic then you might hauled in front of Congress: https://www.scmp.com/news/china/diplomacy/article/3362616/us... [dgellow] What I take from this is that models are already commoditized, and it’s pretty clear nobody has a moat: routing for the models, they can be swapped whenever new models are released, AI labs will have to continue to run on the treadmill non stop or be replaced. Long term I cannot imagine that business will be high margin. Routing for the harness, so anything that differentiate a provider vs another isn’t exposed to the user and isn’t too relevant. One more datapoint for the thesis that OpenAI and anthropic aren’t viable, sustainable businesses, and cannot justify their $1T valuation and the level of compute commitment (reminder that OpenAI committed to >$750B in infra spending for 2030)