O que a comunidade está perguntando.

Lakebase CDF — destination Delta table not created after successful UI setup (Free Edition)

00today

A wave of new Lakeflow Connect connectors

00today

Passed Two Databricks Certification Exams, but "No Credentials Found" After 3+ Days

00today

Databricks Community: Share your Lakebase story and receive a $50 gift card!

CUSTOMER STORY | Electrolux: Self-service Appliance insights with Genie and AI/BI

Bundle Validation Error: Volume lifecycle Field Rejected Despite Being in Schema

Announcement | How Databricks is turning video into searchable, actionable intelligence

How to Prepare Enterprise Data for AI Agents: A Practical Guide

Databricks CommunityTraining offerings

Last week

82 questions

Scorm Not showing completed .

Getting 'Unauthorized Access' when Serverless compute is trying to read s3 bucket data

whether I am eligible for the 50% discount on the Databricks Data Engineer Associate certification?

Databricks CommunityGet Started Discussions

Bryo Appetite Control Coffee: (We Tried It 365) My Honest Review!

Databricks CommunityMachine Learning

Notebooks for Labs

Delta Table in DLT pipeline

Resilience, cross-region, and cross-cloud

003d ago

Intialization stage is taking time in lakehouse pipeline

003d ago

Announcement | How Stagwell built privacy-safe ID matching on Databricks

Reverse ETL

DB runtime 18

How to limit the AI/BI dashboard sharing people-picker to workspace members

Announcement | Building an open ecosystem for AI governance with Unity AI Gateway

Databricks CommunityGenerative AI

Generative AI Development: What Does It Take to Move from PoC to Production?

lakeflow connect postgres sql

lakeflow connect postgres sql in uae north

My test got supended, I need this issue resolved asap please.

Databricks CommunityCommunity Articles

The Next Chapter of Data Engineering Is Intelligent Decisions

Header Alignment in databricks dashboard

Databricks CommunityAdministration & Architectureanswered

Zerobus ingestion fails in Databricks Free Edition using Service Principal OAuth

Databricks CommunityGenerative AI

AI Search Endpoint Quota in Azure

DBR 17.3 – sporadic import failures from bundle

Databricks CommunityTechnical Blog

Databricks AI Model Serving in production: scaling, cost, and latency lessons

Databricks CommunityAdministration & Architectureanswered

Shipping Unity AI Gateway OTel to other collectors

Announcement | What’s new in Databricks Platform security and compliance at Data + AI Summit 2026

Lakebase CDF databricks error synced

Guidance Needed on Publishing a Databricks App to the Databricks Marketplace

NOT NULL constraint violated for column during OPTIMIZE

Dashboard "Export as HTML" fails for job-associated notebook dashboards (UI only)

Auto Loader duplicate tracking

Databricks CommunityGet Started Discussions

Guidance Needed on Publishing a Databricks App to the Databricks Marketplace

Do databricks partners need to pay for databricks account?

HackerNews

Databricks LTAP Explained

40shenli35145d ago

StatusCode.UNIMPLEMENTED error: DatabricksConnect library using AKS/PySpark to calling Spark cluster

🌟 Community Pulse: Your Weekly Roundup! June 22 – 28, 2026

Announcement | Design Beautiful Dashboards in AI/BI

Databricks CommunityCommunity Articles

HackerNews

My Notes After Databricks Data and AI Summit 2026

40redskyluan6d ago

Learning Series | Databricks Performance Optimization

Correlate Databricks App with Logs

Databricks CommunityMachine Learning

what is the GPU quota limit for free edition?

Unable to Subscribe to the 14 day free trial

Databricks CommunityWarehousing & Analytics

Launch HN: Parsewise (YC P25) – Reason Across Documents with an API

Hi all, it’s Greg and Max, founders of Parsewise here (https://www.parsewise.ai/api). Parsewise transforms a bucket of unstructured data into schema compliant data, retaining lineage for values resolved across documents. Imagine giving Claude a bunch of files and asking for a CSV or JSON output. If you have tried this, you know both the system limitations (number of files, type of inputs, cost, latency) but also the human-facing challenge of having no way to validate the results quickly. We solve both. We help tech teams simplify their unstructured data ETL, and loop in business experts for the definitions and for instant validation. Here is a video with a few use cases: https://www.youtube.com/watch?v=dbRllnnh47w Parsewise in the words of someone coming to us: ”I need to extract information from insurance policy PDFs, phone calls that have been transcribed, emails, etc. I am NOT looking for something that would just extract data point by data point, page by page into a structured well-defined schema but more something more agentic that can understand that information might be across documents and that it should reason over what to extract.” We started the company based on a decade of experience (and pain) in complex data transformation and data analysis / synthesis. Greg was building both classical ETL and implemented AI workflows at Palantir. At Bain, Max did highly complex data analysis in the financial sector, similar to many of our customers. Parsewise works by taking in a bucket of data (think hundreds or thousands of pdfs, excels etc.), and outputting schema compliant data where every single value is traceable down to word level citations across multiple documents in the bucket. We provide API customers with ways to show the lineage in their own applications, or they can use our platform for internal operations. At the core of the data processing we have self-improving agent definitions. They define the acceptable sources, the logic for resolving or combining values, and the rule for highlighting uncertainty to the end user. The underlying tech is model and cloud agnostic and can be deployed in private networks. We have seen the best results with Gemini models for visual reasoning, achieving SOTA (beating Claude Fable) on the strongest grounded reasoning benchmark we have found (Databricks OfficeQA). Notably, we focused more on the “human harness” rather than the model harness, leaning into the actual friction we saw in uptake, which is around verifiability. That means optimizing the time and clicks required to trust the outcomes. We use vLLMs for parsing, and then we use small models for efficient large scale exhaustive search. Unlike RAG, we do not sample; instead, we exhaustively find all relevant values for a given query. We use larger models for decision making around resolutions and flagging inconsistencies to users. This exhaustiveness and explicit value sourcing is unique to our platform, and it goes beyond the first step of data parsing that many existing providers cover. We would love to welcome builders and tinkerers to try Parsewise on your complex document challenges. We have a ton of ideas on how we can expand the product and make it better, but would appreciate feedback and ideas from the community! --- top comments --- [whinvik] Document parsing is top of my mind lately because in some of the areas we work on the bottleneck is starting to become being able to query documents the same way one queries an api. I keep thinking the most obvious analogue is we need some way to represent documents the same way we can represent structured data in parquet. Parquet allows easy range bases queries and there is so much tooling built around Arrow. But for documents I keep hitting a wall to figure out what the right abstractions are. Parquet allows filterable metadata. But what such metadata is there for documents. Then there is the arbitrrariness of chunking, vectorization. If we could just do this in a […truncated]

5654gergelycsegzi6d ago

Asset Bundles Setup

Databricks CommunityGenerative AI

Failed to save agent Supervisor Agent is not available. To access this feature, please contact sales

Name Correction

HackerNews

Postgres data stored in Parquet on S3: LTAP architecture explained

--- top comments --- [dsauerbrun] Maybe I'm too stupid to understand the article... How does this achieve performant querying for olap and oltp purposes? Based on my understanding, olap queries will go to the parquet files which are stored in a columnar fashion and oltp style queries will go to a caching layer that sits on top of those parquet files? What's the special sauce here? Seems like they're just caching the data which, for all intents and purposes, seems like the same solution of storing another copy of the data which is what they say they're avoiding. [ianberdin] Surprisingly, I'm already encountering a second solution that involves storing data chunks on S3 — and this is all within the same week. This is becoming popular. At Playcode, we built what we believe is a revolutionary file system for our Playcode Cloud (https://playcode.io/cloud), which enables the creation of full-stack web software. The FS built completely from scratch using Rust. We thought we were the smartest ones around and that nobody else had figured this out. But it turns out Databricks, Neon, and several others have as well. The idea behind a *Bottomless File System* is really cool, and it works very well for us. Essentially, as described here: - There is a *page server* - A *Linux file system* split into chunks (let's call them chunks instead of pages) - A *cache on NVMe* - And of course, *object storage*, where everything is asynchronously synchronized It works quite well, though it has its downsides. One clear advantage is that NVMe drives have become expensive lately, while object storage remains cheap — so the benefits are undeniable. That said, latency is also a factor. On top of that, uplink costs are rising. To run an object storage-backed file system, you need a very strong uplink with consistent speed — 1 Gbps is simply not enough. Ideally, you want *5 to 10 Gbps*, depending on the load. We spend a lot of time optimizing and experimenting with different hosting providers — specifically bare metal hardware. The main challenges are: - *Slow disks* - *Slow uplink* - And as it turns out, *object storage can be unreliable* — unless you're using S3 But AWS hardware is expensive, so nothing in life is ever that simple. [Avalaxy] Super cool stuff. Being able to combine your analytical platform and transactional database into one storage layer without having to set up ETL pipelines in between is really a game changer. Especially since it's just postgres, instead of some proprietary database. [saisrirampur] But why? I’m skeptical of the idea of unifying storage just because it sounds “elegant” or “cool”. It’s not obvious to me how a single storage engine can compete with purpose-built OLTP and OLAP systems like Postgres and ClickHouse, without significant tradeoffs. You also mention removing CDC pipelines. I’m curious if the materialization (conversion across formats) can catchup to an OLTP workload that is heavy (50K+ tps), which is pretty common these days. Also CDC if done right and with care can be magical for users and stays native to the OLTP/OLAP data-store. Third, data Lakes and open formats are suitable for Data Warehousing / Data analyst use-cases than real-time customer facing apps. Sure, you might work on changing that, which is what you are upto, but you’ll always run into tradeoffs, which will make it hard to unleash the best performance, much needed for the latter category. [lmeyerov] I have to wonder: What's the open protocol - some sort of LTAP read/write path - for compute engines to target LTAP / lakebases? Sort of like how iceberg > deltalake, except now one layer up (OLTP+OLAP). LTAP / lakebase poses an interesting gap we have been tracking ever since iceberg as we build gfql (the first oss fully-vectorized property graph compute engine with CPU+GPU targets): There's a natural desire to support both OLAP + OLTP modes for new engines, but also a gap for the cloud-native storage tier we don't want to own. Ic […truncated]

18257andrenotgiant6d ago