Skip to content
All topics

Compliance

Recent items mentioning Compliance across the Databricks ecosystem — releases, news, videos, and community Q&A. Updated hourly.

9 recent items7 news2 community threads
What's happening in ComplianceAI synthesis · updated Nov 16, 2026

Databricks is expanding into the AWS European Sovereign Cloud and regional Azure deployments to support sensitive workloads under DORA, NIS2, and the EU AI Act 1, which will classify clinical AI systems as high-risk by 2026 alongside HIPAA mandates 34. At the implementation level, teams are operationalizing these requirements using two-tier tagging architectures to govern GDPR-regulated PII 5 while deploying Unity Catalog-native agents to halve onboarding timelines in regulated environments 2.

Generated daily from the 5 most recent items mentioning Compliance. Click any [N] to jump to the source.

Reddit

Is Delta Sharing the end of external ETL? My notes on Open vs D2D.

Just spent some time digging into the Unity Catalog and Delta Sharing documentation. Here’s the TL;DR on the two patterns for anyone looking to implement this: Open Sharing (The "Universal" Path): Audience: Recipients without Databricks (using Power BI, Pandas, Tableau). Mechanism: Presigned URLs + Bearer Tokens. Constraint: Tabular data only (tables/views). Security: Can be restricted via IP filtering. D2D (The "Native" Path): Audience: Both parties are on Unity Catalog-enabled Databricks. Assets: Tables, Views, Volumes, Notebooks, and Models. Pros: No token management; Zero egress costs if in the same region. Performance: DBR 16.2+ uses scoped credentials, making it basically as fast as a local table read. Observations: The "Zero-Copy" aspect is the real winner here. The audit trail in system tables alone covers about 95% of standard compliance needs (GDPR/SOC). The Coastal Community Bank case study (2 months to 2 days for onboarding) seems to be the gold standard for ROI here. Discussion: Is anyone actually using the Open Sharing side with external vendors in production? How are you handling the token rotation/security for non-Databricks clients? Or are you finding that most of your partners are already on the Databricks ecosystem? submitted by /u/SalamanderOpen859 [link] [comments]

00SalamanderOpen8591mo ago
Reddit

Apache Spark ships with 28 known CVEs in production. In 2026. And nobody thinks this is a problem worth talking about publicly

This is not acceptable any longer in a modern developer world and community. I run security scans on our Spark deployment and came back with 28 vulnerabilities — high severity, public CVEs, all sitting in transitive dependencies like Netty and Apache Thrift. Nothing exotic. Netty published fixes for 22 of them in a single batch in June. Thrift fixed their issues in 0.23.0. The fixes exist. Spark just didn’t include them. What bothers me more than the CVEs themselves is the process — or the lack of one from Apaches side. In 2026, any serious DevSecOps pipeline is expected to have mandatory quality gates that block releases on high/critical CVEs in dependencies. This isn’t cutting-edge practice — it’s baseline hygiene. The full toolchain is free and mature There is no automated dependency CVE gate in Spark’s release pipeline. No Trivy. No Dependabot . No OWASP Dependency-Check. Nothing that would block a release because a bundled library has a known high-severity vulnerability. These are free tools. Adding one to a CI pipeline is an afternoon of work. It hasn’t been done. Problem Honest Assessment No automated dependency scanning Inexcusable in 2026. Free tools exist. One CI step. Decoupled release calendars Real coordination challenge, but solvable with Dependabot PRs that can be reviewed and merged quickly Volunteer PMC Doesn’t excuse Databricks, Google, Apple, and Amazon — all Spark committers with paid engineers — from contributing a security gate “Good enough” culture Actively harmful when Spark is used in AI/ML pipelines processing personal data under GDPR The July 2026 maintenance releases — 4.2.0, 4.1.3, 4.0.4, 3.5.9 — all shipped after Netty 4.1.135.Final was available . They didn’t include the bump. There was no public statement that the team was aware of the issue and working on it. It just shipped, vulnerable, into production systems everywhere. The “volunteer PMC” argument doesn’t land anymore. Databricks is worth somewhere around $62 billion. Google, Apple, Amazon, and Microsoft all have paid Spark committers . The resources to fix this governance gap before lunch exist. The will apparently doesn’t. The EU Cyber Resilience Act (CRA) — which came into force in 2024 and has mandatory compliance deadlines rolling in through 2027 — specifically targets software supply chain security, including transitive dependencies and SBOM (Software Bill of Materials) requirements. The ASF has acknowledged this directly, stating they need to prepare projects for “CRA and U.S. CISA guidance”. Spark ships into commercial products. This will become a compliance problem for a lot of companies very soon , and the fix is genuinely trivial from an engineering standpoint. The ASF did make progress in 2025 — launching “Apache Trusted Releases (ATR)” for distribution security and ratifying CycloneDX 1.7 for SBOM standards — but none of this yet translates to a blocking CVE gate on the release pipeline for projects like Spark . In the meantime: if you’re running Spark and using JFrog Xray or Trivy on your deployment, you can force-override the affected Netty and Thrift versions in your own build. It’s not clean but it works until the next maintenance release, expected sometime in Q4. Is anyone else tracking this or pushing upstream to get a CVE gate added to the build? submitted by /u/hrpedersen [link] [comments]

00hrpedersen2mo ago

Get Tuesday's version of this

Tracking Compliance? The Tuesday email carries what moved across the whole ecosystem, not just this topic. Free, one-click unsubscribe.

Read past issues first