Skip to content
LakeFlowChangelog · September 2026

Lakeflow Connect added managed marketing connectors and Lakeflow Jobs released task output in for_each loops.

Databricks expanded Lakeflow Connect with native managed connectors for ad and marketing platforms, including Salesforce, HubSpot, Google Ads, and Meta Ads, landing raw ingestion data into Unity Catalog [29]. Accompanying guides demonstrated how to model these feeds for marketing analytics [28]. Within Lakeflow Jobs, support for passing task outputs inside for_each task loops entered public beta [3], while Databricks delivered updates to Lakeflow Designer [14].

Teams adopting Lakeflow Connect shared early production experiences and raised infrastructure questions [19]. Users deploying the SQL Server change data capture gateway asked how to select specific virtual machine types [10], handle elastic compute requirements [1], and avoid connection saturation on source transactional databases [22]. Practitioners also shared architectural patterns for managing upstream schema drift and executing historical backfills alongside ongoing change streams [6].

Lakeflow Jobs sparked discussion around production orchestration maturity. Teams evaluated replacing legacy Airflow schedules with data-aware triggers in Lakeflow [16, 26], though engineers debated whether the platform provided adequate tooling for service level objectives and SLA monitoring across large fleets [15, 25]. Community members also worked through practical job configurations, such as implementing conditional task retries [8] and enabling predictive optimization for streaming tables in declarative pipelines [18].

Everything cited

  1. [1]Lakeflow connect SQL server ingestion can be made elastic? community · 2026-09-30
  2. [2]system.query.history is now Generally Available in Databricks community · 2026-09-30
  3. [3]Lakeflow Jobs update! Task output in foreach now available in Beta! community · 2026-09-30
  4. [4]Time to Swap the Cookies for Jetfuel - New Dataset & New Databricks Genie Tutorial community · 2026-09-29
  5. [5]Trigger a job automatically on databricks DABs bundle deploy video · 2026-09-28
  6. [6]Operationalizing Lakeflow Connect:Handling Upstream Schema Evolution & Historical Backfill community · 2026-09-24
  7. [7]Running open-Jev in SQL on Databricks news · 2026-09-24
  8. [8]Conditionally retrying a task on Lakeflow Jobs community · 2026-09-24
  9. [9]How I built agent-based security reviews on Databricks news · 2026-09-24
  10. [10]How can I configure Lakeflow Connect SQL Server CDC Gateway to use a desired VM type? community · 2026-09-24
  11. [11]I didn't want to choose between doing engineering and staying up to date with Databricks, so I built this. Hope it's useful to you too! community · 2026-09-23
  12. [12]Real-Time SAP Accounts Receivable on Databricks Lakeflow Declarative Pipelines community · 2026-09-23
  13. [13]Data Engineering Project On Real Company | Real Data | Pyspark | Databricks | Production Ready community · 2026-09-23
  14. [14]New features in Lakeflow Designer community · 2026-09-22
  15. [15]Anyone running actual SLOs on Lakeflow Jobs, or is it still “it failed, rerun it”? community · 2026-09-22
  16. [16]Do you still use Airflow with Databricks? community · 2026-09-21
  17. [17]Environments on classic compute community · 2026-09-19
  18. [18]Predictive Optimization for Streaming tables in Lakeflow pipelines community · 2026-09-19
  19. [19]Lakeflow Connect in production: What has your experience been? community · 2026-09-19
  20. [20]system tables data retention community · 2026-09-18
  21. [21]BrickTalk Recording | One Platform, Any Source: Unifying Enterprise Data with Lakeflow Connect community · 2026-09-18
  22. [22]Lakeflow connect CT pipeline keeps running multiple connections in source community · 2026-09-15
  23. [23][Discussion] Lakeflow Jobs: If your data catalog could trigger a job on any change what would it be? community · 2026-09-15
  24. [24]Private Network Gateway community · 2026-09-13
  25. [25]Best practices for SLA monitoring and automated retries across hundreds of Lakeflow Jobs community · 2026-09-12
  26. [26]igrating from Cron-based Airflow to Lakeflow's Data-Aware Triggers — Real-world experiences? community · 2026-09-12
  27. [27]Lakeflow Connect SQL Server Connector community · 2026-09-12
  28. [28]Tutorial: Transform your Lakeflow Connect ad data into visual and conversational analytics community · 2026-09-11
  29. [29]Unify your marketing data with Lakeflow Connect news · 2026-09-11
  30. [30]What is Data Transformation? news · 2026-09-03

A frozen monthly snapshot, generated from the brickster.ai archive and never rewritten. For the live view of this topic, see the LakeFlow hub. brickster.ai is an independent community project, not affiliated with Databricks, Inc.