databricks TaskValue and Lakeflow - orchestrate using Python lists and arrays
Description
Databricks taskValues can pass a Python list straight into a For each loop in Lakeflow Jobs. You can generate the list dynamically and run the same task for every country, table, file, etc. ✨ Explore Databricks AI insights and workflows—read more: https://databrickster.medium.com/ 🔔𝐃𝐨𝐧'𝐭 𝐟𝐨𝐫𝐠𝐞𝐭 𝐭𝐨 𝐬𝐮𝐛𝐬𝐜𝐫𝐢𝐛𝐞 𝐭𝐨 𝐦𝐲 𝐜𝐡𝐚𝐧𝐧𝐞𝐥 𝐟𝐨𝐫 𝐦𝐨𝐫𝐞 𝐮𝐩𝐝𝐚𝐭𝐞𝐬. https://www.youtube.com/@hubert_dudek/?sub_confirmation=1 🔗 Support Me Here! ☕Buy me a coffee: https://ko-fi.com/hubertdudek 🔗 Stay Connected With Me. Medium: https://databrickster.medium.com/ #DABs #lakeflow #databricks
Description from YouTube. Full content on the video page.
More from databricks MVP Hubert Dudek
NewsDatabricks News: ZeroOps, DABs, Indexes, Genie, sandboxes, migration from PowerBI, secrets
Zero Ops automatically detects errors in jobs and data quality with lineage analysis and proposes code fixes, while DABs now default to direct mode instead of Terraform with automatic state migration. Full-text search indexes deliver 400x faster queries on billion-row tables, Genie automatically converts PowerBI dashboards to Databricks metric views, and Unity Catalog secrets support granular read and reference-only permissions.
NewsDatabricks News: RT Lakehouse (Reyden), Lakebase, TTL
This video highlights recent Databricks updates, including the beta release of the high-performance "Raiden" real-time lakehouse engine and new lakeflow connectors. It also demonstrates administrative changes to user groups, new time data types, predictive optimization TTL deletes, user home volumes, and advanced search capabilities in Lakebase.