Skip to content
All videos
newsDatabricks·August 5, 2020

Fine Tuning and Enhancing Performance of Apache Spark Jobs

Summary

Apache Spark defaults perform decently on large data sets, but tuning parameters to the resources and job can bring significant gains. Best practices from real-world problems cover garbage collector selection, serialization, executor counts, partitioning and skew, fair scheduler pools, Java heap settings, reading Spark UI DAGs for bottlenecks, and optimizing joins.

Summary generated by brickster.ai from the video transcript.

More from Databricks