Skip to content
All videos
newsDatabricks·July 7, 2020

Running Apache Spark on Kubernetes: Best Practices and Pitfalls

Summary

Running Apache Spark on Kubernetes in a stable, performant, cost-efficient and secure way is covered through lessons from building Data Mechanics, a serverless Spark platform powered by Kubernetes. Topics include core setup, configuration for performance and resource sharing, dynamic allocation and cluster autoscaling, data I/O performance, monitoring and security, and current limitations.

Summary generated by brickster.ai from the video transcript.

More from Databricks