Spark Engine - Shuffle

About

shuffle means moving data rows by rows between partition.

Articles Related

Conf

• spark.sql.shuffle.partitions - Configures the number of partitions to use when shuffling data for joins or aggregations.