Spark - Core (Slot)

Spark Cluster

Spark - Core (Slot)

About

Cores (or slots) are the number of available threads for each executor (Spark daemon also ?)

They are unrelated to physical CPU cores. See below

slots indicate threads available to perform parallel work for Spark. Spark documentation often refers to these threads as cores, which is a confusing term, as the number of slots available on a particular machine does not necessarily have any relationship to the number of physical CPU cores on that machine.

Spark Cluster Tasks Slot

Configuration

Number of core can be configured :

Documentation / Reference





Discover More
Spark Cluster
Spark - Driver

The driver is a (daemon|service) wrapper created when you get a spark context (connection) that look after the lifecycle of the Spark job. cluster managerapplication manager The driver: start as its...



Share this page:
Follow us:
Task Runner