Yarn - Application Master Container (AM) - Job tracker
About
An Application Master (AM) is a per-application daemon to look after the lifecycle of the job.
For instance, in Spark, it's called the driver
The Application Master daemon is created when an application is started in the very first container.
The AM:
- communicates with YARN cluster
- handles application execution
- negotiate resources from the ResourceManager
- work with the NodeManager(s) to execute and monitor the tasks.
It has the responsibility to provide a web UI and to send that link to the RM.
During application launch time, the main tasks of the ApplicationMaster are:
- a) communicating with the ResourceManager to negotiate and allocate resources for future containers
- b) after container allocation, communicating YARN NodeManager's (NMs) to launch application containers on them.
Articles Related
Management
Rest Api
MapReduce Application Master REST API’s
Memory
- Required AM memory is (asked memory+384 MB)
- Limit: max threshold yarn.scheduler.maximum-allocation-mb