首页 > 解决方案 > spark Listner OnApplicationEnd event stops only Driver, executors are not getting cleaned up

问题描述

I like to use sparklistner to end application execution, while I stop the application, it only stops the driver and it did not clean or stop the Executors.

whatever the Executors Added when I start the application using onApplicationStart Spark Listner should end both Driver and Executors, when I invoke a method onApplicaitonEnd, but actually it only terminates the driver and not the Executors

class testListner extends SparkListner {
    override def onApplicationEnd(appEnded:SparkListnerApplicationEnd):Unit ={}
}
object slyest extends App {sc.addSparkListner(new testListner)}

标签: apache-sparkapache-spark-sqlspark-streaming

解决方案


推荐阅读