java - 使用 spark-submit 命令失败的 Kerberos 身份验证

Question

所以我正在开发一个 Spring Batch 引擎。大约有 5 个进程正在运行，它们通过传入特定的命令行参数来触发。对于这 5 个进程中的 4 个，我正在运行 java -jar 命令。对于第 5 个进程，我正在使用 spark-submit 命令，因为由于正在处理的数据的大小，该进程需要利用 spark。我在使用 spark-submit 命令时遇到了 JDBC 和 Kerberos 身份验证问题。Oracle @Configuration 对于所有这些进程都是相同的。它工作正常，并且使用 java -jar 命令进行身份验证。但是当我运行 spark-submit 命令时，我得到以下错误......

Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))
        at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770) ~[na:1.8.0_152]
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) ~[na:1.8.0_152]
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[na:1.8.0_152]
        ... 83 common frames omitted
Caused by: sun.security.krb5.KrbException: Server not found in Kerberos database (7)
        at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:70) ~[na:1.8.0_152]
        at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:251) ~[na:1.8.0_152]
        at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:262) ~[na:1.8.0_152]
        at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:308) ~[na:1.8.0_152]
        at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:126) ~[na:1.8.0_152]
        at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) ~[na:1.8.0_152]
        at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693) ~[na:1.8.0_152]
        ... 85 common frames omitted
Caused by: sun.security.krb5.Asn1Exception: Identifier doesn't match expected value (906)
        at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) ~[na:1.8.0_152]
        at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65) ~[na:1.8.0_152]
        at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60) ~[na:1.8.0_152]
        at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ~[na:1.8.0_152]
        ... 91 common frames omitted

我的火花提交命令如下...

spark-submit --conf "spark.driver.userClassPathFirst=true" --conf "spark.executor.userClassPathFirst=true" --conf "spark.driver.extraJavaOptions=-DSPARK_LOG=/home/${USERID}/payfloat_sim_input_log.spark -Dlog4j.configuration=file:${CEMS_COMMON_CFG}/log4jSpark.properties" --conf spark.ui.port=${SP_UI_PORT} --conf spark.hadoop.mapred.output.compress=${MAPRED_OUTPUT_COMPRESS} --master ${MASTER_TYPE} --deploy-mode ${DEPLOY_MODE} --queue ${SPARK_QUEUE} --executor-cores ${EXECUTOR_CORES} --num-executors ${NUM_EXECUTORS} --executor-memory ${EXECUTOR_MEMORY} ${AEGIS_INCLEARING_JAR} LoadTrialBalanceFile

标签： javaspringapache-sparkjdbckerberos