security - 用户的 Kerberized Hadoop 登录失败 ... LoginException:校验和失败
问题描述
环境:
- Hadoop 2.9.2
- Kerberos 5 版本 1.15.1
- RHEL 7
错误
Hadoop 数据节点日志中的异常阻止启动。
日志条目是:
INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: org.apache.hadoop.security.KerberosAuthException: Login failure for user: datanode/_HOST@<REALM> from keytab /etc/security/keytabs/<file.keytab> javax.security.auth.login.LoginException: Checksum failed
全栈跟踪:
org.apache.hadoop.security.KerberosAuthException: Login failure for user: datanode/_HOST@<REALM> from keytab /etc/security/keytabs/datanode.keytab javax.security.auth.login.LoginException: Checksum failed
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2596)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2645)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2789)
at org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
Caused by: javax.security.auth.login.LoginException: Checksum failed
at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:808)
at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1095)
... 10 more
Caused by: KrbException: Checksum failed
at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:102)
at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:94)
at sun.security.krb5.EncryptedData.decrypt(EncryptedData.java:175)
at sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:150)
at sun.security.krb5.KrbAsRep.decryptUsingKeyTab(KrbAsRep.java:121)
at sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:308)
at sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:447)
at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:780)
... 23 more
Caused by: java.security.GeneralSecurityException: Checksum failed
at sun.security.krb5.internal.crypto.dk.AesDkCrypto.decryptCTS(AesDkCrypto.java:451)
at sun.security.krb5.internal.crypto.dk.AesDkCrypto.decrypt(AesDkCrypto.java:272)
at sun.security.krb5.internal.crypto.Aes256.decrypt(Aes256.java:76)
at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:100)
... 30 more
诊断
如果运行kdiag来诊断问题:
bin/hadoop org.apache.hadoop.security.KDiag --principal namenode/_HOST@<REALM> --keytab /etc/security/keytab/namenode.keytab
遇到与上面相同的异常...
最后一行有用的输出是:
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
正常的健康输出应该是:
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> CksumType: sun.security.krb5.internal.crypto.HmacSha1Aes256CksumType
>>> KrbAsRep cons in KrbAsReq.getReply datanode/_HOST
/etc/krb5.conf
Kerberos 配置的内容
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
rdns = false
forwardable = true
# pkinit_anchors = /etc/pki/tls/certs/ca-bundle.crt
default_realm = EXAMPLE.COM
default_ccache_name = KEYRING:persistent:%{uid}
[realms]
EXAMPLE.COM = {
kdc = kdc.example.com
admin_server = kdc.example.com
dict_file = /usr/share/dict/words
}
[domain_realm]
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
解决方案
Hadoop 文档解释了这是 Java 不支持可更新票证的问题。
1.8.0_242当客户端请求可更新票证并且 KDC 返回不可更新票证时,Kerberos Java 客户端将因“消息流已修改 (41)”而失败。如果不允许您的委托人获得可更新票证,则必须从 krb5.conf 中删除“renew_lifetime”设置。
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions
选项1
renew_lifetime
从中删除krb5.conf
注意:如果您不想在系统范围内禁用可更新的 kerberos 票证,您可以通过将这些参数传递给 jvmkrb5.conf
来配置自定义:java
# non-windows
-Djava.security.krb5.conf=krb5.conf
# windows
-Djava.security.krb5.conf=krb5.ini
据此_
选项 2
确保renew_lifetime
和ticket_lifetime
已max_renewable_life
设置。
示例工作配置
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = EXAMPLE.COM
dns_lookup_kdc = false
dns_lookup_realm = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
default_tgs_enctypes = aes256-cts aes128-cts des3-hmac-sha1 arcfour-hmac des-hmac-sha1 des-cbc-md5 des- cbc-crc
default_tkt_enctypes = aes256-cts aes128-cts des3-hmac-sha1 arcfour-hmac des-hmac-sha1 des-cbc-md5 des-cbc-crc
permitted_enctypes = aes256-cts aes128-cts des3-hmac-sha1 arcfour-hmac des-hmac-sha1 des-cbc-md5 des-cbc-crc
kdc_timeout = 3000
[realms]
EXAMPLE.COM = {
kdc = kdc.example.com
admin_server = kdc.example.com
dict_file = /usr/share/dict/words
max_renewable_life = 7d 0h 0m 0s
}
[domain_realm]
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
推荐阅读
- java - Primefaces 6.2 PhotoCam - Firefox / Safari 的网络摄像头错误
- swift - 尝试快速停止计时器对象时出现致命错误
- python - 启动 dask.distributed Client/LocalClient 后无法捕获 KeyboardInterrupt 异常
- android - Cordova:自适应图标显示前景图像,但不显示背景颜色
- mongodb - Apache Drill - 首次启动时间较长
- c - 从现有 C 文件、头文件和 makefile 创建 Visual Studio 项目
- mongodb - 带有 upsert false 的 Mongo updateOne 不起作用?
- php - 匹配数据并使用 IF 语句打印数据
- python - 在 python 中使用 mysql 连接器从 mysql 数据库中正确获取 blob
- ios-simulator - IOS Xcode 10 模拟器慢速重绘