hadoop - Active Directory - Hortonworks 跨领域信任建立

Question

我们正在尝试为 SAS Data Loader for Hadoop (DLH) 配置 IWA。SAS 服务器正在 Active Directory 域下运行，并且 SSO 已成功配置。我们需要配置 DLH 以使用客户端生成的票证与 Hortonworks Hadoop MIT Kerberos 对话。该功能不起作用。

所以基本上我们对 AD (ABC.COM) 和 Hadoop MIT Kerberos (xyz - Hadoop 领域名称没有任何 FQDN 并且都是小写字母) 2 方式信任建立有问题。我们已根据以下链接（https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_security/content/kerb-config-realm-kdc.html）配置信任，一切正常，但不知何故，使用 AD 的 HTTP 票证，我们无法登录到 hadoop，我们收到以下错误消息：

com.sas.svcs.dm.hadoop.spi.exception.HadoopConfigurationException：找不到 GSSCredential。检查 Kerberos 配置

我们已经尝试了很多故障排除。现在终于确定我们存在信任问题，并且测试以下是要求执行的步骤。

在 SAS 服务器 (linux) 上

kinit -f HTTP/[e-mail]

klist -eaf

kvno hive/xyz@xyz

如果上述所有步骤都有效，则意味着我们启用了信任

这是我们看到的错误

kvno：KDC 返回错误字符串：PROCESS_TGS 同时获取 hive/xyz@xyz 的凭据

和

kvno：在获取 hive/xyz@xyz 的凭据时解密完整性检查失败

kinit -f HTTP/[e-mail] (this works fine)

# kinit -k -t xxx.host.keytab HTTP/[e-mail]
[65181] 1559895039.846538: Getting initial credentials for HTTP/[e-mail]
[65181] 1559895039.846539: Looked up etypes in keytab: des-cbc-crc, des, des-cbc-crc, rc4-hmac, aes256-cts, aes128-cts
[65181] 1559895039.846541: Sending unauthenticated request
[65181] 1559895039.846542: Sending request (220 bytes) to ABC.COM
[65181] 1559895039.846543: Sending initial UDP request to dgram 10.68.5.219:88
[65181] 1559895039.846544: Received answer (819 bytes) from dgram 10.68.5.219:88
[65181] 1559895039.846545: Response was from master KDC
[65181] 1559895039.846546: Processing preauth types: PA-ETYPE-INFO2 (19)
[65181] 1559895039.846547: Selected etype info: etype aes256-cts, salt "ABC.COMHTTPxxx.abc.com", params ""
[65181] 1559895039.846548: Produced preauth for next request: (empty)
[65181] 1559895039.846549: Getting AS key, salt "ABC.COMHTTPxxx.abc.com", params ""
[65181] 1559895039.846550: Retrieving HTTP/[e-mail] from FILE:xxx.host.keytab (vno 0, enctype aes256-cts) with result: 0/Success
[65181] 1559895039.846551: AS key obtained from gak_fct: aes256-cts/8AEB
[65181] 1559895039.846552: Decrypted AS reply; session key is: aes256-cts/E734
[65181] 1559895039.846553: FAST negotiation: unavailable
[65181] 1559895039.846554: Initializing FILE:/tmp/krb5cc_0 with default princ HTTP/[e-mail]
[65181] 1559895039.846555: Storing HTTP/[e-mail] -> krbtgt/[e-mail] in FILE:/tmp/krb5cc_0

##########################################################

klist -e (this shows the ticket is generated)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# klist -e
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: HTTP/[e-mail]

Valid starting       Expires              Service principal
06/07/2019 13:40:39  06/07/2019 13:50:39  krbtgt/[e-mail]
        Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

###########################################################


kvno hive/xyz@xyz (this command fails)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# kvno hive/xyz@xyz
[65247] 1559895064.242178: Getting credentials HTTP/[e-mail] -> hive/xyz@xyz using ccache FILE:/tmp/krb5cc_0
[65247] 1559895064.242179: Retrieving HTTP/[e-mail] -> hive/xyz@xyz from FILE:/tmp/krb5cc_0 with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_0)
[65247] 1559895064.242180: Retrieving HTTP/[e-mail] -> krbtgt/xyz@xyz from FILE:/tmp/krb5cc_0 with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_0)
[65247] 1559895064.242181: Retrieving HTTP/[e-mail] -> krbtgt/[e-mail] from FILE:/tmp/krb5cc_0 with result: 0/Success
[65247] 1559895064.242182: Starting with TGT for client realm: HTTP/[e-mail] -> krbtgt/[e-mail]
[65247] 1559895064.242183: Retrieving HTTP/[e-mail] -> krbtgt/xyz@xyz from FILE:/tmp/krb5cc_0 with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_0)
[65247] 1559895064.242184: Requesting TGT krbtgt/[e-mail] using TGT krbtgt/[e-mail]
[65247] 1559895064.242185: Generated subkey for TGS request: aes256-cts/C142
[65247] 1559895064.242186: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts, des-cbc-crc, des, des-cbc-md4
[65247] 1559895064.242188: Encoding request body and padata into FAST request
[65247] 1559895064.242189: Sending request (1001 bytes) to ABC.COM
[65247] 1559895064.242190: Sending initial UDP request to dgram 10.68.5.219:88
[65247] 1559895064.242191: Received answer (873 bytes) from dgram 10.68.5.219:88
[65247] 1559895064.242192: Response was from master KDC
[65247] 1559895064.242193: Decoding FAST response
[65247] 1559895064.242194: FAST reply key: aes256-cts/9192
[65247] 1559895064.242195: TGS reply is for HTTP/[e-mail] -> krbtgt/[e-mail] with session key des-cbc-crc/330F
[65247] 1559895064.242196: TGS request result: 0/Success
[65247] 1559895064.242197: Storing HTTP/[e-mail] -> krbtgt/[e-mail] in FILE:/tmp/krb5cc_0
[65247] 1559895064.242198: Received TGT for service realm: krbtgt/[e-mail]
[65247] 1559895064.242199: Requesting tickets for hive/xyz@xyz, referrals on
[65247] 1559895064.242200: Generated subkey for TGS request: des-cbc-crc/FB8F
[65247] 1559895064.242201: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts, des-cbc-crc, des, des-cbc-md4
[65247] 1559895064.242203: Encoding request body and padata into FAST request
[65247] 1559895064.242204: Sending request (935 bytes) to xyz
[65247] 1559895064.242205: Resolving hostname xyz
[65247] 1559895064.242206: Sending initial UDP request to dgram 10.68.166.7:88
[65247] 1559895064.242207: Received answer (138 bytes) from dgram 10.68.166.7:88
[65247] 1559895064.242208: Response was not from master KDC
[65247] 1559895064.242209: TGS request result: -1765328324/KDC returned error string: PROCESS_TGS
[65247] 1559895064.242210: Requesting tickets for hive/xyz@xyz, referrals off
[65247] 1559895064.242211: Generated subkey for TGS request: des-cbc-crc/01C2
[65247] 1559895064.242212: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts, des-cbc-crc, des, des-cbc-md4
[65247] 1559895064.242214: Encoding request body and padata into FAST request
[65247] 1559895064.242215: Sending request (935 bytes) to xyz
[65247] 1559895064.242216: Resolving hostname xyz
[65247] 1559895064.242217: Sending initial UDP request to dgram 10.68.166.7:88
[65247] 1559895064.242218: Received answer (138 bytes) from dgram 10.68.166.7:88
[65247] 1559895064.242219: Response was not from master KDC
[65247] 1559895064.242220: TGS request result: -1765328324/KDC returned error string: PROCESS_TGS
kvno: KDC returned error string: PROCESS_TGS while getting credentials for hive/xyz@xyz

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@


Also just for troubleshooting I add enctypes on my AD server using the following command:

ksetup /SetEncTypeAttr xyz DES-CBC-CRC DES-CBC-MD5 RC4-HMAC-MD5 AES128-CTS-HMAC-SHA1-96 AES256-CTS-HMAC-SHA1-96

So, after running the above command when I try to run the kvno command, my error message changes 

:from 

kvno: KDC returned error string: PROCESS_TGS while getting credentials for hive/xyz@xyz

:to

kvno: Decrypt integrity check failed while getting credentials for hive/xyz@xyz

full kvno cmmand trace is as below:

# kvno hive/xyz@xyz
[128763] 1559917554.849763: Getting credentials HTTP/[e-mail] -> hive/xyz@xyz using ccache FILE:/tmp/krb5cc_0
[128763] 1559917554.849764: Retrieving HTTP/[e-mail] -> hive/xyz@xyz from FILE:/tmp/krb5cc_0 with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_0)
[128763] 1559917554.849765: Retrieving HTTP/[e-mail] -> krbtgt/xyz@xyz from FILE:/tmp/krb5cc_0 with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_0)
[128763] 1559917554.849766: Retrieving HTTP/[e-mail] -> krbtgt/[e-mail] from FILE:/tmp/krb5cc_0 with result: 0/Success
[128763] 1559917554.849767: Starting with TGT for client realm: HTTP/[e-mail] -> krbtgt/[e-mail]
[128763] 1559917554.849768: Retrieving HTTP/[e-mail] -> krbtgt/xyz@xyz from FILE:/tmp/krb5cc_0 with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_0)
[128763] 1559917554.849769: Requesting TGT krbtgt/[e-mail] using TGT krbtgt/[e-mail]
[128763] 1559917554.849770: Generated subkey for TGS request: aes256-cts/4F0F
[128763] 1559917554.849771: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts, des-cbc-crc, des, des-cbc-md4
[128763] 1559917554.849773: Encoding request body and padata into FAST request
[128763] 1559917554.849774: Sending request (1022 bytes) to ABC.COM
[128763] 1559917554.849775: Sending initial UDP request to dgram 10.68.5.219:88
[128763] 1559917554.849776: Received answer (969 bytes) from dgram 10.68.5.219:88
[128763] 1559917554.849777: Response was from master KDC
[128763] 1559917554.849778: Decoding FAST response
[128763] 1559917554.849779: FAST reply key: aes256-cts/944C
[128763] 1559917554.849780: TGS reply is for HTTP/[e-mail] -> krbtgt/[e-mail] with session key aes256-cts/B3D3
[128763] 1559917554.849781: TGS request result: 0/Success
[128763] 1559917554.849782: Storing HTTP/[e-mail] -> krbtgt/[e-mail] in FILE:/tmp/krb5cc_0
[128763] 1559917554.849783: Received TGT for service realm: krbtgt/[e-mail]
[128763] 1559917554.849784: Requesting tickets for hive/xyz@xyz, referrals on
[128763] 1559917554.849785: Generated subkey for TGS request: aes256-cts/DF91
[128763] 1559917554.849786: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts, des-cbc-crc, des, des-cbc-md4
[128763] 1559917554.849788: Encoding request body and padata into FAST request
[128763] 1559917554.849789: Sending request (1013 bytes) to xyz
[128763] 1559917554.849790: Resolving hostname xyz
[128763] 1559917554.849791: Sending initial UDP request to dgram 10.68.166.7:88
[128763] 1559917554.849792: Received answer (138 bytes) from dgram 10.68.166.7:88
[128763] 1559917554.849793: Response was not from master KDC
[128763] 1559917554.849794: TGS request result: -1765328353/Decrypt integrity check failed
[128763] 1559917554.849795: Requesting tickets for hive/xyz@xyz, referrals off
[128763] 1559917554.849796: Generated subkey for TGS request: aes256-cts/34D1
[128763] 1559917554.849797: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts, des-cbc-crc, des, des-cbc-md4
[128763] 1559917554.849799: Encoding request body and padata into FAST request
[128763] 1559917554.849800: Sending request (1013 bytes) to xyz
[128763] 1559917554.849801: Resolving hostname xyz
[128763] 1559917554.849802: Sending initial UDP request to dgram 10.68.166.7:88
[128763] 1559917554.849803: Received answer (138 bytes) from dgram 10.68.166.7:88

[128763] 1559917554.849805: TGS request result: -1765328353/Decrypt integrity check failed
kvno: Decrypt integrity check failed while getting credentials for hive/xyz@xyz

标签： hadoopactive-directorykerberoshortonworks-data-platform

Answer 1

问题在于 AD 和 Hadoop Trust 无法正常工作。因此，在故障排除期间，我在 AD 上的 Hadoop 主体上添加了 enctypes。我在其中一个网站上找到了以下注释

“主体（帐户）是使用系统默认编码类型创建的。更改 enctype 时，还必须重新创建主体，或者至少更新主体的密码。”

所以，我做了密码的重置

netdom trust xyz /Domain:ABC.COM /reset /realm /passwordt:xxxxXXXxxxx

另外，KVNO 在 AD 和 Hadoop 之间不匹配，所以我在 Hadoop 端更新了 kvno

在 Hadoop Server 上重新启动以下服务

/sbin/service krb5kdc 重启 /sbin/service kadmin 重启

瞧……我能够运行 kvno 命令。

kinit -k -t xxx.host.keytab HTTP/[e-mail]

[74264] 1561019777.500742：在 FILE:/tmp/krb5cc_1001 中存储 HTTP/[e-mail] -> krbtgt/[e-mail]

klist-eaf

票证缓存：FILE:/tmp/krb5cc_1001 默认主体：HTTP/[e-mail]

有效开始到期服务主体 06/20/2019 14:06:17 06/21/2019 00:06:17 krbtgt/[e-mail] 续订至 06/27/2019 14:06:17，标志：FRI Etype（skey，tkt）：aes256-cts-hmac-sha1-96，aes256-cts-hmac-sha1-96 地址：（无）

kvno 蜂巢/xyz@xyz

[74362] 1561019789.592571：收到所需服务 hive/xyz@xyz 的凭据 [74362] 1561019789.592572：将 HTTP/[e-mail] -> hive/xyz@xyz 存储在 FILE:/tmp/krb5cc_1001 hive/xyz@ xyz：kvno = 1