java - 无法使用来自集群外部机器（Windows）的 java 类的 JDBC 连接到 Kerberized HIVE/hadoop 集群（linux）

Question

我有一个 HIVE/hadoop 集群，我可以在节点上使用 beeline 连接到表并从表中检索数据 beeline -u "jdbc:hive2://xxxxx02:10000/default;principal=hive/[e-mail] .COM"，在使用 kinit 生成 Kerberos 票证之后。

任何帮助是极大的赞赏 ！

当我尝试在 java 类中使用 apache JDBC 时，我无法连接同一个集群

1. Eclipse 是我在 Windows 机器上的 IDE。
2. 我正在使用 Hive JDBC 驱动程序

<dependency>
<groupId>org.apache.hive</groupId>
 <artifactId>hive-jdbc</artifactId>
<version>3.1.2</version>
</dependency>

3. 集群上的 Hive 版本：

    $ rpm -qa|grep hadoop                             
        hadoop_2_6_0_3_8-yarn-2.7.3.2.6.0.3-8
        hadoop_2_6_0_3_8-libhdfs-2.7.3.2.6.0.3-8
        teradata-connector-1.4.3-hadoop2.x
        ambari-metrics-hadoop-sink-2.5.0.3-7
        hadoop_2_6_0_3_8-mapreduce-2.7.3.2.6.0.3-8
        teradata-hadoop-tools-2.16.12.48-1
        teradata-hadoop-builder-2.6.25.22-1
        hadoop_2_6_0_3_8-2.7.3.2.6.0.3-8
        hadoop_2_6_0_3_8-client-2.7.3.2.6.0.3-8
        hadoop_2_6_0_3_8-hdfs-2.7.3.2.6.0.3-8

4. 我已经从节点生成了一个 keytab 文件并复制到我正在运行 Java 类的 Windows 机器上

5. 这是代码`

    try {
                Class.forName("org.apache.hive.jdbc.HiveDriver");
                System.setProperty("sun.security.jgss.debug", "true");
                System.setProperty("java.security.auth.login.config", "jaas1.conf");
                System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");
                System.setProperty("java.security.krb5.conf", "krb5.ini");
                System.out.println("Connecting...");
                Connection con = DriverManager.getConnection(
                        "jdbc:hive2://xxx02.xxx.com:10000/default;principal=hive/[e-mail];");
                System.out.println("...Connected");
                con.close();

            } catch (ClassNotFoundException e) {
                logger.error("Driver not found");
            } catch (Exception e) {
                e.printStackTrace();
            }

5. jaas1.conf 和 krb5.ini 位于 Eclipse 根项目文件夹中。

jaas1.conf 如下：

>     Client {
>      com.sun.security.auth.module.Krb5LoginModule required
>     useKeyTab=true
>     keyTab=”c:\users\shiredd\spr.keytab”
>     principal="[e-mail]"
>     doNotPrompt=true;
>     };

6. 错误是：

    connecting...

    Search Subject for Kerberos V5 INIT cred (<<DEF>>, sun.security.jgss.krb5.Krb5InitCredential)
    15686 [WARN] HiveConnection: Failed to connect to xxx02.abc.com:10000

    java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://xxx02.abc.com:10000/default;principal=hive/[e-mail];: java.io.IOException: Configuration Error:
        Line 4: expected [option key], found [null]
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:256)
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
        at java.sql.DriverManager.getConnection(Unknown Source)
        at java.sql.DriverManager.getConnection(Unknown Source)
        at ca.loblaw.eiBuildingBlocks.HiveJDBCOverHTTP.main(HiveJDBCOverHTTP.java:36)
    Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error:
        Line 4: expected [option key], found [null]
        at sun.security.provider.ConfigFile$Spi.<init>(Unknown Source)
        at sun.security.provider.ConfigFile.<init>(Unknown Source)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at java.lang.Class.newInstance(Unknown Source)
        at javax.security.auth.login.Configuration$2.run(Unknown Source)
        at javax.security.auth.login.Configuration$2.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.login.Configuration.getConfiguration(Unknown Source)
        at sun.security.jgss.LoginConfigImpl$1.run(Unknown Source)
        at sun.security.jgss.LoginConfigImpl$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.jgss.LoginConfigImpl.<init>(Unknown Source)
        at sun.security.jgss.GSSUtil.login(Unknown Source)
        at sun.security.jgss.krb5.Krb5Util.getInitialTicket(Unknown Source)
        at sun.security.jgss.krb5.Krb5InitCredential$1.run(Unknown Source)
        at sun.security.jgss.krb5.Krb5InitCredential$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.jgss.krb5.Krb5InitCredential.getTgt(Unknown Source)
        at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Unknown Source)
        at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Unknown Source)
        at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Unknown Source)
        at sun.security.jgss.GSSManagerImpl.getMechanismContext(Unknown Source)
        at sun.security.jgss.GSSContextImpl.initSecContext(Unknown Source)
        at sun.security.jgss.GSSContextImpl.initSecContext(Unknown Source)
        at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(Unknown Source)
        at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
        at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
        at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:51)
        at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:48)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Unknown Source)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.hive.metastore.security.TUGIAssumingTransport.open(TUGIAssumingTransport.java:48)
        at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:343)
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:228)
        ... 4 more
    Caused by: java.io.IOException: Configuration Error:
        Line 4: expected [option key], found [null]
        at sun.security.provider.ConfigFile$Spi.ioException(Unknown Source)
        at sun.security.provider.ConfigFile$Spi.match(Unknown Source)
        at sun.security.provider.ConfigFile$Spi.parseLoginEntry(Unknown Source)
        at sun.security.provider.ConfigFile$Spi.readConfig(Unknown Source)
        at sun.security.provider.ConfigFile$Spi.init(Unknown Source)
        at sun.security.provider.ConfigFile$Spi.init(Unknown Source)
        ... 43 more**

标签： javajdbchivekerberosjaas