时间戳

首页 > 解决方案 > Python 与 HDInsight 中的 Hive 数据库的连接

python - Python 与 HDInsight 中的 Hive 数据库的连接

问题描述

我正在尝试通过我的 python 脚本创建与托管在 HDInsight 群集中的 Hive 的连接并低于错误-

    Traceback (most recent call last):
  File "ClassLoader.java", line 357, in java.lang.ClassLoader.loadClass
  File "Launcher.java", line 349, in sun.misc.Launcher$AppClassLoader.loadClass
  File "ClassLoader.java", line 424, in java.lang.ClassLoader.loadClass
  File "URLClassLoader.java", line 382, in java.net.URLClassLoader.findClass
java.lang.ClassNotFoundException: java.lang.ClassNotFoundException: org.apache.thrift.transport.TTransportException
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "org.jpype.JPypeContext.java", line 330, in org.jpype.JPypeContext.callMethod
  File "Method.java", line 498, in java.lang.reflect.Method.invoke
  File "DelegatingMethodAccessorImpl.java", line 43, in sun.reflect.DelegatingMethodAccessorImpl.invoke
  File "NativeMethodAccessorImpl.java", line 62, in sun.reflect.NativeMethodAccessorImpl.invoke
  File "NativeMethodAccessorImpl.java", line -2, in sun.reflect.NativeMethodAccessorImpl.invoke0
  File "DriverManager.java", line 247, in java.sql.DriverManager.getConnection
  File "DriverManager.java", line 664, in java.sql.DriverManager.getConnection
  File "HiveDriver.java", line 105, in org.apache.hive.jdbc.HiveDriver.connect
Exception: Java Exception
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "s.py", line 5, in <module>
    "/root/jdbc/hive-jdbc-1.2.1000.2.6.5.3009-43.jar")
  File "/usr/local/lib64/python3.6/site-packages/jaydebeapi/__init__.py", line 412, in connect
    jconn = _jdbc_connect(jclassname, url, driver_args, jars, libs)
  File "/usr/local/lib64/python3.6/site-packages/jaydebeapi/__init__.py", line 230, in _jdbc_connect_jpype
    return jpype.java.sql.DriverManager.getConnection(url, *dargs)
java.lang.NoClassDefFoundError: java.lang.NoClassDefFoundError: org/apache/thrift/transport/TTransportException

我的脚本是 -

import jaydebeapi
conn = jaydebeapi.connect("org.apache.hive.jdbc.HiveDriver",
       "jdbc:hive2://10.20.30.40:10001/default;transportMode=http;ssl=false;httpPath=/hive2",
       ["username", "password"],
       "/root/jdbc/hive-jdbc-1.2.1000.2.6.5.3009-43.jar")

我已经导出了所有 jar 文件的 CLASSPATH。

标签： pythonpython-3.xhiveazure-hdinsight

解决方案

该错误java.lang.ClassNotFoundException: java.lang.ClassNotFoundException指定执行无法找到 jar /root/jdbc/hive-jdbc-1.2.1000.2.6.5.3009-43.jar。我相信它只放置在您正在执行的主机中代码。我建议将 jar 文件放在集群中所有节点的相同目录结构中，并检查权限，以便执行作业的用户可以访问该路径。

推荐阅读