Hive2.3版本
Hadoop2.7版本
执行hive命令报错:
报错内容:
CONSOLE#21/03/24 17:32:54 ERROR ql.Driver: FAILED: Hive Internal Error: java.lang.NoSuchMethodError(com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V)
CONSOLE#java.lang.NoSuchMethodError: com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
CONSOLE#Caused by: java.lang.NoSuchMethodError: com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
CONSOLE# 21/03/24 17:32:54 WARN session.SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory. CONSOLE# 21/03/24 17:32:54 INFO hive.metastore: Mestastore configuration hive.metastore.filter.hook changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook CONSOLE# FAILED: Hive Internal Error: java.lang.NoSuchMethodError(com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V) CONSOLE# java.lang.NoSuchMethodError: com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V CONSOLE# at com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436) CONSOLE# at com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430) CONSOLE# at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:555) CONSOLE# at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) CONSOLE# at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) CONSOLE# at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) CONSOLE# at java.lang.reflect.Method.invoke(Method.java:498) CONSOLE# at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:173) CONSOLE# at com.sun.proxy.$Proxy40.close(Unknown Source) CONSOLE# at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) CONSOLE# at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) CONSOLE# at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) CONSOLE# at java.lang.reflect.Method.invoke(Method.java:498) CONSOLE# at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2336) CONSOLE# at com.sun.proxy.$Proxy40.close(Unknown Source) CONSOLE# at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:409) CONSOLE# at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:325) CONSOLE# at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:312) CONSOLE# at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288) CONSOLE# at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:917) CONSOLE# at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:881) CONSOLE# at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1483) CONSOLE# at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1154) CONSOLE# at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1751) CONSOLE# at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1526) CONSOLE# at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) CONSOLE# at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:474) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:490) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) CONSOLE# at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) CONSOLE# at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) CONSOLE# at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) CONSOLE# at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) CONSOLE# at java.lang.reflect.Method.invoke(Method.java:498) CONSOLE# at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:331) CONSOLE# at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241) CONSOLE# at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537) CONSOLE# at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) CONSOLE# at org.apache.sqoop.Sqoop.run(Sqoop.java:147) CONSOLE# at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) CONSOLE# at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) CONSOLE# at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) CONSOLE# at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) CONSOLE# at org.apache.sqoop.Sqoop.main(Sqoop.java:252) CONSOLE# 21/03/24 17:32:54 ERROR ql.Driver: FAILED: Hive Internal Error: java.lang.NoSuchMethodError(com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V) CONSOLE# java.lang.NoSuchMethodError: com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V CONSOLE# at com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436) 。。。 CONSOLE# 21/03/24 17:32:54 INFO ql.Driver: Completed executing command(queryId=hdfs_20210324173252_4b4f6011-7ab8-494c-8128-be519ac0868b); Time taken: 0.044 seconds CONSOLE# 21/03/24 17:32:54 INFO conf.HiveConf: Using the default value passed in for log id: 3786c09f-2d67-410c-b1ba-d5264d3e3eea CONSOLE# 21/03/24 17:32:54 INFO session.SessionState: Resetting thread name to main CONSOLE# 21/03/24 17:32:54 INFO conf.HiveConf: Using the default value passed in for log id: 3786c09f-2d67-410c-b1ba-d5264d3e3eea CONSOLE# 21/03/24 17:32:54 INFO session.SessionState: Deleted directory: /tmp/hive23/hdfs/3786c09f-2d67-410c-b1ba-d5264d3e3eea on fs with scheme hdfs CONSOLE# 21/03/24 17:32:54 INFO session.SessionState: Deleted directory: /usr/local/hive23/scratchdir/3786c09f-2d67-410c-b1ba-d5264d3e3eea on fs with scheme file CONSOLE# 21/03/24 17:32:54 ERROR tool.ImportTool: Import failed: java.io.IOException: Exception thrown in Hive CONSOLE# at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:358) CONSOLE# at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241) CONSOLE# at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537) CONSOLE# at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) CONSOLE# at org.apache.sqoop.Sqoop.run(Sqoop.java:147) CONSOLE# at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) CONSOLE# at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) CONSOLE# at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) CONSOLE# at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) CONSOLE# at org.apache.sqoop.Sqoop.main(Sqoop.java:252) CONSOLE# Caused by: java.lang.reflect.InvocationTargetException CONSOLE# at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) CONSOLE# at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) CONSOLE# at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) CONSOLE# at java.lang.reflect.Method.invoke(Method.java:498) CONSOLE# at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:331) CONSOLE# ... 9 more CONSOLE# Caused by: java.lang.NoSuchMethodError: com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
报错原因:没有找到hive的libfb303的jar包。
但是从hive的lib目录有找到对应的jar包。最后怀疑是环境变量有问题,导致没有找到对应jar包的正确路径。
[hdfs@centos lib]$ pwd /usr/local/hive23/lib [hdfs@centos lib]$ ll | grep libfb -rw-r--r-- 1 hdfs hdfs 313702 Mar 24 17:35 libfb303-0.9.3.jar
查看环境变量配置,果然是环境变量配置有问题,后面多了*符号。
把export HADOOP_CLASSPATH=$HIVE_HOME/lib/*改成export HADOOP_CLASSPATH=$HIVE_HOME/lib就可以了。
[hdfs@centos lib]$ cat /etc/profile ### hive export HIVE_HOME=/usr/local/hive23 export HADOOP_CLASSPATH=$HIVE_HOME/lib/* export PATH=$PATH:$HIVE_HOME/bin:$HADOOP_CLASSPATH
## 修改配置 [hdfs@centos lib]$ cat /etc/profile ### hive export HIVE_HOME=/usr/local/hive23 export HADOOP_CLASSPATH=$HIVE_HOME/lib export PATH=$PATH:$HIVE_HOME/bin:$HADOOP_CLASSPATH [hdfs@centos lib]$ source /etc/profile