apache-spark - 存储流数据时 Hive 出现问题

Question

我有一个 Kafka Streaming Consumer 作业，它将数据存储在 Hive 表中。问题是有时作业会因 Hive 而失败，必须再次重新启动。所有尝试访问 Hive 的流式消费者作业都会发生这种情况。无法追查原因。感谢您的建议。

以下是高级错误日志

INFO ApplicationMaster: Starting the user application in a separate Thread
INFO ApplicationMaster: Waiting for spark context initialization...
WARN SparkConf: The configuration key 'spark.yarn.applicationMaster.waitTries' has been deprecated as of Spark 1.3 and may be removed in the future. Please use the new key 'spark.yarn.am.waitTime' instead.
INFO metastore: Trying to connect to metastore with URI thrift://<server1:port1>
WARN metastore: Failed to connect to the MetaStore Server...
INFO metastore: Trying to connect to metastore with URI thrift://<server1:port1>
WARN metastore: Failed to connect to the MetaStore Server...
INFO metastore: Waiting 1 seconds before next connection attempt.
INFO metastore: Trying to connect to metastore with URI thrift://<server2:port1>
WARN metastore: Failed to connect to the MetaStore Server...
INFO metastore: Trying to connect to metastore with URI thrift://<server1:port1>
WARN metastore: Failed to connect to the MetaStore Server...
INFO metastore: Waiting 1 seconds before next connection attempt.
INFO metastore: Trying to connect to metastore with URI thrift://<server2:port1>
WARN metastore: Failed to connect to the MetaStore Server...
INFO metastore: Trying to connect to metastore with URI thrift://<server1:port1>
WARN metastore: Failed to connect to the MetaStore Server...
INFO metastore: Waiting 1 seconds before next connection attempt.
WARN Hive: Failed to access metastore. This class should not accessed in runtime.
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
Caused by: java.lang.reflect.InvocationTargetException
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: Peer indicated failure: DIGEST-MD5: IO error acquiring password

 INFO metastore: Trying to connect to metastore with URI thrift://<server2:port1>
 WARN metastore: Failed to connect to the MetaStore Server...
 INFO metastore: Trying to connect to metastore with URI thrift://<server1:port1>
 WARN metastore: Failed to connect to the MetaStore Server...
 INFO metastore: Waiting 1 seconds before next connection attempt.
 INFO metastore: Trying to connect to metastore with URI thrift://<server2:port1>
 WARN metastore: Failed to connect to the MetaStore Server...
 INFO metastore: Trying to connect to metastore with URI thrift://<server1:port1>
 WARN metastore: Failed to connect to the MetaStore Server...
 INFO metastore: Waiting 1 seconds before next connection attempt.
 INFO metastore: Trying to connect to metastore with URI thrift://<server2:port1>
 WARN metastore: Failed to connect to the MetaStore Server...
 INFO metastore: Trying to connect to metastore with URI thrift://<server1:port1>
 WARN metastore: Failed to connect to the MetaStore Server...
 INFO metastore: Waiting 1 seconds before next connection attempt.
 ERROR ApplicationMaster: User class threw exception: java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionStateBuilder':
java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionStateBuilder':
Caused by: org.apache.spark.sql.AnalysisException: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
Caused by: java.lang.reflect.InvocationTargetException
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: Peer indicated failure: DIGEST-MD5: IO error acquiring password

标签： apache-sparkhiveapache-kafka