apache-spark - 如何在使用 Apache Spark 连接到 Presto 时解决 SQL Exception-Unsupported type JAVA_OBJECT？

Question

我对 Apache Spark 非常陌生，并试图从 Apache Spark 连接到 Presto。下面是我的连接字符串，它给出了错误。

val jdbcDF = spark.read.format("jdbc").options(Map("url" -> "jdbc:presto://host:port/hive?user=username&SSL=true&SSLTrustStorePath=/path/certificatefile", "driver" -> "com.facebook.presto.jdbc.PrestoDriver", "dbtable" -> "tablename", "fetchSize" ->  "10000", "partitionColumn" -> "columnname", "lowerBound" -> "1988", "upperBound" -> "2016", "numPartitions" -> "28")).load()

我首先在 spark/sbin 中启动了 start-master.sh。我还尝试在 spark-shell 中设置 jar 和驱动程序类路径，如下所示：

./spark-shell  --driver-class-path com.facebook.presto.jdbc.PrestoDriver --jars /path/jar/file

仍然出现以下错误：

java.sql.SQLException: Unsupported type JAVA_OBJECT
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.org$apache$spark$sql$execution$datasources$jdbc$JdbcUtils$$getCatalystType(JdbcUtils.scala:251)
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$8.apply(JdbcUtils.scala:316)
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$8.apply(JdbcUtils.scala:316)
  at scala.Option.getOrElse(Option.scala:121)
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.getSchema(JdbcUtils.scala:315)
  at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:63)
  at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
  at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
  at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)

有人可以帮我吗？谢谢

标签： apache-sparkjdbcprestospark-shell