scala - 原因：java.lang.IllegalArgumentException: Can't get JDBC type for null

Question

Null在 spark 将值加载到数据库时出现以下错误。Datatype目标表是smallint

Caused by: java.lang.IllegalArgumentException: Can't get JDBC type for null

代码 ：

val hivedata = spark.sql(s"""select 1 as column1 , B a column2 , NULL as column3 from table""")

hivedata .write.mode(SaveMode.Append).jdbc(url = con, table = targettable, Pconnectionropertiess)

谁能帮帮我吗

标签： scalaapache-sparkhadoophiveapache-spark-sql

Answer 1

cast(NULL as smallint)你必须这样做......这会将 null 转换为short类型，如下面的架构所示。

val df1 =spark.sql(
     " select 1 as column1 , 2 column2 , cast(NULL as smallint) as column3 from table  ")
  df1.show
df1.printSchema()

结果 ：

+-------+-------+-------+
|column1|column2|column3|
+-------+-------+-------+
|      1|      2|   null|
+-------+-------+-------+

root
 |-- column1: integer (nullable = false)
 |-- column2: integer (nullable = false)
 |-- column3: short (nullable = true)

否则你的方式将是 nulltype 而不是小的 int 类型..

val df1 =spark.sql(" select 1 as column1 , 2 column2 ,  NULL   as column3 from table  ")
  df1.show
df1.printSchema()

+-------+-------+-------+
|column1|column2|column3|
+-------+-------+-------+
|      1|      2|   null|
+-------+-------+-------+

root
 |-- column1: integer (nullable = false)
 |-- column2: integer (nullable = false)
 |-- column3: null (nullable = true)

这就是你得到例外的原因。