scala - Spark：架构为空

Question

我试图弄清楚如何为 Row 对象设置模式。我从文档https://spark.apache.org/docs/2.4.0/api/scala/index.html#org.apache.spark.sql.types.StructType复制代码

 import org.apache.spark.sql._
 import org.apache.spark.sql.types._

 val innerStruct =
   StructType(
     StructField("f1", IntegerType, true) ::
     StructField("f2", LongType, false) ::
     StructField("f3", BooleanType, false) :: Nil)

 val struct = StructType(
   StructField("a", innerStruct, true) :: Nil)

 // Create a Row with the schema defined by struct
 val row = Row(Row(1, 2, true))

但是架构为空

println(row, row.schema)
// ([[1,2,true]],null)

那么如何设置架构呢？

标签： scalaapache-spark

Answer 1

您可以尝试使用GenericRowWithSchema而不是仅使用 Row 并使用架构对其进行初始化：

import org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema
val row = new GenericRowWithSchema((1, 2, true),schema)