scala - 不明确的错误列 Spark 2.3.2、Scala 2.11.8

Question

模棱两可的错误 Spark 2.3.2、Scala 2.11.8

这真的是Scala引擎问题还是修复了？

请让我知道你的想法和答案

import spark.implicits._

val llist = Seq(("bob", "2015-01-13", 4), ("alice", "2015-04-23",10))
val left = llist.toDF("name","date","duration")//.registerTempTable("table1")
val right = Seq(("alice", 100),("bob", 23)).toDF("name","upload")//.registerTempTable("table2")
val df = left.join(right, left.col("name") === right.col("name"),
  "LeftOuter").filter("duration > 5")
  .select(left.col("name").toString(), "date")
df.show()


Exception in thread "main" org.apache.spark.sql.AnalysisException: **Reference 'name' is ambiguous, could be: name, name.;**
    at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolve(LogicalPlan.scala:213)
    at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveChildren(LogicalPlan.scala:97)
    at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences$$anonfun$37.apply(Analyzer.scala:826)
    at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences$$anonfun$37.apply(Analyzer.scala:828)

标签： scalaapache-spark

Answer 1

问题是转换toString()似乎丢失了表信息。一般来说，toString是出于调试目的。这应该工作

val llist = Seq(("bob", "2015-01-13", 4), ("alice", "2015-04-23",10))
val left = llist.toDF("name","date","duration")//.registerTempTable("table1")
val right = Seq(("alice", 100),("bob", 23)).toDF("name","upload")//.registerTempTable("table2")
val df = left.join(right, left.col("name") === right.col("name"),
    "LeftOuter").filter("duration > 5")
    .select(left.col("name"), $"date")
df.show()

输出：

+-----+----------+
| name|      date|
+-----+----------+
|alice|2015-04-23|
+-----+----------+