scala - 如何在 Apache Spark 中反转排列 DataFrame

Question

如何使用 Scala 反转这个 DataFrame。我看到了sort函数，但必须是特定的列，我只想反转它们

+---+--------+-----+
|id |    name|note |
+---+--------+-----+
|1  | james  |any  |
|3  | marry  |some |
|2  | john   |some |
|5  | tom    |any  |
+---+--------+-----+

至：

+---+--------+-----+
|id |    name|note |
+---+--------+-----+
|5  | tom    |any  |
|2  | john   |some |
|3  | marry  |some |
|1  | james  |any  |
+---+--------+-----+

标签： scalaapache-sparkapache-spark-sql

Answer 1

您可以添加具有递增 id 的列，使用 ofmonotonically_increasing_id() 并按降序排序

val dff = Seq(
  (1, "james", "any"),
  (3, "marry", "some"),
  (2, "john", "some"),
  (5, "tom", "any")
).toDF("id", "name", "note")

dff.withColumn("index", monotonically_increasing_id())
  .sort($"index".desc)
  .drop($"index")
  .show(false)

输出：

+---+-----+----+
|id |name |note|
+---+-----+----+
|5  |tom  |any |
|2  |john |some|
|3  |marry|some|
|1  |james|any |
+---+-----+----+