时间戳

首页 > 解决方案 > spark_write_csv 不再起作用（使用 Sparklyr）

r - spark_write_csv 不再起作用（使用 Sparklyr）

问题描述

spark_write_csv 函数不再起作用了，可能是因为我升级了 Spark 版本。有人可以帮忙吗？

这是代码示例，以及下面的错误消息：

    library(sparklyr)
    library(dplyr)
    spark_conn <- spark_connect(master = "local")
    iris <- copy_to(spark_conn, iris, overwrite = TRUE)
    spark_write_csv(iris, path = "iris.csv")

Error: org.apache.spark.SparkException: Job aborted.
    at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:231)
    at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:188)
    at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:108)
    at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:106)
    at org.apache.spark.sql.execution.command.DataWritingCommandExec.doExecute(commands.scala:131)
    at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:180)
    at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:218)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
    at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:215)
    at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:176)
    at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:132)

标签： rapache-sparksparklyr

解决方案

推荐阅读