首页 > 解决方案 > Glue-AWS - 文件拼花没有信息为什么有人可以帮助我

问题描述

运行作业时没有错误,但成功消息而且我希望在生成文件之前它会覆盖文件

/*
 import sys
from awsglue.transforms import 
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
##new
from pyspark.sql.functions import lit
import datetime

DataSource0, connection_type = "s3", 
format = "parquet", connection_options = {"path": "s3://xxxxxx", "partitionKeys": []}, transformation_ctx = "DataSink0")
now = datetime.datetime.now()
datasourceDF0 = DataSource0.toDF().withColumn("date-new", lit(str(now)[:10]))
datasourceDF0.write.format("parquet").mode("overwrite").save("s3://xxxxxx")
job.commit()**strong text**

*/

标签: amazon-web-servicesparquet

解决方案


推荐阅读