scala - 使用 Spark/scala 将字符串转换为 DataFrame

Question

我想将字符串列转换为时间戳列，但它始终返回空值。

  val t = unix_timestamp(col("tracking_time"),"MM/dd/yyyy").cast("timestamp")
   val df=   df2.withColumn("ts", t)

任何想法 ？

谢谢你 。

标签： scalaapache-spark

Answer 1

确保您String column与指定的格式匹配MM/dd/yyyy。

• 如果不匹配null则将被退回。

Example:

val df2=Seq(("12/12/2020")).toDF("tracking_time")
val t = unix_timestamp(col("tracking_time"),"MM/dd/yyyy").cast("timestamp")

df2.withColumn("ts", t).show()
//+-------------+-------------------+
//|tracking_time|                 ts|
//+-------------+-------------------+
//|   12/12/2020|2020-12-12 00:00:00|
//+-------------+-------------------+

df2.withColumn("ts",unix_timestamp(col("tracking_time"),"MM/dd/yyyy").cast("timestamp")).show()
//+-------------+-------------------+
//|tracking_time|                 ts|
//+-------------+-------------------+
//|   12/12/2020|2020-12-12 00:00:00|
//+-------------+-------------------+
//(or)  by using to_timestamp function.

df2.withColumn("ts",to_timestamp(col("tracking_time"),"MM/dd/yyyy")).show()
//+-------------+-------------------+
//|tracking_time|                 ts|
//+-------------+-------------------+
//|   12/12/2020|2020-12-12 00:00:00|
//+-------------+-------------------+