string - Pyspark DD-MMM-YYYY (string format) to timestamp
问题描述
Hello I am new to the Pyspark, I have a string Variable that contain date DD-MMM-YYYY format and i want to convert that into a time stamp ?
24-MAY-2019 - String format to timestamp
解决方案
尝试to_timestamp(preferred) (or) from_unixtime and unix_timestamp
功能:
Example:
from pyspark.sql.functions import *
from pyspark.sql.types import *
df.selectExpr("to_timestamp(dt,'dd-MMM-yyyy') as tt").show()
+-------------------+
| tt|
+-------------------+
|2019-05-24 00:00:00|
+-------------------+
df1.withColumn("ts",to_timestamp(col("dt"),'dd-MMM-yyyy')).show()
+-----------+-------------------+
| dt| ts|
+-----------+-------------------+
|24-MAY-2019|2019-05-24 00:00:00|
+-----------+-------------------+
#using from_unixtime and unix_timestamp
df1.withColumn("ts",from_unixtime(unix_timestamp(col("dt"),'dd-MMM-yyyy'),'yyyy-MM-dd HH:mm:ss.SSS').cast("timestamp")).show(10,False)
+-----------+-----------------------+
|dt |ts |
+-----------+-----------------------+
|24-MAY-2019|2019-05-24 00:00:00.000|
+-----------+-----------------------+
#using unix_timestamp and casting to timestamp
df1.withColumn("ts",unix_timestamp(col("dt"),'dd-MMM-yyyy').cast("timestamp")).show()
#+-----------+-------------------+
#| dt| ts|
#+-----------+-------------------+
#|24-MAY-2019|2019-05-24 00:00:00|
#+-----------+-------------------+
推荐阅读
- python - 如何在另一个文件夹中创建文件夹?- Python
- git - “'/home/upulie/Desktop' 的 git 存储库有太多的活动更改,只会启用部分 Git 功能”
- python - typed-ast 的构建轮子失败
- java - Rest tamplate 400 null - 错误请求
- c++ - c++传递成员函数
- ios - 引用不同文件(Xcode-swift)时如何修复 EXC_BAD_ACCESS?
- algorithm - 没有连续重复的字典顺序
- r - R不会将数据中的URL读入Web-Crawler
- c - 在 SHM 段中分配字符串会破坏整个结构
- python - Python Pandas:与多列等效的 vlookup