python - Speed up date columns conversion (pandas) from string to datetime

Question

I am working with a large .csv file in python and its date column is 'str'. I am using the following code to convert the records in this column to datetime.

df[date_column].fillna('1900-01-01',inplace=True)
df[date_column] = df[date_column].apply(lambda x : pd.to_datetime(x, format = datetime_format))

But this seems to be taking quite a long time to execute. Any suggestions on how to handle this is welcomed. Thanks.

标签： pythonpandasdatetimeruntimedate-parsing

Answer 1

当您阅读 csv 时，您可以使用parse_dates

df = pd.read_csv('yourcsv.csv',parse_dates = date_column)

然后让我们使用converters

pd.read_csv('yourcsv.csv', converters={'date_column':lambda x : pd.to_datetime(x,errors = 'coerce')})