python - Pandas .loc[] 与 dict 相比非常慢

Question

我有一个大约 10 列和 100K 行的 DataFrame。我想使用.loc[]索引来获取循环中的每一行。然而.loc[]，与 Python 的 dict 相比，它的速度非常慢。

这是重现的代码：

import pandas as pd
import random
import time

data = {}

for i in range(100000):
    data[i] = {
        'id': i,
        'a': random.randint(1, 40000),
        'b': random.randint(1, 40000),
        'c': random.randint(1, 40000),
        'd': random.randint(1, 40000),
        'e': random.randint(1, 40000),
        'f': random.randint(1, 40000),
    }

df = pd.DataFrame.from_dict(
    data=data,
    orient="index",
    dtype=int,
)
df.set_index('id', inplace=True)

dict_objs = df.to_dict('index')

start_time_dataframe = time.time()
for i in range(100000):
    obj = df.loc[i]
end_time_dataframe = time.time() - start_time_dataframe

start_time_dict = time.time()
for i in range(100000):
    obj = dict_objs[i]

end_time_dict = time.time() - start_time_dict

print(f"Time needed for DataFrame: {end_time_dataframe}")  # 12.08s
print(f"Time needed for Dict: {end_time_dict}")  # 0.01s

为什么 DataFrame.loc[]运行这么慢？

标签： pythonpandasdataframe