时间戳

首页 > 解决方案 > 在熊猫中连接并保存多对CSV

python - 在熊猫中连接并保存多对CSV

问题描述

我是python的初学者。我有一百对 CSV 文件。该文件如下所示：

25_13oct_speed_0.csv    
26_13oct_speed_0.csv
25_13oct_speed_0.1.csv
26_13oct_speed_0.1.csv
25_13oct_speed_0.2.csv
26_13oct_speed_0.2.csv
and others

我想连接 25 和 26 文件之间的对文件。每对文件都有一个速度阈值（Speed_0, 0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0,1.1,1.2,1.3,1.4,1.5,1.6,1.7,1.8,1.9, 2.0) 标记在文件名上。这些文件具有相同的结构数据。

Mac Annotation  X   Y
A   first       0   0
A   last        0   0
B   first       0   0
B   last        0   0

因此，concatenate analyze 足以连接这两个数据。我使用这种方法：

df1 = pd.read_csv('25_13oct_speed_0.csv')
df2 = pd.read_csv('26_13oct_speed_0.csv')
frames = [df1, df2]
result = pd.concat(frames)

对于每对文件。但这需要时间而不是优雅的方式。有没有一种自动组合配对文件并同时保存的好方法？

标签： pythonpandas

解决方案

想法是通过文件列表创建 DataFrame 并Series.str.split通过 first添加 2 个新列_：

print (files)
['25_13oct_speed_0.csv', '26_13oct_speed_0.csv', 
 '25_13oct_speed_0.1.csv', '26_13oct_speed_0.1.csv', 
 '25_13oct_speed_0.2.csv', '26_13oct_speed_0.2.csv']

df1 = pd.DataFrame({'files': files})
df1[['g','names']] = df1['files'].str.split('_', n=1, expand=True)
print (df1)
                    files   g                names
0    25_13oct_speed_0.csv  25    13oct_speed_0.csv
1    26_13oct_speed_0.csv  26    13oct_speed_0.csv
2  25_13oct_speed_0.1.csv  25  13oct_speed_0.1.csv
3  26_13oct_speed_0.1.csv  26  13oct_speed_0.1.csv
4  25_13oct_speed_0.2.csv  25  13oct_speed_0.2.csv
5  26_13oct_speed_0.2.csv  26  13oct_speed_0.2.csv

然后循环每列每组names，按组循环DataFrame.itertuples并创建新的DataFrame read_csv，如有必要，添加由来自的值填充的新列g，追加到列表，concat并按列的名称最后洞穴到新文件names：

for i, g in df1.groupby('names'):
    out = []
    for n in g.itertuples():
        df = pd.read_csv(n.files).assign(source=n.g)
        out.append(df)
    dfbig = pd.concat(out, ignore_index=True)
    print (dfbig)
    dfbig.to_csv(g['names'].iat[0])

推荐阅读