时间戳

首页 > 解决方案 > Pandas Dataframe drop duplicates in a column of lists?

python - Pandas Dataframe drop duplicates in a column of lists?

问题描述

I'm trying to remove duplicates in columns a and c.

        a      b    c
0  [1, 0]      1    ab
1  [0, 0]      2    bc
2  [1, 0]      3    ab

Resultant Output:

        a      b    c
0  [1, 0]      1    ab
1  [0, 0]      2    bc

What i have tried: With out a column being list. df.drop_duplicates(['a','c']) works.

Without c column being str. pd.DataFrame(np.unique(df), columns=df.columns) works for droping duplicate lists.

How to proceed if one the columns is a list and other string.

标签： pythonpandasnumpydataframe

解决方案

方法一

列表在 pandas 中不可散列，但您可以使用元组。

df['d'] = df['a'].apply(lambda x : tuple(x) if type(x) is list else x)

          a  b   c       d
0    [1, 0]  1  ab  (1, 0)
1    [0, 0]  2  bc  (0, 0)
2    [1, 0]  3  ab  (1, 0)

然后

df = df.drop_duplicates(subset=['c', 'd'])

结果 ：

         a  b   c       d
0    [1, 0]  1  ab  (1, 0)
1    [0, 0]  2  bc  (0, 0)

方法二

您可以将包含列表的列转换为 str。

df['a'] = df['a'].astype(str)
df = df.drop_duplicates(subset=['a', 'c'])

输出

    a      b    c
0  [1, 0]      1    ab
1  [0, 0]      2    bc

推荐阅读