时间戳

首页 > 解决方案 > 用熊猫中的中位数替换异常值

python - 用熊猫中的中位数替换异常值

问题描述

我想用熊猫做点什么......

我已将异常值与数据框分开，但我不知道如何将异常值年龄设置为中位数......我可以寻求帮助吗？

这是我的代码

users = pd.read_table('user.txt', sep='|', index_col='user_id')
print(users)

         age gender occupation zip_code
user_id                                
1         24      M     artist    85711
2         53      F     artist    94043
3         23      M   educator    32067
4         24      M     artist    43537
5         33      F     artist    15213
...      ...    ...        ...      ...
939       26      F    student    33319
940       32      M   engineer    02215
941       20      M    student    97229
942       48      F   educator    78209
943       22      M    student    77841

[943 rows x 4 columns]

users_age = users.reset_index()[['occupation','age']]
users_age_grouped = users_age.groupby('occupation')
users_age['Lower'] = users_age_grouped['age'].transform(
    lambda x: x.quantile(q=.25) - 1.5 * (x.quantile(q=.75) - x.quantile(q=.25)))
users_age['Median'] = users_age_grouped['age'].transform(lambda x: x.quantile(q=.5))
users_age['Upper'] = users_age_grouped['age'].transform(
    lambda x: x.quantile(q=.75) + 1.5 * (x.quantile(q=.75) - x.quantile(q=.25)))

users_age['Outlier'] = (users_age['Lower'] > users_age['age']) |\
    (users_age['Upper'] < users_age['age'])

users_age_outlier = users_age[users_age['Outlier']]
users_age_wo_outlier = users_age[users_age['Outlier'] == False]

print(users_age_outlier)

    occupation  age  Lower  Median  Upper  Outlier
29     student    7    8.5      22   36.5     True
38     student   41    8.5      22   36.5     True
187    student   42    8.5      22   36.5     True
210     artist   66    3.5      31   63.5     True
417    student   55    8.5      22   36.5     True
422     artist   64    3.5      31   63.5     True
558   engineer   69    6.5      36   66.5     True
564    student   40    8.5      22   36.5     True
766   engineer   70    6.5      36   66.5     True
802   engineer   70    6.5      36   66.5     True
838    student   38    8.5      22   36.5     True
860    student   38    8.5      22   36.5     True
914    student   50    8.5      22   36.5     True
925    student   49    8.5      22   36.5     True

我可以使用什么方法？

标签： pythonpandasdataframeoutliers

解决方案

我认为您Series.between需要numpy.where：

mask = users_age['age'].between(users_age['Lower'], users_age['Upper'], inclusive=False)

users_age['age'] = np.where(mask, users_age['Median'], users_age['age'])

推荐阅读