python - 将 ColumnTransformer() 结果附加到管道中的原始数据?
问题描述
这是我的输入数据:
这是对 r、f 和 m 列应用转换的所需输出,并将结果附加到原始数据
这是代码:
import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import PowerTransformer
df = pd.DataFrame(np.random.randint(0,100,size=(10, 3)), columns=list('rfm'))
column_trans = ColumnTransformer(
[('r_std', StandardScaler(), ['r']),
('f_std', StandardScaler(), ['f']),
('m_std', StandardScaler(), ['m']),
('r_boxcox', PowerTransformer(method='box-cox'), ['r']),
('f_boxcox', PowerTransformer(method='box-cox'), ['f']),
('m_boxcox', PowerTransformer(method='box-cox'), ['m']),
])
transformed = column_trans.fit_transform(df)
new_cols = ['r_std', 'f_std', 'm_std', 'r_boxcox', 'f_boxcox', 'm_boxcox']
transformed_df = pd.DataFrame(transformed, columns=new_cols)
pd.concat([df, transformed_df], axis = 1)
我还需要额外的变压器,所以我需要将原始列保留在管道中。有没有更好的方法来处理这个?特别是在管道中进行连接和列命名?
解决方案
一种方法是使用一个虚拟转换器,它只返回具有原始值的转换列:
import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import PowerTransformer
np.random.seed(1714)
class NoTransformer(BaseEstimator, TransformerMixin):
def fit(self, X, y=None):
return self
def transform(self, X):
assert isinstance(X, pd.DataFrame)
return X
我正在向数据集添加一个id列,以便我可以在 中显示剩余参数的使用ColumnTransformer()
,我觉得这非常有用。
df = pd.DataFrame(np.hstack((np.arange(10).reshape((10, 1)),
np.random.randint(1,100,size=(10, 3)))),
columns=["id"] + list('rfm'))
使用带有值passthrough的余数(默认值为drop)可以保留未转换的列;从文档。
并且使用NoTransformer()
虚拟类,我们可以将列 'r'、'f'、'm' 转换为具有相同的值。
column_trans = ColumnTransformer(
[('r_original', NoTransformer(), ['r']),
('f_original', NoTransformer(), ['f']),
('m_original', NoTransformer(), ['m']),
('r_std', StandardScaler(), ['r']),
('f_std', StandardScaler(), ['f']),
('m_std', StandardScaler(), ['m']),
('r_boxcox', PowerTransformer(method='box-cox'), ['r']),
('f_boxcox', PowerTransformer(method='box-cox'), ['f']),
('m_boxcox', PowerTransformer(method='box-cox'), ['m']),
], remainder="passthrough")
如果您想转换更多列,则提示:拟合ColumnTransformer()
类(在您的情况下为column_trans)有一个transformers_方法,可让您以['r_std', 'f_std', 'm_std', 'r_boxcox', 'f_boxcox', 'm_boxcox']
编程方式访问名称:
column_trans.transformers_
#[('r_original', NoTransformer(), ['r']),
# ('f_original', NoTransformer(), ['f']),
# ('m_original', NoTransformer(), ['m']),
# ('r_std', StandardScaler(copy=True, with_mean=True, with_std=True), ['r']),
# ('f_std', StandardScaler(copy=True, with_mean=True, with_std=True), ['f']),
# ('m_std', StandardScaler(copy=True, with_mean=True, with_std=True), ['m']),
# ('r_boxcox',
# PowerTransformer(copy=True, method='box-cox', standardize=True),
# ['r']),
# ('f_boxcox',
# PowerTransformer(copy=True, method='box-cox', standardize=True),
# ['f']),
# ('m_boxcox',
# PowerTransformer(copy=True, method='box-cox', standardize=True),
# ['m']),
# ('remainder', 'passthrough', [0])]
最后,我认为您的代码可以这样简化:
column_trans_2 = ColumnTransformer(
([
('original', NoTransformer(), ['r', 'f', 'm']),
('std', StandardScaler(), ['r', 'f', 'm']),
('boxcox', PowerTransformer(method='box-cox'), ['r', 'f', 'm']),
]), remainder="passthrough")
transformed_2 = column_trans_2.fit_transform(df)
column_trans_2.transformers_
#[('std',
# StandardScaler(copy=True, with_mean=True, with_std=True),
# ['r', 'f', 'm']),
# ('boxcox',
# PowerTransformer(copy=True, method='box-cox', standardize=True),
# ['r', 'f', 'm'])]
并通过transformers_以编程方式分配列名:
new_col_names = []
for i in range(len(column_trans_2.transformers)):
new_col_names += [column_trans_2.transformers[i][0] + '_' + s for s in column_trans_2.transformers[i][2]]
# The non-transformed columns ('id' in this case) will be appended on the right of
# the array and do not show up in the 'transformers_' method.
# Add the id columns to the col_names manually
new_col_names += ['id']
# ['original_r', 'original_f', 'original_m', 'std_r', 'std_f', 'std_m', 'boxcox_r',
# 'boxcox_f', 'boxcox_m', 'id']
pd.DataFrame(transformed_2, columns=new_col_names)
推荐阅读
- docker - 合并时“已经存在”
- google-oauth - 没有获得令牌谷歌 Oauth
- azure-devops - 如何将预定义的管道构建参数传递给 Azure DevOps 管道中的模板?
- c++ - 如何在二维数组 C++ 中找到最大值输入的索引
- r - 如何使用 R 根据共享元素对两个组进行排序?
- azure - Azure 策略 - 防止在没有身份验证的情况下创建应用服务
- html - 当我缩小屏幕时,某些列文本在 Td 元素内不可见
- vuetify.js - Vuetify:仅针对某些标签从右到左显示
- java - com.mysql.cj.jdbc.exceptions.CommunicationsException:GCP 中的通信链路故障 - CloudSQL
- javascript - 如何在 select 和 option html 标签上添加和删除自定义错误注释类?