python - 基于两列标记数据
问题描述
我有一个包含几列的数据框,如下所示:
id | x1 | text | x2 | num | x3 | class
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT |
2nn| TT | word | QQ | 1 | TT | # They don't all have the same vals
2nn| TT | word | QQ | 1 | TT | # This is just to illustrate it
我也有以下lists
字符串:
class1 = ["",...]
class2 = ["",...]
class3 = ["",...]
class4 = ["",...]
class5 = ["",...] # Multiple strings, I just used '...' for simplicity
我正在尝试在列中分配类,class
以便如果事务(行)text
列中的单词包含在任何 中找到的任何单词lists
,则将列表名称分配为类。
我这样做是为了标记一些我最终将用于分类的数据。
我只想对从第 10,000 行开始的数据执行此标记。我正在使用的是:
# last 6000 rows
for index, row in df.tail(6000).iterrows():
if df[df['text'].str.contains(class1)==True]:
df.loc[row, 'class'] = "class1"
if df[df['text'].str.contains(class2)==True]:
df.loc[row, 'class'] = "class2"
if df[df['text'].str.contains(class3)==True]:
df.loc[row, 'class'] = "class3"
if df[df['text'].str.contains(class4)==True]:
df.loc[row, 'class'] = "class4"
if df[df['text'].str.contains(class5)==True]:
df.loc[row, 'class'] = "class5"
我收到打印以下错误的响应:
TypeError: unhashable type: 'list'
以下是我在尝试 Chris A 的回复时的代码:
# Word bins for the various labels
complaint = ["sucks", "worst", "doesn't", "didn't", "won't", "bad", "horrible", "unusable", "cannot", "can't", "not", "did not", "waste", "hate", "hated", "awful", "useless", "sucked", "freezing", "freezes", "froze", "does not", "crap", "stupid"]
compliment = ["awesome", "great", "amazing", "cool", "good", "nice", "nicest", "successful", "thanks", ":)", "successfully"]
neutral = ["Eh", "meh", "works"]
bug = ["please", "fix", "won't", "cannot", "can't", "not", "freezing", "freezes", "froze", "does not", "did not", "help", "plz"]
feature = ["it would be", "id like", "i'd like", "could", "can you", "implement", "feature", "lacks", "wish"]
def label_data(df):
d = {'Compliment': compliment,
'Complaint': complaint,
'Neutral': neutral,
'Bug': bug,
'Feature': feature}
for name, values in d.items():
df.loc[df['review'].isin(values), 'label'] = name
我的主类从文本文件中调用数据,然后使用以下方法调用此方法:
df_orig = pd.read_table("PRIVATEPATH/data.txt", delimiter=",")
label_data(df_labelled)
解决方案
为此使用列表中的 python 可能会有所帮助dictionary
。
使用时,str.contains
您还必须通过使用|
正则表达式“OR”运算符连接每个值来“构建”正则表达式字符串。
笔记
正如您所发现的那样,这里的一个问题是,以这种方式构建正则表达式模式将需要您转义列表中的任何特殊正则表达式字符。示例 - 您:)
的赞美列表中有“ ”。这将需要成为 ' \:\)
'
d = {'class1': class1,
'class2': class2,
'class3': class3,
'class4': class4}
for name, values in d.items():
# Create a regex string joining all the values in the list with the regex OR '|'
pat = '|'.join(values)
df.loc[df['text'].str.contains(pat), 'class'] = name
简化示例
df = pd.DataFrame({'id': {0: '2nn',1: '2nn',2: '2nn',3: '2nn',4: '2nn',5: '2nn',6: '2nn',7: '2nn',8: '2nn',9: '2nn',10: '2nn',11: '2nn',12: '2nn'},
'x1': {0: 'TT',1: 'TT',2: 'TT',3: 'TT',4: 'TT',5: 'TT',6: 'TT',7: 'TT',8: 'TT',9: 'TT',10: 'TT',11: 'TT',12: 'TT'},
'text': {0: 'abc',1: 'abc',2: 'e',3: 'h',4: 'm',5: 'p',6: 'q',7: 'd',8: 's',9: 'j',10: 'h',11: 'o',12: 'z'},
'x2': {0: 'QQ',1: 'QQ',2: 'QQ',3: 'QQ',4: 'QQ',5: 'QQ',6: 'QQ',7: 'QQ',8: 'QQ',9: 'QQ',10: 'QQ',11: 'QQ',12: 'QQ'},
'num': {0: 1,1: 1,2: 1,3: 1,4: 1,5: 1,6: 1,7: 1,8: 1,9: 1,10: 1,11: 1,12: 1},
'x3': {0: 'TT',1: 'TT',2: 'TT',3: 'TT',4: 'TT',5: 'TT',6: 'TT',7: 'TT',8: 'TT',9: 'TT',10: 'TT',11: 'TT',12: 'TT'},
'class': {0: np.nan,1: np.nan,2: np.nan,3: np.nan,4: np.nan,5: np.nan,6: np.nan,7: np.nan,8: np.nan,9: np.nan,10: np.nan,11: np.nan,12: np.nan}})
class1 = list('abcde')
class2 = list('fghi')
class3 = list('jklmn')
class4 = list('opqrs')
d = {'class1': class1,
'class2': class2,
'class3': class3,
'class4': class4}
for name, values in d.items():
pat = '|'.join(values)
df.loc[df['text'].str.contains(pat), 'class'] = name
print(df)
[出去]
id x1 text x2 num x3 class
0 2nn TT a QQ 1 TT class1
1 2nn TT b QQ 1 TT class1
2 2nn TT e QQ 1 TT class1
3 2nn TT h QQ 1 TT class2
4 2nn TT m QQ 1 TT class3
5 2nn TT p QQ 1 TT class4
6 2nn TT q QQ 1 TT class4
7 2nn TT d QQ 1 TT class1
8 2nn TT s QQ 1 TT class4
9 2nn TT j QQ 1 TT class3
10 2nn TT h QQ 1 TT class2
11 2nn TT o QQ 1 TT class4
12 2nn TT z QQ 1 TT NaN
推荐阅读
- websocket - JHipster 在微服务架构中支持 websockets
- android - 如何在模型类中保存复杂的json数组和jsonobject
- perl - 在 Perl 中替换变量时将文件复制到另一个文件?
- angular - 从类似 ngForOf 的指令中删除 @Inputs()
- html - “break-inside:避免列”在 Firefox 中不起作用
- java - Java 计数领域结果
- angular - Angular 4从订阅返回数据
- encoding - SQL Developer 口音编码
- magento-1.8 - Magento 1.x - 未找到自定义 JS/CSS
- xampp - 致命错误:未捕获错误:调用未定义函数 mysql_connect?