时间戳

首页 > 解决方案 > 如何计算标记化数据框中的匹配项

python - 如何计算标记化数据框中的匹配项

问题描述

我有一个数据框，其中包含 599 个标记化文本，每行一个。我也有这些列表：

grwoth = ['growth', 'grow', 'growing', 'grows']
syergies = ['synergies', 'synergy' ,'accretive', 'accretion','efficiencies' ,'efficient', 'efficiently' ]
intangibles = ['brand','branded','branding','brands','goodwill','patent','patents','goodwil']
customers = ['customer', 'customers' ,'consumer' ,'consumers' ]
technology = ['technological', 'technologically', 'technologies', 'technology', 'innovate', 'innovation', 'innovations', 'innovative', 'innovator', 'innovators']
human = ['employee', 'employees', 'employees', 'team', 'teamed', 'teaming', 'teams', 'Expertise' ]

我想在我的数据框中为每个列表创建一个新列，并计算列表中的单词在每个文本中被计算的频率。

我试图将它们输入到我的原始数据框中（没有标记化），但这也不起作用。我有以下代码：

%%time

    growth = ['growth', 'grow', 'growing', 'grows']
    synergies = ['synergies', 'synergy' ,'accretive', 'accretion','efficiencies' ,'efficient', 'efficiently' ]
    intangibles = ['brand','branded','branding','brands','goodwill','patent','patents','goodwil']
    customers = ['customer', 'customers' ,'consumer' ,'consumers' ]
    technology = ['technological', 'technologically', 'technologies', 'technology', 'innovate', 'innovation', 'innovations', 'innovative', 'innovator', 'innovators']
    human = ['employee', 'employees', 'employees', 'team', 'teamed', 'teaming', 'teams', 'expertise' ]
    the = 'Wire'

    result_list=[]
    count_growth = 0
    count_human = 0
    count_technology= 0
    count_customers = 0
count_intagibles = 0
count_synergies = 0
count_the = 0
for file in file_list:
    name = file[len(input_path):]
    date = name[11:17]
    type_1 = name[17:20]
    with open(file, "r", encoding="utf-8", errors="surrogateescape") as rfile:
            # We need to encode/decode as some text files are not in utf-8 format
            text = rfile.read()
            text = text.encode('utf-8', 'ignore')
            text = text.decode('utf-8', 'ignore')

    for word in text.split():
        if word in growth:
            count_growth = count_growth +1
        if word in synergies:
            count_synergies = count_synergies +1 
        if word in intagibles:
            count_intagibles = count_intagibles+1
        if word in customers:
            count_customers = count_customers +1
        if word in technology:
            count_technology = count_technology +1
        if word in human:
            count_human = count_human +1
        if word == 'The':
            count_the = count_the +1
    length = len(text.split())




    a={"File": name, "Text": text,'the':count_the, 'Datum': date, 'File_type': type_1, 'length':length, 'grwoth':count_growth, 'synergies': count_synergies,'intagibles':count_intagibles,'customers':count_customers, 'technology':count_technology,'human':count_human,}
    result_list.append(a)

这里的问题是，它为每行创建一个总和，而不是为长度创建一个总和。

提前感谢任何解决方案！

标签： pythonpython-3.xpandas

解决方案

您只需清除 for 循环内的变量。通过这种方式，它输出各种文件的计数，就像它输出长度一样。

我希望我能正确理解你想要做什么。

下面的代码：

for file in file_list:
    count_growth = 0
    count_human = 0
    count_technology= 0
    count_customers = 0
    count_intagibles = 0
    count_synergies = 0
    count_the = 0
    name = file[len(input_path):]
    date = name[11:17]
    type_1 = name[17:20]
    with open(file, "r", encoding="utf-8", errors="surrogateescape") as rfile:
            # We need to encode/decode as some text files are not in utf-8 format
            text = rfile.read()
            text = text.encode('utf-8', 'ignore')
            text = text.decode('utf-8', 'ignore')

    for word in text.split():
        if word in growth:
            count_growth = count_growth +1
        if word in synergies:
            count_synergies = count_synergies +1 
        if word in intagibles:
            count_intagibles = count_intagibles+1
        if word in customers:
            count_customers = count_customers +1
        if word in technology:
            count_technology = count_technology +1
        if word in human:
            count_human = count_human +1
        if word == 'The':
            count_the = count_the +1
    length = len(text.split())    
    a={"File": name, "Text": text,'the':count_the, 'Datum': date, 'File_type': type_1, 'length':length, 'grwoth':count_growth, 'synergies': count_synergies,'intagibles':count_intagibles,'customers':count_customers, 'technology':count_technology,'human':count_human,}
    result_list.append(a)

推荐阅读