时间戳

首页 > 解决方案 > 将文本段落拆分为句子时迭代行

python - 将文本段落拆分为句子时迭代行

问题描述

我有一列评论，我想把每条评论分成几个句子。我还希望这些句子是单行的列表。现在，使用我一直在使用的代码，我必须指定行，但是我希望代码能够遍历包含评论的每一行。这是一个大数据集（大约 75000 行，每条评论包含大约 4-10 个句子）。

我尝试在“用于列中的文本”上方添加“用于 df.iterrows() 中的行：”，但这不起作用。

我还包含了一个我正在使用的评论示例： example_reviews

import re

alphabets= "([A-Za-z])"
prefixes = "(Mr|St|Mrs|Ms|Dr|Prof|Capt|Cpt|Lt|Mt)[.]"
suffixes = "(Inc|Ltd|Jr|Sr|Co)"
starters = "(Mr|Mrs|Ms|Dr|He\s|She\s|It\s|They\s|Their\s|Our\s|We\s|But\s|However\s|That\s|This\s|Wherever)"
acronyms = "([A-Z][.][A-Z][.](?:[A-Z][.])?)"
websites = "[.](com|net|org|io|gov|me|edu)"
digits = "([0-9])" 

def split_into_sentences1(column):

    for text in column:
            text = " " + text + "  "
            text = text.replace("\n"," ")
            text = re.sub(prefixes,"\\1<prd>",text)
            text = re.sub(websites,"<prd>\\1",text)
            text = re.sub(digits + "[.]" + digits,"\\1<prd>\\2",text) 
            if "Ph.D" in text: text = text.replace("Ph.D.","Ph<prd>D<prd>")
            text = re.sub("\s" + alphabets + "[.] "," \\1<prd> ",text)
            text = re.sub(acronyms+" "+starters,"\\1<stop> \\2",text)
            text = re.sub(alphabets + "[.]" + alphabets + "[.]" + alphabets + "[.]","\\1<prd>\\2<prd>\\3<prd>",text)
            text = re.sub(alphabets + "[.]" + alphabets + "[.]","\\1<prd>\\2<prd>",text)
            text = re.sub(" "+suffixes+"[.] "+starters," \\1<stop> \\2",text)
            text = re.sub(" "+suffixes+"[.]"," \\1<prd>",text)
            text = re.sub(" " + alphabets + "[.]"," \\1<prd>",text)
            if "e.g." in text: text = text.replace("e.g.","e<prd>g<prd>") 
            if "i.e." in text: text = text.replace("i.e.","i<prd>e<prd>") 
            if "..." in text: text = text.replace("...","<prd><prd><prd>")
            if "”" in text: text = text.replace(".”","”.")
            if "\"" in text: text = text.replace(".\"","\".")
            if "!" in text: text = text.replace("!\"","\"!")
            if "?" in text: text = text.replace("?\"","\"?")
            text = text.replace(".",".<stop>")
            text = text.replace("?","?<stop>")
            text = text.replace("!","!<stop>")
            text = text.replace("<prd>",".")
            sentences = text.split("<stop>")
            sentences = sentences[:-1]
            sentences = [s.strip() for s in sentences]
            return sentences

标签： pythonpandas

解决方案

尝试申请

df[[review]].apply(split_into_sentences1, axis=1)

推荐阅读