python - Python RegEx 从语音到文本生成的字符串中删除社会保险号
问题描述
出于符合 GDPR 的原因,我正在尝试从语音转文本生成的杂乱数据中删除社会安全号码 (SSN)。这是一个示例字符串(翻译成英语,解释了为什么列出 SSN 时出现“和”):
sample1 = "hello my name is sofie my social security number is thirteen zero four five and seventy eighteen seven and forty and I live on mountain street number twelve"
我的目标是删除该部分"thirteen ... forty "
,同时保留字符串中可能出现的其他数字,从而导致:
sample1_wo_ssn = "hello my name is sofie my social security number is and I live on mountain street number twelve"
社会安全号码的长度可能会因数据的生成方式而异(3-10 个单独的数字)。
我的做法:
- 使用 dict 用数字替换书面数字
- 使用正则表达式查找只有空格或
"and"
分隔它们的 3 个或更多数字出现的位置,并将它们与这 3 个数字后面的任何数字一起删除。
这是我的代码:
import re
number_dict = {
'zero': '0',
'one': '1',
'two': '2',
'three': '3',
'four': '4',
'five': '5',
'six': '6',
'seven': '7',
'eight': '8',
'nine': '9',
'ten': '10',
'eleven': '11',
'twelve': '12',
'thirteen': '13',
'fourteen': '14',
'fifteen': '15',
'sixteen': '16',
'seventeen': '17',
'eighteen': '18',
'nineteen': '19',
'twenty': '20',
'thirty': '30',
'forty': '40',
'fifty': '50',
'sixty': '60',
'seventy': '70',
'eighty': '80',
'ninety': '90'
}
sample1 = "hello my name is sofie my social security number is thirteen zero four five and seventy eighteen seven and forty and I live on mountain street number twelve"
sample1_temp = [number_dict.get(item,item) for item in sample1.split()]
sample1_numb = ' '.join(sample1_temp)
re_results = re.findall(r'(\d+ (and\s)?\d+ (and\s)?\d+\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?\s?(and\s)?(\d+)?)', sample1_numb)
print(re_results)
输出:
[('13 0 4 5 and 70 18 7 and 40 and ', '', '', '', '5', 'and ', '70', '', '18', '', '7', 'and ', '40', 'and ', '', '', '', '', '')]
这就是我卡住的地方。
在这个例子中,我可以做一些事情sample1_wh_ssn = re.sub(re_results[0][0],'',sample1_numb)
来获得想要的结果,但这不会一概而论。
任何帮助将不胜感激。
解决方案
这是您当前逻辑的实现,即:
- 将单词数字从
1
through99
转换为数字 - 删除以空格分隔的 3 个或更多数字的所有实例
- 将数字两位数的数字转换回单词。
学分:
- 将单词转换为数字:有没有办法将数字单词转换为整数?通过递归
- 将数字转换为单词:我如何通过kindall告诉 Python 将整数转换为单词
请参阅Python 代码:
import re
number_words = [ "zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen", "sixteen", "seventeen", "eighteen", "nineteen"]
number_words_tens =[ "twenty", "thirty", "forty", "fifty", "sixty", "seventy", "eighty", "ninety" ]
number_words_rx = re.compile(r'\b(?:(?:{0})?(?:{1})|(?:{0}))\b'.format("|".join(number_words_tens),"|".join(number_words)))
main_rx = re.compile(r'\s*\d+(?:\s+(?:and\s+)?\d+){2,}')
numbers_1_99 = number_words
numbers_1_99.extend(tens if ones == "zero" else (tens + "-" + ones) # stackoverflow.com/a/8982279/3832970
for tens in "twenty thirty forty fifty sixty seventy eighty ninety".split()
for ones in numbers_1_99[0:10])
def text2int(textnum, numwords={}): # stackoverflow.com/a/493788/3832970
units = [
"zero", "one", "two", "three", "four", "five", "six", "seven", "eight",
"nine", "ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen",
"sixteen", "seventeen", "eighteen", "nineteen",
]
tens = ["", "", "twenty", "thirty", "forty", "fifty", "sixty", "seventy", "eighty", "ninety"]
numwords["and"] = (1, 0)
for idx, word in enumerate(units):
numwords[word] = (1, idx)
for idx, word in enumerate(tens):
numwords[word] = (1, idx * 10)
current = result = 0
for word in textnum.split():
if word not in numwords:
raise Exception("Illegal word: " + word)
scale, increment = numwords[word]
current = current + increment
return result + current
sample1 = "hello my name is sofie my social security number is thirteen zero four five and seventy eighteen seven and forty and I live on mountain street number twelve"
sample1 = number_words_rx.sub(lambda x: str(text2int(x.group())), sample1)
re_results = main_rx.sub('', sample1)
print( re.sub(r'\d{1,2}', lambda x: numbers_1_99[int(x.group())], re_results) )
输出:hello my name is sofie my social security number is and I live on mountain street number twelve
推荐阅读
- perl - 从 Perl 脚本到 Sybase DB 的连接问题
- json - Logstash Grok JSON 错误 - 不同类型的映射器
- macos - 意外字符 { 在第 1 行
- javascript - 离子:添加 HAMMER_GESTURE_CONFIG 后垂直滚动不起作用
- python - 如何在嵌套列表中使用 python 拆分具有特定字符的字符串?
- python - 如何使用 pandas 和 yfinance 在多级 Dataframe 中添加一列?
- python - 使用 Selenium Python 客户端在不同浏览器中处理无头模式
- google-apps-script - onEdit 多个功能特定单元格
- angular - 如何在 mat-table 中的 ParentArray(nested) 中绑定 childArray
- css - 媒体查询永远不会在 scss 中呈现