apache-spark - 在 PySpark 映射中使用带有 for 循环的函数

Question

我正在尝试从我的 RDD 中的每个单词中删除特殊字符：

special_characters = '~!@#$%^&*()_+-=[]{};:,<.>/?'
    def remove_special_characters(word):
        for character in special_characters[0: len(special_characters)]:
            word = word.replace(character, '')
            return word
words = lines.flatMap(lambda line: line.split(" "))
words_lower = words.map(lambda word: word.lower())

clean_words_1 = words_lower.map(lambda word: remove_special_characters(word))
clean_words_2 = words_lower.map(remove_special_characters)

每个单词只替换第一个特殊字符。

标签： apache-sparkdictionaryfor-looppyspark

Answer 1

return 需要放在 for 循环之外。