python - 如何从文档（数据集）中查找和打印不匹配/不相似的单词？

Question

我正在尝试重写基本上采用输入文本文件并与不同文档进行比较并得出相似性的算法。

现在我想打印不匹配单词的输出并输出一个带有不匹配单词的新纺织品。

从此代码中，“hello force”是输入，并根据 raw_documents 检查并打印出 0-1 之间匹配文档的排名（单词“force”与第二个文档匹配，输出为第二个文档提供更高的排名，但“hello”不在任何 raw_document 我想打印不匹配的单词“hello”作为不匹配），但我想要打印与任何 raw_document 都不匹配的不匹配的输入单词

import gensim
import nltk

from nltk.tokenize import word_tokenize

raw_documents = ["I'm taking the show on the road",
                 "My socks are a force multiplier.",
             "I am the barber who cuts everyone's hair who doesn't cut their own.",
             "Legend has it that the mind is a mad monkey.",
            "I make my own fun."]

gen_docs = [[w.lower() for w in word_tokenize(text)]
            for text in raw_documents]

dictionary = gensim.corpora.Dictionary(gen_docs)

corpus = [dictionary.doc2bow(gen_doc) for gen_doc in gen_docs]

tf_idf = gensim.models.TfidfModel(corpus)
s = 0
for i in corpus:
    s += len(i)
sims = gensim.similarities.Similarity('/usr/workdir/',tf_idf[corpus],
                                      num_features=len(dictionary))
query_doc = [w.lower() for w in word_tokenize("hello force")]

query_doc_bow = dictionary.doc2bow(query_doc)

query_doc_tf_idf = tf_idf[query_doc_bow]
result = sims[query_doc_tf_idf]
print result

标签： pythondictionarynltkgensimnltk-trainer