python - python 中最常见的单词，运行时出现错误 (TypeError: unhashable type: 'list')

Question

我在下面写了代码，但在运行时出现错误（TypeError: unhashable type: 'list'），你能帮帮我吗？我想要我的令牌中最常用的单词。

! pip install wget
import wget
url = 'https://raw.githubusercontent.com/dirkhovy/NLPclass/master/data/moby_dick.txt'
wget.download(url, 'moby_dick.txt')
documents = [line.strip() for line in open('moby_dick.txt', encoding='utf8').readlines()]

import spacy

nlp = spacy.load('en')

tokens = [[token.text for token in nlp(sentence)] for sentence in documents[:200]]



from collections import Counter
# your code here

# Pass the split_it list to instance of Counter class. 

Counter = Counter(tokens) 

# most_common() produces k frequently encountered 
# input values and their respective counts. 
most_occur = Counter.most_common(10) 

print(most_occur) 

错误：TypeError Traceback (most recent call last) in () 4 # 将 split_it 列表传递给 Counter 类的实例。5 ----> 6 Counter = Counter(tokens) 7 8 # most_common() 产生 k 经常遇到

1 帧 /usr/lib/python3.6/collections/ init .py in update(*args, **kwds) 620 super(Counter, self).update(iterable) # 计数器为空时的快速路径 621 else: -- > 622 _count_elements(self, iterable) 623 if kwds: 624 self.update(kwds)

标签： pythonnlp

Answer 1

! pip install wget
import wget
url = 'https://raw.githubusercontent.com/dirkhovy/NLPclass/master/data/moby_dick.txt'
wget.download(url, 'moby_dick.txt')
documents = [line.strip() for line in open('moby_dick.txt', encoding='utf8').readlines()]

import spacy

nlp = spacy.load('en')

tokens = [token.text for sentence in documents[:200] for token in nlp(sentence)]



from collections import Counter

Counter = Counter(tokens) 

most_occur = Counter.most_common(10) 

print(most_occur)

update your syntax of list comprehension