python-3.x - UnicodeDecodeError：“utf-8”编解码器无法解码位置 257 中的字节 0x92：无效的起始字节

Question

I am new in python and want to apply p reprocessing steps 
so here is decoding error 

import nltk
from nltk.tokenize import word_tokenize,sent_tokenize
from nltk.corpus import stopwords
from nltk.tag import pos_tag
from nltk.stem import PorterStemmer

`ps=PorterStemmer()
print ("\n Reading file with out stopwords.")
text_file=open('preprocessing.txt',encoding='utf-8').read()
stop_words= set(stopwords.words("english"))
words=word_tokenize(text_file)
filtered_sentence = [w for w in words if not w in stop_words]
print(filtered_sentence)
print ("\n Removed stopword.")
print(stop_words)
print ("\n Stemming.")
for w in text_file:
print (ps.stem(w))
print(w)
print(sent_tokenize(text_file))
print ("\n tokenization.")
print(word_tokenize(text_file))
print ("\n part of speech tagging.")
print (pos_tag(words))   `

“我想以特定格式显示结果，但输出是”，第 322 行，在 decode (result, used) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't解码位置 257 中的字节 0x92：无效的起始字节”

标签： python-3.x

Answer 1

请尝试使用 读取数据encoding='unicode_escape'。例如：

text_file=open('preprocessing.txt',encoding ='unicode_escape').read()

这为我解决了 UnicodeDecodeError 问题。

否则你可以尝试如下：

text_file=open(r'preprocessing.txt',encoding ='unicode_escape').read()