scikit-learn - 词嵌入降低分类精度
问题描述
我目前正在尝试将文本分类为 7 个类。到目前为止,我已经能够使用多数投票(使用 SVM、多项式 NB、随机森林和 KNN)达到 90% 的精度分数。
我想尝试通过使用词嵌入来提高这种精度,从而减少样本的维度。我使用 gensim word2vec 创建我的模型和 NLTK 停用词列表和标记器:
with open('data.pkl','r') as f:
corpus=pickle.load(f)
with open('targets.pkl','r') as f:
targets=pickle.load(f)
tokenized_corpus=[word_tokenize(anomaly) for anomaly in corpus]
stop_words=['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', "you're", "you've", "you'll", "you'd", 'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', "she's", 'her', 'hers', 'herself', 'it', "it's", 'its', 'itself', 'they', 'them', 'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom', 'this', 'that', "that'll", 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does', 'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until', 'while', 'of', 'at', 'by', 'for', 'with', 'about', 'against', 'between', 'into', 'through', 'during', 'before', 'after', 'above', 'below', 'to', 'from', 'up', 'down', 'in', 'out', 'on', 'off', 'over', 'under', 'again', 'further', 'then', 'once', 'here', 'there', 'when', 'where', 'why', 'how', 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other', 'some', 'such', 'no', 'nor', 'not', 'only', 'own', 'same', 'so', 'than', 'too', 'very', 's', 't', 'can', 'will', 'just', 'don', "don't", 'should', "should've", 'now', 'd', 'll', 'm', 'o', 're', 've', 'y', 'ain', 'aren', "aren't", 'couldn', "couldn't", 'didn', "didn't", 'doesn', "doesn't", 'hadn', "hadn't", 'hasn', "hasn't", 'haven', "haven't", 'isn', "isn't", 'ma', 'mightn', "mightn't", 'mustn', "mustn't", 'needn', "needn't", 'shan', "shan't", 'shouldn', "shouldn't", 'wasn', "wasn't", 'weren', "weren't", 'won', "won't", 'wouldn', "wouldn't"]
for i,anomaly in enumerate(tokenized_corpus):
for w in anomaly:
if w in stop_words:
tokenized_corpus[i].remove(w)
model = gensim.models.Word2Vec(
tokenized_corpus,
window=5,
size=100)
model.train(tokenized_corpus, total_examples=len(corpus), epochs=10)
该模型看起来不错,当我使用单词之间的相似性时,我得到了令人满意的结果。
我使用这个类来获得我的样本的平均表示:
class MeanEmbeddingVectorizer(object):
def __init__(self, word2vec):
self.word2vec = word2vec
if len(word2vec)>0:
self.dim=len(word2vec[next(iter(word2vec))])
else:
self.dim=0
def fit(self, X, y):
return self
def transform(self, X):
return np.array([
np.mean([self.word2vec[w] for w in words if w in self.word2vec]
or [np.zeros(self.dim)], axis=0) #moyenne des vecteurs des mots (a.word2vec[w])[ou 0 si il connait pas le mot] composant un élément de X
for words in X
])
和
w2v = dict(zip(model.wv.index2word, model.wv.syn0))
a=MeanEmbeddingVectorizer(w2v)
X_transformed=a.transform(tokenized_corpus)
最后,我构建了一个通用的 sklearn 管道,并让 sklearn 对我的数据执行 GridSearchCV:
test_param_n_estimators=[i for i in range(1,50)]
parameters = {'clf2__n_estimators': test_param_n_estimators}
pip=Pipeline([['clf2',RandomForestClassifier()]])
gs_clf2 = GridSearchCV(pip, parameters,verbose=10,n_jobs=2)
gs_clf2 = gs_clf2.fit(X_transformed, targets)
print(gs_clf2.best_score_)
print(gs_clf2.best_params_)
问题是我得到随机精度分数(大约 0.5)。
也许降维并不总是能够提高精度,但我想我不明白什么或者我做错了什么,你知道出了什么问题吗?
先感谢您
解决方案
看起来您没有以正确的方式从模型中提取向量。你应该使用:
np.mean([model.wv[w] for w in words if w in model.wv.vocab]
or [np.zeros(model.dim)], axis=0)
此外,由于您同时传递tokenized_corpus
给 Word2Vec 模型constructor
和model.train
.
推荐阅读
- c# - 跟踪到 DLL
- android - Android Studio 构建 java.exe 第二次暂停
- macos - 如何在 MACOS 终端中显示面包屑?
- ios - UIPanGestureRecognizer 处理程序使我的应用程序崩溃 - swift
- android - 片段内的谷歌地图片段
- sonarqube - SonarScanner for MsBuild 可以扫描 TSQL
- angular - 集合引用转换为 angularfirestorecollection 或 observable
- r - 仅一列加载列信息中途停止
- c# - 尝试将对象添加到 ObservableCollection 时出现问题
- csv - 使用 Cloud Shell 命令将 .csv 文件从 Google Drive 同步/上传到 Google Drive Bucket