时间戳

首页 > 解决方案 > 多类分类的校准预测

calibration - 多类分类的校准预测

问题描述

我们如何进行多类分类的校准预测？

我尝试遵循https://machinelearningmastery.com/calibrated-classification-model-in-scikit-learn/，但这不适用于多类问题，因为当我使用 sklearn.calibration.calibration_curve 时出现以下错误：

ValueError：仅支持二进制分类。提供标签 ['x' 'y' 'z' 'a' 'b']。

标签： calibration

解决方案

一种方法是使用OneVsRestClassifier重新训练您的多类模型，然后将每个类视为一个单独的模型。我在下面粘贴了一个在 NLP 项目中使用的简单示例，希望对您有所帮助。

# Binarize the output
X_train = train_df['Text']
X_test = test_df['Text']

lb = LabelBinarizer()
y_train = lb.fit_transform(train_df['Target'])
y_test = lb.transform(test_df['Target'])

# Train a model with tfidf-vectorizer and LinearSVC
tfidf = TfidfVectorizer()
clf = LinearSVC()
clf = CalibratedClassifierCV(clf)
clf = OneVsRestClassifier(clf)

# Fit the model
pipe = Pipeline([('tfidf', tfidf), ('clf', clf)])
pipe.fit(X_train, y_train)

# Plot the Calibration Curve for every class
plt.figure(figsize=(20, 10))
ax1 = plt.subplot2grid((3, 1), (0, 0), rowspan=2)
ax2 = plt.subplot2grid((3, 1), (2, 0))
ax1.plot([0, 1], [0, 1], "k:", label="Perfectly calibrated")

targets = range(len(lb.classes_))
for target in targets:
    prob_pos = pipe.predict_proba(X_test)[:, target]
    fraction_of_positives, mean_predicted_value = calibration_curve(y_test[:, target], prob_pos, n_bins=10)
    name = lb.classes_[target]

    ax1.plot(mean_predicted_value, fraction_of_positives, "s-", label="%s" % (name, ))
    ax2.hist(prob_pos, range=(0, 1), bins=10, label=name, histtype="step", lw=2)

ax1.set_ylabel("The proportion of samples whose class is the positive class")
ax1.set_xlabel("The mean predicted probability in each bin")
ax1.set_ylim([-0.05, 1.05])
ax1.legend(loc="lower right")
ax1.set_title('Calibration plots (reliability curve)')

ax2.set_xlabel("Mean predicted value")
ax2.set_ylabel("Count")
ax2.legend(loc="upper center", ncol=2)

plt.tight_layout()
plt.show()

推荐阅读