首页 > 解决方案 > 网格搜索和交叉验证 SVM

问题描述

我在 10 倍交叉验证上使用网格搜索的最佳参数实现 svm,我需要了解预测结果为什么不同我在训练集上得到了两个准确度结果测试通知我需要训练集上最佳参数的预测结果以进行进一步分析代码和结果如下所述。任何解释

from __future__ import print_function

from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import classification_report
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score
from time import *
from sklearn import metrics
X=datascaled.iloc[:,0:13]
y=datascaled['num']

np.random.seed(1)
# Split the dataset in two equal parts
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=0)

# Set the parameters by cross-validation
tuned_parameters =  [{'kernel': ['rbf'], 'gamma': [1e-2, 1e-3, 1e-4, 1e-5],
                     'C': [0.001, 0.10, 0.1, 10, 25, 50, 100, 1000]},
                    {'kernel': ['sigmoid'], 'gamma': [1e-2, 1e-3, 1e-4, 1e-5],
                     'C': [0.001, 0.10, 0.1, 10, 25, 50, 100, 1000] },{'kernel': ['linear'], 'C': [0.001, 0.10, 0.1, 10, 25, 50, 100, 1000]}]              





print()

clf = GridSearchCV(SVC(), tuned_parameters, cv=10,
                       scoring='accuracy')
t0 = time()

clf.fit(X_train, y_train)
t = time() - t0
print("Best parameters set found on development set:")
print()
print(clf.best_params_)
print()
print('Training accuracy')
print(clf.best_score_)
print(clf.best_estimator_)
print()
print()
print('****Results****')
svm_pred=clf.predict(X_train)
#print("\t\taccuracytrainkfold: {}".format(metrics.accuracy_score(y_train, svm_pred)))
print("=" * 52)
print("time cost: {}".format(t))
print()
print("confusion matrix\n", metrics.confusion_matrix(y_train, svm_pred))
print()
print("\t\taccuracy: {}".format(metrics.accuracy_score(y_train, svm_pred)))
print("\t\troc_auc_score: {}".format(metrics.roc_auc_score(y_train, svm_pred)))
print("\t\tcohen_kappa_score: {}".format(metrics.cohen_kappa_score(y_train, svm_pred)))
print()
print("\t\tclassification report")
print("-" * 52)
print(metrics.classification_report(y_train, svm_pred)) 

Best parameters set found on development set:

{'C': 1000, 'gamma': 0.01, 'kernel': 'rbf'}

Training accuracy
0.9254658385093167


****Results****
====================================================
time cost: 7.728448867797852

confusion matrix
 [[77  2]
 [ 4 78]]

        accuracy: 0.9627329192546584
        roc_auc_score: 0.9629515282494597
        cohen_kappa_score: 0.9254744638173121

        classification report
----------------------------------------------------
             precision    recall  f1-score   support

          0       0.95      0.97      0.96        79
          1       0.97      0.95      0.96        82

avg / total       0.96      0.96      0.96       161

标签: python

解决方案


您正在使用 10 折交叉验证进行训练,并要求在每次折叠后计算预测准确度。我建议执行以下操作。

使用sklearn.model_selection.KFold将数据拆分为 10 折,并创建一个循环通过每个折,如下所示:

for train_index, test_index in kf.split(X):
    print("TRAIN:", train_index, "TEST:", test_index)
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]

在该循环中,使用下面重复的先前使用的行来构建和训练模型。但是在GridSearchCV()中使用cv=1而不是cv=10

    clf = GridSearchCV(SVC(), tuned_parameters, cv=1, scoring='accuracy')
    clf.fit(X_train, y_train)

在使用来自一个折叠的数据训练模型后,然后根据代码中使用的以下行使用相同折叠的数据预测其准确性。

    svm_pred=clf.predict(X_train)
    print("\t\taccuracy: {}".format(metrics.accuracy_score(y_train, svm_pred)))

完整代码如下:

for train_index, test_index in kf.split(X):
    print("TRAIN:", train_index, "TEST:", test_index)
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]

    clf = GridSearchCV(SVC(), tuned_parameters, cv=1, scoring='accuracy')
    clf.fit(X_train, y_train)

    svm_pred=clf.predict(X_train)
    print("\t\taccuracy: {}".format(metrics.accuracy_score(y_train, svm_pred)))

希望有帮助:)


推荐阅读