matlab - 如何在 Matlab 中将 k 折交叉验证与“模式网络”神经网络一起使用？

Question

patternnet我正在尝试对神经网络使用 k 折交叉验证。

inputs1是一个特征向量，targets1是来自“iris_dataset”的标签向量。和xtrain、xtest、ytrain和分别是使用函数ytest拆分后的训练和测试特征和标签。cvpartition

步骤如下：

1.首先，创建模式识别网络（patternnet）。

In the first and second scripts: net = patternnet; 

2.使用 将数据划分为 k-foldscvpartition后，创建了两个训练和测试特征和标签 ( k=10)。

In the first and second scripts: fold = cvpartition(targets_Vec, 'kfold', kfold);

3.然后，configure命令用于配置网络对象，同时初始化网络的权重和偏置；

In the first script: net = configure(net, xtrain', dummyvar(ytrain)'); % xtrain and ytrain are features and labels from step (2).

或者

In the second script: net = configure(net, inputs1, targets1); % inputs1 and targets1 are features and labels before splitting up.

4.初始化参数和超参数后，使用训练数据（通过train()函数）训练网络。

In the first script: [net, tr] = train(net, xtrain', dummyvar(ytrain)'); % xtrain and ytrain are features and labels from step (2).

或者

In the second script: [net, tr] = train(net, inputs1, targets1); % inputs1 and targets1 are features and labels before splitting up.

5.最后，使用训练好的网络（通过net()函数）估计目标。

In the first script: pred = net(xtest'); % testing features from step (2).

或者

In the second script: pred = net(inputs1); 

由于训练和测试特征使用 分开cvpartition，因此网络应该使用训练特征及其标签进行训练，然后应该通过测试特征（新数据）进行测试。

虽然该train()函数用于训练网络，但它会将自己的输入（步骤 (2) 中的训练数据和标签）拆分为训练、验证和测试数据，而步骤 (2) 中的原始测试数据仍未使用。

因此，我需要一个函数，它使用第 2 步（训练和验证）中的训练特征和标签进行学习，还需要另一个函数来对新数据进行分类（第 2 步中的测试特征）。

经过搜索，我写了两个脚本，我认为第一个不正确，但我不确定第二个是否也不正确？如何解决？

第一个脚本：

clc; close all; clearvars;
load iris_dataset
max_iter = 10;
kfold = 10;
[inputs, targets] = iris_dataset;  
inputs = inputs';
targets = targets';

targets_Vec= [];
for j = 1 : size(targets, 1)
    if max(targets(j, 1:3) == 1) && find(targets(j, 1:3))==1
        targets_Vec = [targets_Vec; 1];
    elseif max(targets(j, 1:3) == 1) && find(targets(j, 1:3))==2
        targets_Vec = [targets_Vec; 2];
    elseif max(targets(j, 1:3) == 1) && find(targets(j, 1:3))==3
        targets_Vec = [targets_Vec; 3];
    end
end

net = patternnet; ... Create a Pattern Recognition Network 

rng('default');     
... Divide data into k-folds
fold = cvpartition(targets_Vec, 'kfold', kfold);
... Pre
pred2 = []; ytest2 = []; Afold = zeros(kfold,1); 

... Neural network start
for k = 1 : kfold
    ... Call index of training & testing sets
    trainIdx = fold.training(k); testIdx = fold.test(k);
    ... Call training & testing features and labels
    xtrain = inputs(trainIdx,:); ytrain = targets_Vec(trainIdx);
    xtest = inputs(testIdx,:);   ytest = targets_Vec(testIdx);

    ... configure
    net = configure(net, xtrain', dummyvar(ytrain)');

    ... Initialize neural network
    net.layers{1}.name='Hidden Layer 1';
    net.layers{2}.name='Output Layer';
    net.layers{1}.size = 20;
    net.layers{1}.transferFcn = 'tansig'; 
    net.trainFcn = 'trainscg'; 
    net.performFcn = 'crossentropy'; 

    ... Choose Input and Output Pre/Post-Processing Functions
    net.input.processFcns = {'removeconstantrows','mapminmax'};
    net.output.processFcns = {'removeconstantrows','mapminmax'};

    ... Train the Network
    [net, tr] = train(net, xtrain', dummyvar(ytrain)'); 
    ... Estimate the targets using the trained network.(Test)
    pred = net(xtest'); 

    ... Confusion matrix
    [c, cm] = confusion(dummyvar(ytest)',pred);        
    ... Get accuracy for each fold
    Afold(k) = 100*sum(diag(cm))/sum(cm(:));
    ... Store temporary result for each fold
    pred2 = [pred2(1:end,:), pred]; 
    ytest2 = [ytest2(1:end); ytest]; 
end

... Overall confusion matrix
[~,confmat] = confusion(dummyvar(ytest2)', pred2); 
confmat=transpose(confmat);
... Average accuracy over k-folds
acc = mean(Afold);
... Store results 
NN.fold = Afold;
NN.acc = acc; 
NN.con = confmat; 
fprintf('\n Final classification Accuracy (NN): %g %%',acc);  

第二个脚本：

clc; close all; clearvars;
load iris_dataset
max_iter = 10;
kfold = 10;
[inputs1, targets1] = iris_dataset;  
inputs = inputs1';
targets = targets1';

targets_Vec= [];
for j = 1 : size(targets, 1)
    if max(targets(j, 1:3) == 1) && find(targets(j, 1:3))==1
        targets_Vec = [targets_Vec; 1];
    elseif max(targets(j, 1:3) == 1) && find(targets(j, 1:3))==2
        targets_Vec = [targets_Vec; 2];
    elseif max(targets(j, 1:3) == 1) && find(targets(j, 1:3))==3
        targets_Vec = [targets_Vec; 3];
    end
end

net = patternnet; ... Create a Pattern Recognition Network

rng('default');     
... Divide data into k-folds
fold = cvpartition(targets_Vec, 'kfold', kfold);
... Pre
pred2 = []; ytest2 = []; Afold = zeros(kfold,1); 

... Neural network start
for k = 1 : kfold
    ... Call index of training & testing sets
    trainIdx = fold.training(k); testIdx = fold.test(k);
    ... Call training & testing features and labels
    xtrain = inputs(trainIdx,:); ytrain = targets_Vec(trainIdx);
    xtest = inputs(testIdx,:);   ytest = targets_Vec(testIdx);

    ... configure
    net = configure(net, inputs1, targets1);

    trInd = find(trainIdx);   tstInd = find(testIdx);
    net.divideFcn = 'divideind'; 
    net.divideParam.trainInd = trInd;
    net.divideParam.testInd = tstInd;

    ... Initialize neural network
    net.layers{1}.name='Hidden Layer 1';
    net.layers{2}.name='Output Layer';
    net.layers{1}.size = 20;                      
    net.layers{1}.transferFcn = 'tansig'; 
    net.trainFcn = 'trainscg'; 
    net.performFcn = 'crossentropy';

    ... Choose Input and Output Pre/Post-Processing Functions
    net.input.processFcns = {'removeconstantrows','mapminmax'};
    net.output.processFcns = {'removeconstantrows','mapminmax'};

    ... Train the Network
    [net, tr] = train(net, inputs1, targets1);
    pred = net(inputs1); ... Estimate the targets using the trained network (Test) 

    ... Confusion matrix
    [c, cm] = confusion(targets1, pred);
    y = net(inputs1);
    e = gsubtract(targets1, y);
    performance = perform(net, targets1, y);
    tind = vec2ind(targets1);
    yind = vec2ind(y);  

    percentErrors = sum(tind ~= yind)/numel(tind);

    ... Recalculate Training, Validation and Test Performance
    trainTargets = targets1 .* tr.trainMask{1};
%         valTargets = targets1 .* tr.valMask{1};
    testTargets = targets1 .* tr.testMask{1};
    trainPerformance = perform(net, trainTargets, y);
%         valPerformance = perform(net, valTargets, y);
    testPerformance = perform(net, testTargets, y);
    test_Fold(k) = testPerformance;

end

test_Fold_mean = mean(test_Fold);
acc = 100*(1-test_Fold_mean);   
fprintf('\n Final classification Accuracy (NN): %g %%',acc);   

标签： matlabneural-networkcross-validationk-fold