r - 在基准实验后绘制训练指标

Question

我想在基准实验后访问和绘制训练准确度和测试准确度。我使用准确性作为衡量标准。

如果我将准确度的聚合设置为 train.acc 并创建 test.acc 和 train.acc 的列表，则无法绘制基准结果，因为数据框中有两列“acc”类，它们是偶然相同。但是，即使未指定聚合，我也可以看到基准结果包含训练准确度，因为我已将学习者的 predict.type 设置为“both”。

我想到了一种解决方法，即从基准对象中提取 train.acc 并将其聚合并自己绘制。

我怎么做？有没有更简单的方法？

谢谢！

#Learners
learner_GLM <- makeLearner(cl = "classif.glmnet")
learner_SVM <- makeLearner(cl = "classif.ksvm")
learner_PCA <- cpoPca(rank=2) %>>% learner_GLM

#Data
dataA = datasets::iris
dataB = datasets::iris

#Task
task.A = makeClassifTask(data = dataA,target = "Species" )
task.B = makeClassifTask(data = dataB,target = "Species" )
task = list(task.A, task.B )

#Resample
inner = makeResampleDesc("CV", iters = 2, predict = "both")
outer = makeResampleDesc("CV", iters = 2, predict = "both")

#Tune wrappers
##Ctrl
ctrl = makeTuneControlRandom(maxit = 3L)
#1
numeric_ps =  makeParamSet(
  makeNumericParam("s", lower = -2, upper = 2, trafo = function(x) 2^x))

learner_GLM = makeTuneWrapper(learner_GLM, resampling =inner, par.set = numeric_ps, control = ctrl, show.info = FALSE)
#2
learner_PCA <- makeTuneWrapper(learner_PCA, resampling =inner, par.set = numeric_ps, control = ctrl, show.info = FALSE)
#3
numeric_ps =  makeParamSet(
  makeNumericParam("C", lower = -2, upper = 2, trafo = function(x) 2^x),
  makeNumericParam("sigma", lower = -2, upper = 2, trafo = function(x) 2^x)
)
learner_SVM = makeTuneWrapper(learner_SVM, resampling = inner, par.set = numeric_ps, control = ctrl)

#Measures
trainaccuracy = setAggregation(acc, train.mean)
measures =  list(acc, trainaccuracy)

#BMR
learners = list(learner_GLM,learner_SVM, learner_PCA)
bmr =  benchmark(learners, task, outer, measures = measures, show.info = FALSE)

#Plot
plotBMRBoxplots(bmr, acc, style = "violin")
bmr$results$dataA$classif.glmnet.tuned$measures.train
bmr$results$dataA$classif.glmnet.tuned$measures.test

标签： rmlr