时间戳

首页 > 解决方案 > 如果我使用“holdout”重采样，如何访问训练集的度量（例如：“classif.acc”或其他度量）？

r - 如果我使用“holdout”重采样，如何访问训练集的度量（例如：“classif.acc”或其他度量）？

问题描述

我正在学习 R 中机器学习的 mlr3 包。我使用“holdout”重采样将数据分成训练集和测试集，如何获得训练集的度量？即使我将“predict_set”指定为“train.

> library(mlr3)
> task <- tsk("iris")
> learner <- lrn("classif.rpart")
> rs <- rsmp("holdout")
> rr <- resample(task, learner, rs)
INFO  [12:28:49.941] Applying learner 'classif.rpart' on task 'iris' (iter 1/1) 
> rr$aggregate(msr("classif.acc"))
classif.acc 
   0.96 
> rr$score(msr("classif.acc"), predict_sets = "train")
        task task_id               learner    learner_id          resampling
1: <TaskClassif>    iris <LearnerClassifRpart> classif.rpart <ResamplingHoldout>
resampling_id iteration classif.acc
1:       holdout         1        0.96
> rr$score(msr("classif.acc"), predict_sets = "test")
        task task_id               learner    learner_id          resampling
1: <TaskClassif>    iris <LearnerClassifRpart> classif.rpart <ResamplingHoldout>
resampling_id iteration          prediction classif.acc
1:       holdout         1 <PredictionClassif>        0.96

> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19041)

Matrix products: default

locale:
[1] LC_COLLATE=Chinese (Simplified)_China.936 
[2] LC_CTYPE=Chinese (Simplified)_China.936   
[3] LC_MONETARY=Chinese (Simplified)_China.936
[4] LC_NUMERIC=C                              
[5] LC_TIME=Chinese (Simplified)_China.936    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] mlr3_0.7.0

loaded via a namespace (and not attached):
[1] lgr_0.3.4          paradox_0.4.0      mlr3misc_0.5.0     codetools_0.2-16  
[5] listenv_0.8.0      future_1.17.0      digest_0.6.25      crayon_1.3.4      
[9] R6_2.4.1           mlr3measures_0.3.0 backports_1.1.6    future.apply_1.5.0
[13] uuid_0.1-4         data.table_1.12.8  rstudioapi_0.11    rpart_4.1-15      
[17] checkmate_2.0.0    tools_3.6.3        parallel_3.6.3     compiler_3.6.3    
[21] globals_0.12.5

标签： rmlr3

解决方案

这不是一个错误。您可以像这样访问它：

library(mlr3) 
task <- tsk("iris") 
learner <- lrn("classif.rpart",predict_sets=c("train","test")) 
rs <- rsmp("holdout") 
rr <- resample(task, learner, rs) 
measu <- list( msr("classif.acc", id = "acc_train", predict_sets = "train") 
               ,msr("classif.acc", id = "acc_test", predict_sets = "test") ) 
rr$aggregate(measu)

推荐阅读