首页 > 解决方案 > R:如何将数据拆分为训练集和测试集,同时保留变量的比例和分布?

问题描述

可重现的例子:

library(caTools) #for sample.split function
set.seed(123)
#Creating example data frame
example_df <- data.frame(personID = > c(stringi::stri_rand_strings(1000, 5)),
                           sex = sample(1:2, 1000, replace=TRUE),
                           age = round(rnorm(1000, mean=50, sd=15), 0))

#Example of random splitting:
training_set <- example_df[sample.split(example_df$personID),]
test_set <- example_df[-c(training_set$personID),]

#evaluation of variables in test and training data sets:
  #Has to approximate 1 (in this case it's 1.2, which is too high)
  (sum(training_set$sex == 1) / sum(training_set$sex == 2)) / (sum(test_set$sex == 1) / sum(test_set$sex == 2)) 
  [1] 1.219139
  #Has to approximate 1 along the distribution (it's quite good, this is actually what i would expect)
  summary(training_set$age) / summary(test_set$age)
    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  0.7143  0.9756  1.0000  1.0032  1.0169  1.0000 

尽管 sample.split 函数划分age得当(分布匹配),但男性和女性的比例在变量上存在显着差异sex。使用什么函数将数据自动甚至拆分为多个(在本例中为两个)集合,同时保留变量的比例和分布?

标签: rtesting

解决方案


caret软件包将为您构建平衡的套装。检查涵盖基础知识的包装插图。例如:

inTrain <- createDataPartition(
  y = Sonar$Class,
  ## the outcome data are needed
  p = .75,
  ## The percentage of data in the
  ## training set
  list = FALSE
)

推荐阅读