r - caret deepnet 为所有预测产生相同的值

Question

我对深度学习很陌生。我使用包deepnet和caret. 对于这个回归问题caret，使用 sigmoid 函数作为激活函数，使用线性函数作为输出函数。preprocess = "range"我使用（我认为将预测变量标准化）预处理了预测变量。

library(caret)
library(deepnet)
set.seed(123, kind = "Mersenne-Twister", normal.kind = "Inversion")

# create data
dat <- as.data.frame(ChickWeight)
dat$vari <- sample(LETTERS, nrow(dat), replace = TRUE)
dat$Chick <- as.character(dat$Chick)

preds <- dat[1:100,2:5]
response <- dat[1:100,1]
vali <- dat[101:150,]

# change format of categorical predictors to one-hot encoded format
dmy <- dummyVars(" ~ .", data = preds)
preds_dummies <- data.frame(predict(dmy, newdata = preds))


# specifiy trainControl for tuning mtry and with specified folds
control <- caret::trainControl(search = "grid", method="repeatedcv", number=3, 
                               repeats=2, 
                               savePred = T)

# tune hyperparameters and build final model
tunegrid <- expand.grid(layer1 = c(5,50),
                        layer2 = c(0,5,50), 
                        layer3 = c(0,5,50),
                        hidden_dropout = c(0, 0.1), 
                        visible_dropout = c(0, 0.1))

model <- caret::train(x = preds_dummies,
                      y = response, 
                      method="dnn",
                      metric= "RMSE", 
                      tuneGrid=tunegrid, 
                      trControl= control,
                      preProcess = "range"
)

当我使用带有调优神经网络模型的验证集进行预测时，尽管有各种输入预测变量，但它只产生一个预测值。

# predict with validation set
# create dummies
dmy <- dummyVars(" ~ .", data = vali)
vali_dummies <- data.frame(predict(dmy, newdata = vali))
vali_dummies <- vali_dummies[,which(names(vali_dummies) %in% model$finalModel$xNames)]
# add empty columns for categorical preds of the one used in the model (to have the same matix)
not_included <- setdiff(model$finalModel$xNames, names(vali_dummies))
vali_add <- as.data.frame(matrix(rep(0, length(not_included)*nrow(vali_dummies)), 
                                 nrow = nrow(vali_dummies),
                                 ncol = length(not_included))
)
# change names
names(vali_add) <- not_included
# add to vali_dummies
vali_dummies <- cbind(vali_dummies, vali_add)
# put it in the same order as preds_dummies (sort the columns)
vali_dummies <- vali_dummies[names(preds_dummies)]

# normalize also the validation set
pp = preProcess(vali_dummies, method = c("range"))
vali_dummies <- predict(pp, vali_dummies)

# save obs and pred for predictions with the outer CV out-of-fold test set
temp <- data.frame(obs = vali[,1],
                   pred = caret::predict.train(object = model, newdata = vali_dummies))
temp

当我使用MASS包中不存在分类预测变量的波士顿数据集时，对于验证集的所有不同输入预测变量，我得到的预测值略有不同。我该如何解决这个问题并创建一个神经网络，在使用数字和分类预测变量时预测“不同”的预测？除了规范化，我还应该尝试什么？

标签： rdeep-learningneural-networkr-caret