r - 将每次迭代存储在梯度下降函数中,以便可视化参数更新过程,以及 r 中的成本覆盖过程
问题描述
我正在尝试在 r in 中编写一个批量梯度体面函数,以用于训练和测试数据集。到目前为止,我有以下代码。但是,当我运行它时,它只打印出最后一个参数和它运行的迭代。我想存储每次迭代、测试错误并能够可视化成本收敛过程。我不确定在哪里放置或如何将代码合并到下面的函数中。
GradD <- function(x, y, alpha = 0.006, epsilon = 10^-10){
iter <- 0
i <- 0
x <- cbind(rep(1, nrow(x)), x)
theta <- matrix(c(1,1), ncol(x), 1)
cost <- (1/(2*nrow(x)))* t(x%*% theta - y) %*% (x %*% theta - y)
delta <- 1
while (delta > epsilon){
i <- i + 1
theta <- theta - (alpha / nrow(x)) * (t(x) %*% (x %*% theta - y))
cval <- (1/(2*nrow(x))) * t(x %*% theta - y) %*% (x %*% theta - y)
cost <- append(cost, cval)
delta <- abs(cost[i+1] - cost[i])
if((cost[i+1] - cost[i]) > 0){
print("The cost is increasing. Try reducing alpha.")
return()
}
iter <- append(iter, i)
}
print(sprintf("Completed in %i iterations.", i))
return(theta)
}
TPredict <- function(theta, x){
x <- cbind(rep(1,nrow(x)), x)
return(x %*% theta)
}
编辑 我试图创建一个包含每次迭代的列表......但是现在我在运行代码时遇到错误
error.cost <- function(x, y, theta){
sum( (X %*% theta - y)^2 ) / (2*length(y))
}
num_iters <- 2000
cost_history <- double(num_iters)
theta_history <- list(num_iters)
GradD <- function(x, y, alpha = 0.006, epsilon = 10^-10){
iter <- 2000
i <- 0
x <- cbind(rep(1,nrow(x)), x)
theta <- matrix(c(1,1),ncol(x),1)
cost <- (1/(2*nrow(x))) * t(x %*% theta - y) %*% (x %*% theta - y)
delta <- 1
while(delta > epsilon){
i <- i + 1
theta <- theta - (alpha / nrow(x)) * (t(x) %*% (x %*% theta - y))
cval <- (1/(2*nrow(x))) * t(x %*% theta - y) %*% (x %*% theta - y)
cost <- append(cost, cval)
delta <- abs(cost[i+1] - cost[i])
cost_history[i] <- error.cost(x, y, theta)
theta_history[[i]] <- theta
if((cost[i+1] - cost[i]) > 0){
print("The cost is increasing. Try reducing alpha.")
return()
}
iter <- append(iter, i)
}
print(sprintf("Completed in %i iterations.", i))
return(theta)
}
我在 nrow(x) % % theta: non-conformable arguments 中遇到错误。如果我删除此函数中的 nrow() :
error.cost <- function(x, y, theta){
sum( (x %*% theta - y)^2 ) / (2*length(y))
}
然后它打印出结果,但它们是错误的最终结果,我根本没有存储迭代
解决方案
下面,主要使用您的代码,只是捕获历史的概念证明。alpha 值是通过四处寻找一个可以通过if cost increasing
创建超过 1 或 2 条记录的值来获得的,这似乎是最初的问题:
GradD2 <- function(x, y, alpha = 0.0000056, epsilon = 10^-10) {
cost <- vector(mode = 'numeric')
iter <- vector(mode = 'integer')
delta_hist <- vector(mode = 'numeric')
i <- 0
iter <- 0
x <- cbind(rep(1, nrow(x)), x)
theta <- matrix(c(1,1), ncol(x), 1)
cost <- (1/(2*nrow(x))) * t(x%*% theta -y) %*% (x %*% theta -y)
delta <- 1.0
while(length(iter) < 1000 ) { #todo - change back to while(delta>epsilon)
i <- i +1
theta <- theta - (alpha / nrow(x)) * (t(x) %*% (x %*% theta -y))
cval <- (1/(2*nrow(x))) * t(x %*% theta -y) %*% (x %*% theta -y)
cost <- append(cost, cval, after = length(cost))
delta <- abs(cost[i+1] - cost[i])
delta_hist <- append(delta_hist, delta, after = length(delta_hist))
if((cost[i+1] - cost[i]) > 0) {
print('The cost is increasing. Try reducing alpha.')
return(list(theta = theta,cost = cost, delta_hist = delta_hist, iter = iter))
}
iter <- append(iter, i, after = length(iter))
}
print(sprintf('Completed in %i iterations.', i))
return(list(theta = theta,cost = cost, delta_hist = delta_hist, iter = iter))
}
结果:
> str(GradD2_tst)
List of 4
$ theta : num [1:2, 1] 1.693 0.707
$ cost : num [1:1000] 88564 87541 86587 85698 84870 ...
$ delta_hist: num [1:999] 1024 954 889 828 771 ...
$ iter : num [1:1000] 0 1 2 3 4 5 6 7 8 9 ...
> GradD2_tst$delta_hist[994:999]
[1] 0.08580117 0.08580094 0.08580071 0.08580048 0.08580025 0.08580002
> GradD2_tst$cost[994:999]
[1] 73493.72 73493.63 73493.54 73493.46 73493.37 73493.29
>
如果不知道,我的 x 和 y 是:
> x <- as.matrix(sample(1:1000, 400), ncol =1)
> y <- sample(1:1000, 400)
这个对 x 的选择,当与正确的一起使用时,会while(delta > epsilon) {
为产生警告的 cval 提供不匹配,而现在不会。当。
用 epsilon 10^-1 再次钓鱼 alpha:
> GradD2_tst2 <- GradD2(x,y, alpha=0.006, epsilon = 10^-1)
[1] "The cost is increasing. Try reducing alpha."
> GradD2_tst2 <- GradD2(x,y, alpha=0.001, epsilon = 10^-1)
[1] "The cost is increasing. Try reducing alpha."
> GradD2_tst2 <- GradD2(x,y, alpha=0.0006, epsilon = 10^-1)
[1] "The cost is increasing. Try reducing alpha."
> GradD2_tst2 <- GradD2(x,y, alpha=0.00006, epsilon = 10^-1)
[1] "The cost is increasing. Try reducing alpha."
> GradD2_tst2 <- GradD2(x,y, alpha=0.000056, epsilon = 10^-1)
[1] "The cost is increasing. Try reducing alpha."
> GradD2_tst2 <- GradD2(x,y, alpha=0.000009, epsilon = 10^-1)
[1] "The cost is increasing. Try reducing alpha."
> GradD2_tst2 <- GradD2(x,y, alpha=0.0000056, epsilon = 10^-1)
[1] "Completed in 160 iterations."
>
> str(GradD2_tst2)
List of 4
$ theta : num [1:2, 1] 1.111 0.709
$ cost : num [1:161] 88564 87541 86587 85698 84870 ...
$ delta_hist: num [1:160] 1024 954 889 828 771 ...
$ iter : num [1:161] 0 1 2 3 4 5 6 7 8 9 ...
> GradD2_tst2$cost[155:161]
[1] 73566.06 73565.96 73565.85 73565.75 73565.65 73565.55 73565.45
> GradD2_tst2$delta_hist[155:161]
[1] 0.10493825 0.10364385 0.10243786 0.10131425 0.10026738 0.09929201 NA
>
在一个核心上以 epsilon = 10^-10 运行需要很长时间。我的从来没有在一夜之间完成,Ctl-C。关于历史的 HTH。
推荐阅读
- magento2 - Magento2 中锚定类别的产品计数
- reactjs - 如何在antd的嵌套表中放置一个加载器?
- python - 文件未保存在 Python 的正确目录中
- javascript - 如何在类构造中实现各种方法
- android - 如何一个接一个地垂直对齐芯片
- struct - DFS 在结构向量的邻接列表中查找连通性
- javascript - 如何获取单击的元素的最高父元素,该元素仍然是添加事件侦听器的元素的子元素?
- c++ - 如何从 2d QUEUE 获取值
- python - Python 计划未按计划运行
- php - Powershell 模块导入未保留在 docker 容器映像中