python - RuntimeError:梯度计算所需的变量之一已被就地操作修改?
问题描述
我pytorch-1.5
用来做一些gan
测试。我的代码是非常简单的 gan 代码,正好适合 sin(x) 函数:
import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt
# Hyper Parameters
BATCH_SIZE = 64
LR_G = 0.0001
LR_D = 0.0001
N_IDEAS = 5
ART_COMPONENTS = 15
PAINT_POINTS = np.vstack([np.linspace(-1, 1, ART_COMPONENTS) for _ in range(BATCH_SIZE)])
def artist_works(): # painting from the famous artist (real target)
r = 0.02 * np.random.randn(1, ART_COMPONENTS)
paintings = np.sin(PAINT_POINTS * np.pi) + r
paintings = torch.from_numpy(paintings).float()
return paintings
G = nn.Sequential( # Generator
nn.Linear(N_IDEAS, 128), # random ideas (could from normal distribution)
nn.ReLU(),
nn.Linear(128, ART_COMPONENTS), # making a painting from these random ideas
)
D = nn.Sequential( # Discriminator
nn.Linear(ART_COMPONENTS, 128), # receive art work either from the famous artist or a newbie like G
nn.ReLU(),
nn.Linear(128, 1),
nn.Sigmoid(), # tell the probability that the art work is made by artist
)
opt_D = torch.optim.Adam(D.parameters(), lr=LR_D)
opt_G = torch.optim.Adam(G.parameters(), lr=LR_G)
for step in range(10000):
artist_paintings = artist_works() # real painting from artist
G_ideas = torch.randn(BATCH_SIZE, N_IDEAS) # random ideas
G_paintings = G(G_ideas) # fake painting from G (random ideas)
prob_artist0 = D(artist_paintings) # D try to increase this prob
prob_artist1 = D(G_paintings) # D try to reduce this prob
D_loss = - torch.mean(torch.log(prob_artist0) + torch.log(1. - prob_artist1))
G_loss = torch.mean(torch.log(1. - prob_artist1))
opt_D.zero_grad()
D_loss.backward(retain_graph=True) # reusing computational graph
opt_D.step()
opt_G.zero_grad()
G_loss.backward()
opt_G.step()
但是当我运行它时出现了这个错误:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [128, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
我的代码有问题吗?
解决方案
发生这种情况是因为 opt_D.step() 就地修改了鉴别器的参数。但是这些参数是计算生成器的梯度所必需的。您可以通过将代码更改为:
for step in range(10000):
artist_paintings = artist_works() # real painting from artist
G_ideas = torch.randn(BATCH_SIZE, N_IDEAS) # random ideas
G_paintings = G(G_ideas) # fake painting from G (random ideas)
prob_artist1 = D(G_paintings) # G tries to fool D
G_loss = torch.mean(torch.log(1. - prob_artist1))
opt_G.zero_grad()
G_loss.backward()
opt_G.step()
prob_artist0 = D(artist_paintings) # D try to increase this prob
# detach here to make sure we don't backprop in G that was already changed.
prob_artist1 = D(G_paintings.detach()) # D try to reduce this prob
D_loss = - torch.mean(torch.log(prob_artist0) + torch.log(1. - prob_artist1))
opt_D.zero_grad()
D_loss.backward(retain_graph=True) # reusing computational graph
opt_D.step()
您可以在此处找到有关此问题的更多信息https://github.com/pytorch/pytorch/issues/39141
推荐阅读
- java - @FindBy Selenium 如何找到 ::before
- python - 控制回归中的变量(Python)
- pandas - 删除重复点
- ansible - Ansible - 如何在播放期间重新生成主机 ssh_host_keys 而不会丢失连接
- c#-3.0 - 在c#中获取屏幕上文本的x和y坐标
- javascript - 为什么我只能在 for 循环内部成功找到
- 或标签下
- 使用 JavaScript 中的 getElementsByClassName 方法?
- 或标签下
- mysql - Adonisjs 以编程方式更改数据库
- python - 为什么使用 isin 更改数据框?
- .net - 我无法理解 Linux 中的 dotnet 内存使用情况
- xml - 使用 XSL 将 XML 文档转换为 HTML 时出错