pytorch - 遇到错误,梯度计算所需的变量之一已被就地操作修改
问题描述
我遇到了以下错误。这是一个不同的 rnn 结构。我已经实现在图卷积中使用。问题是隐藏的更新就地操作。但是,我必须在每次转发调用中更新它的值。我怎样才能做到这一点?提前致谢。
RuntimeError Traceback (most recent call last)
<ipython-input-110-b4425651d544> in <module>()
8 out = model(x[i])
9 loss = mael(out, x[i+1])
---> 10 loss.backward(retain_graph=True)
11 optimizer.step()
12 print(loss.item())
1 frames
/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
130 Variable._execution_engine.run_backward(
131 tensors, grad_tensors_, retain_graph, create_graph,
--> 132 allow_unreachable=True) # allow_unreachable flag
133
134
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1, 100]], which is output 0 of SelectBackward, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
这是一个不同的 rnn 结构。我已经实现在图卷积中使用。问题是隐藏的更新就地操作。但是,我必须在每次转发调用中更新它的值。我怎样才能做到这一点?提前致谢。
class RNN(nn.Module):
def __init__(self, input_dim, hidden_dim):
super(RNN,self).__init__()
self.input_dim = input_dim
self.hidden_dim = hidden_dim
self.weight = Parameter(torch.rand(10,input_dim,hidden_dim, requires_grad=True))
self.weight_h = Parameter(torch.rand(10,input_dim,hidden_dim, requires_grad=True))
self.bias = Parameter(torch.rand(10,input_dim,hidden_dim, requires_grad=True))
self.hidden = torch.rand(10,input_dim, hidden_dim)
self.weight_2 = Parameter(torch.rand(10,input_dim,hidden_dim,requires_grad=True))
self.weight_h_2 = Parameter(torch.rand(10,hidden_dim,hidden_dim, requires_grad=True))
self.bias_2 = Parameter(torch.rand(10,input_dim,hidden_dim, requires_grad=True))
self.tanh = Tanh()
self.iteration = 0
self.next_layer = False
self.hidden_init = torch.rand(1,1)
def set_hidden(self,x):
y = self.tanh(mm(x, self.weight[self.iteration]) + mm(self.hidden_init, self.weight_h[self.iteration]) + self.bias[self.iteration])
return y
def set_hidden_state_layer_2(self, x, hidden):
y = self.tanh(mm(x, self.weight_2[self.iteration]) + mm(hidden, self.weight_h_2[self.iteration]) + self.bias_2[self.iteration])
return y
def forward(self, x):
try:
dim_1, dim_2, dim_3 = x.shape
except:
x = torch.unsqueeze(x,0)
if self.iteration == 10:
self.next_layer = True
self.iteration = 0
if self.next_layer:
self.hidden[self.iteration] = self.set_hidden_state_layer_2(x, self.hidden[self.iteration].clone())
self.iteration = self.iteration + 1
return self.hidden[self.iteration - 1]
else:
hidden_init = torch.rand(1,1)
self.hidden[self.iteration] = self.tanh(mm(x, self.weight[self.iteration]) + mm(self.hidden_init, self.weight_h[self.iteration]) + self.bias[self.iteration])
self.iteration = self.iteration + 1
return self.hidden[self.iteration - 1]
model = RNN(1,100)
mael = nn.L1Loss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
x = torch.rand(11,1)
x_2 = torch.rand(11,1)
for i in range(10):
optimizer.zero_grad()
out = model(x[i])
loss = mael(out, x[i+1])
loss.backward(retain_graph=True)
optimizer.step()
print(loss.item())
解决方案
推荐阅读
- c++ - 如何在 WinForm GUI 中使用 Crypto++?
- javascript - 从 html 或 javascript 到 python Django 的通信
- x86 - 像 Denormals-Are-Zero (DAZ) 这样的非正规标志会影响相等比较吗?
- java - 如何使用最新的 Google Sign-In and Drive REST api 上传 SQLite 数据库文件?
- solr - SOLR,如何在 SOLR UI 中设置allowleadingwildcard?
- javascript - 在 Angular 5 中,如何从父组件访问动态添加的子组件?
- ios - UIDocumentBrowserViewController 的启动屏幕(深色外观)
- amazon-web-services - 如何将 AWS Cognito 访问权限限制为特定组的 ADFS 用户?
- r - 如何从 R 中的 FTP 服务器下载 .gz 文件?
- go - OpenGL 和 SDL2 无法打开窗口