时间戳

首页 > 解决方案 > 我使用bertmodel最后一层的输出last_hidden_​​state，输入一维卷积nn.Conv1d()，但是报错

python - 我使用bertmodel最后一层的输出last_hidden_​​state，输入一维卷积nn.Conv1d()，但是报错

问题描述

我使用了bertmodel最后一层的输出last_hidden_​​state，形状为1024×1024，输入一维卷积nn.Conv1d()，但报错。

# Definition of Conv1d()
self.conv_xt_1 = nn.Conv1d(in_channels=1024, out_channels=n_filters, kernel_size=8)

# last_hidden_state input
output = self.bertmodel(input_ids, attention_mask)
last_hidden_state = output.last_hidden_state
last_hidden_state = last_hidden_state.squeeze()
conv_xt = self.conv_xt_1(last_hidden_state)
xt = conv_xt.view(-1, 32 * 1017)

我得到如下所示的错误：</p> 

Traceback (most recent call last):
  
File "training.py", line 81, in <module>
    
  train(model, device, train_loader, optimizer, epoch + 1)
  
File "training.py", line 21, in train
    
  output = model(data)
  
File "/home/yongnayuan/xqliu/anaconda3/envs/zzz/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    
  result = self.forward(*input, **kwargs)
  
File "/home/yongnayuan/wx/GCN-Bert/model/GCN_Bert.py", line 60, in forward
    
  conv_xt = self.conv_xt_1(last_hidden_state)
  
File "/home/yongnayuan/xqliu/anaconda3/envs/zzz/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    
  result = self.forward(*input, **kwargs)
  
File "/home/yongnayuan/xqliu/anaconda3/envs/zzz/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 263, in forward
    
  return self._conv_forward(input, self.weight, self.bias)
  
File "/home/yongnayuan/xqliu/anaconda3/envs/zzz/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 260, in _conv_forward
    
  self.padding, self.dilation, self.groups)

RuntimeError: Given groups=1, weight of size [32, 1024, 8], expected input[32, 3, 1024] to have 1024 channels, but got 3 channels instead

在此处输入图像描述

标签： pythonpytorchhuggingface-transformersbert-language-modelconv1d

解决方案

推荐阅读