python - 'bert-base-multilingual-uncased' dataloader RuntimeError：堆栈期望每个张量大小相等

问题描述

我是 nlp 的初学者，因为我正在参加这个比赛https://www.kaggle.com/c/contradictory-my-dear-watson我正在使用模型“bert-base-multilingual-uncased”并使用 BERT 标记器从同一个。我也在使用 kaggle tpu。这是我创建的自定义数据加载器。

class SherlockDataset(torch.utils.data.Dataset):

def __init__(self,premise,hypothesis,tokenizer,max_len,target = None):
    super(SherlockDataset,self).__init__()
    self.premise = premise
    self.hypothesis = hypothesis
    self.tokenizer = tokenizer
    self.max_len = max_len
    self.target = target

def __len__(self):
    return len(self.premise)

def __getitem__(self,item):
    sen1 = str(self.premise[item])
    sen2 = str(self.hypothesis[item])
    
    encode_dict = self.tokenizer.encode_plus(sen1,
                                        sen2,
                                        add_special_tokens = True,
                                        max_len = self.max_len,
                                        pad_to_max_len = True,
                                        return_attention_mask = True,
                                        return_tensors = 'pt'
                                       )
    input_ids = encode_dict["input_ids"][0]
    token_type_ids = encode_dict["token_type_ids"][0]
    att_mask = encode_dict["attention_mask"][0]
    
    if self.target is not None:
        sample = {
        "input_ids":input_ids,
        "token_type_ids":token_type_ids,
        "att_mask":att_mask,
        "targets": self.target[item]
        }
    else:
        sample = {
        "input_ids":input_ids,
        "token_type_ids":token_type_ids,
        "att_mask":att_mask
        }
    
    return sample

在数据加载器中加载数据期间

def train_fn(model,dataloader,optimizer,criterion,scheduler = None):
model.train()
print("train")
for idx, sample in enumerate(dataloader):
    '''
    input_ids = sample["input_ids"].to(config.DEVICE)
    token_type_ids = sample["token_type_ids"].to(config.DEVICE)
    att_mask = sample["att_mask"].to(config.DEVICE)
    targets = sample["targets"].to(config.DEVICE)
    '''
    print("train_out")
    input_ids = sample[0].to(config.DEVICE)
    token_type_ids = sample[1].to(config.DEVICE)
    att_mask = sample[2].to(config.DEVICE)
    targets = sample[3].to(config.DEVICE)
    
    optimizer.zero_grad()
    output = model(input_ids,token_type_ids,att_mask)
    output = np.argmax(output,axis = 1)
    loss = criterion(outputs,targets)
    accuracy = accuracy_score(output,targets)
    loss.backward()
    torch.nn.utils.clip_grad_norm_(model.parameters(),1.0)
    xm.optimizer_step(optimizer, barrier=True)
    if scheduler is not None:
        scheduler.step()
    if idx%50==0:
        print(f"idx : {idx}, TRAIN LOSS : {loss}")

我一次又一次地面临这个错误

RuntimeError: Caught RuntimeError in DataLoader worker process 0. Original Traceback (most recent 
call last): File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 
178, 
in _worker_loop data = fetcher.fetch(index) File "/opt/conda/lib/python3.7/site- 
packages/torch/utils/data/_utils/fetch.py", line 47, in fetch return self.collate_fn(data) File 
"/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 79, in 
 default_collate return [default_collate(samples) for samples in transposed] File 
"/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 79, in return 
 [default_collate(samples) for samples in transposed] File "/opt/conda/lib/python3.7/site- 
 packages/torch/utils/data/_utils/collate.py", line 55, in default_collate return torch.stack(batch, 
 0, out=out) RuntimeError: stack expects each tensor to be equal size, but got [47] at entry 0 and 
 [36] at entry 1

我试过改变 num_workers 值，改变批量大小。我检查了数据，其中没有任何文本为空、0 或任何意义上的损坏。我也尝试在 tokenizer 中更改 max_len，但我无法找到解决此问题的方法。请检查并让我知道如何解决它。

标签： pythonnlppytorchkaggledataloader

python - 'bert-base-multilingual-uncased' dataloader RuntimeError：堆栈期望每个张量大小相等

问题描述

解决方案

推荐阅读