时间戳

首页 > 解决方案 > 训练期间的Pytorch MSE损失函数nan

python - 训练期间的Pytorch MSE损失函数nan

问题描述

我正在尝试从波士顿数据集进行线性回归。自第一次迭代以来，MSE 损失函数为 nan。我尝试改变学习率和 batch_size 但没有用。

from torch.utils.data import TensorDataset , DataLoader
inputs  = torch.from_numpy(Features).to(torch.float32)
targets = torch.from_numpy(target).to(torch.float32)
train_ds = TensorDataset(inputs , targets)
train_dl = DataLoader(train_ds , batch_size = 5 , shuffle = True)

model = nn.Linear(13,1)
opt = optim.SGD(model.parameters(), lr=1e-5)
loss_fn = F.mse_loss

def fit(num_epochs, model, loss_fn, opt, train_dl):
    
    # Repeat for given number of epochs
    for epoch in range(num_epochs):
        
        # Train with batches of data
        for xb,yb in train_dl:
            
            # 1. Generate predictions
            pred = model(xb)
            
            # 2. Calculate loss
            loss = loss_fn(pred, yb)
            
            # 3. Compute gradients
            loss.backward()
            
            # 4. Update parameters using gradients
            opt.step()
            
            
            # 5. Reset the gradients to zero
            opt.zero_grad()
        
        # Print the progress
        if (epoch+1) % 10 == 0:
            print('Epoch [{}/{}], Loss: {}'.format(epoch+1, num_epochs, loss.item()))


fit(100, model, loss_fn , opt , train_dl)

输出

标签： pythonpytorch

解决方案

注意：

  1. 使用标准化：x = (x - x.mean()) / x.std()
  2. y_train / y_test必须是 (-1, 1) 形状。使用y_train.view(-1, 1)（如果 y_train 是 torch.Tensor 什么的）
  3. （不是你的情况，而是其他人）如果你使用torch.nn.MSELoss(reduction='sum')than 你必须减少总和的意思。可以使用 torch.nn.MSELoss() 或在 train-loop: 中完成l = loss(y_pred, y) / y.shape[0]。

例子：

    ...
    loss = torch.nn.MSELoss()  
    ...
    for epoch in range(num_epochs):  
        for x, y in train_iter:  
            y_pred = model(x)  
            l = loss(y_pred, y)  
            optimizer.zero_grad()  
            l.backward()  
            optimizer.step()  
        print("epoch {} loss: {:.4f}".format(epoch + 1, l.item()))

推荐阅读