python - PyTorch 中的损失函数，并非我所有的训练示例都具有相同的权重

Question

我想在 PyTorch 中训练神经网络。我有我的训练数据集，但是我更关心一些示例而不是其他示例。我想将这些信息包含在损失函数中——让 NN 知道正确获取一些示例非常重要，并且不要过多地惩罚其他示例上的错误。我想通过加权训练示例的损失来做到这一点，比如说： loss = weight_for_example*(y_true - y_pred)^2 在 PyTorch 中是否有一种简单的方法可以做到这一点？

标签： pythonpytorchloss

Answer 1

It mainly depends on your task: for instance, BCEWithLogitsLoss has a weight parameter that allows a custom weight for each batch. Many other built-in losses also provide this option.

Aside from solutions already available in the framework such as this, a simple approach could be the following:

• build a custom dataset, returning your data and a scalar weight for that sample in your __getitem__
• proceed with the forward pass
• compute your loss, which you can now multiply by the factors you provided.

There's only a caveat (which is the same of the BCELoss): you probably iterate on batches with size > 1, so your dataloader will provide a batch of data, with a batch of weights. You need to make sure you don't reduce your loss beforehand, so that you can still multiply it by your batch weight, then you can proceed with a manual reduction (e.g. loss = loss.mean()).

See some examples here.