machine-learning - 如何在成本函数中包含参数向量之间的相关性？

Question

我有以下模型：

class Model(nn.Module):
    def __init__(self, dim_in, lambda_=.3):
        super(FeatExtractorGR, self).__init__()

       '''The linear transform layer'''
        self.lt = nn.Linear(in_features = dim_in, out_features = 10, bias=True)
    
    '''The encoder'''
    self.feature_extractor = \
        nn.Sequential(
            #nn.Linear(in_features = 10, out_features = 30, bias=True),
            nn.BatchNorm1d(10),
            nn.ReLU(),
            nn.Linear(in_features = 10, out_features = 20, bias=True),
            nn.BatchNorm1d(20),
            nn.ReLU(),
            nn.Linear(in_features = 20, out_features = 10, bias=True),
            nn.BatchNorm1d(10),
            nn.ReLU()
        )

def forward(self, x):
    transformed = self.lt(x)
    return self.feature_extractor(transformed)

我想强制线性变换层的权重向量不相关。我试图在成本函数的向量中包含点积：

        params=list(model.lt.parameters())[0]
        dotprod=torch.tensordot(params, params, dims=([1],[1])).abs().fill_diagonal_(0).sum()/2

        loss = other_losses + dotprod * weight

但这不起作用，即使非常高weight。该lt层的权重向量仍然高度相关。我也试过删除other_losses，但没有效果。我究竟做错了什么？

标签： machine-learningpytorchregularized