machine-learning - Implementing Group Lasso on PyTorch weight matrices
问题描述
I am trying to implement Group Lasso on weight matrices of a neural network in PyTorch.
I have written the code to implement Group Lasso but am unsure if this is correct, confirmation or correction of my code will be very helpful.
def gl_norm(model, gl_lambda, num_blk):
gl_reg = torch.tensor(0., dtype=torch.float32).cuda()
for key in model:
for param in model[key].parameters():
dim = param.size()
if dim.__len__() > 1 and not model[key].skip_regularization:
div1 = list(torch.chunk(param,int(num_blk),1))
all_blks = []
for div2 in div1:
temp = list(torch.chunk(div2,int(num_blk),0))
for blk in temp:
all_blks.append(blk)
for l2_param in all_blks:
gl_reg += torch.norm(l2_param, 2)
return gl_reg * float(gl_lambda)
I expect the torch.chunk function to break up the weight matrix into small blocks which then go through L2 norm for the block and L1 norm between all the blocks.
解决方案
推荐阅读
- android-studio - 在 3 次输入/更改文本后禁用 editText
- android - ConstraintLayout wrap_content 排除孩子
- json - 如何在 PostgreSQL 存储过程中重用 JSON 参数
- java - element.style 在一台服务器中禁用
- c# - 实体框架模型属性未在更新数据库命令上创建
- python - Django 列表视图反复失败,但详细信息页面有效
- .net - ASP.NET MVC 5 项目文件夹中缺少脚本文件夹
- javascript - 如何使用javascript中的提示输入在对象类中添加新对象
- c++ - 如何在 g++ 10 中使用范围?
- excel - 关于如何将硬编码值转换为变量的 VBA Excel 宏问题