cerebras.modelzoo.common.optim_utils.configure_param_groups#
- cerebras.modelzoo.common.optim_utils.configure_param_groups(model, optimizer_params)[source]#
Groups the optimizer parameters into non-overlapping groups. The groups are formed along the two axis: i) if weight_decay is >0 or not for the param ii) unique adjust_learning_rate (learning_rate_scaling) for the param
- Parameters
model (dict) – Pytorch model
optimizer_params (dict) – optimizer paramters
- Returns
_description_
- Return type
_type_