cerebras.modelzoo.common.utils.model.mup_utils.scale_initializers_by_dimension

cerebras.modelzoo.common.utils.model.mup_utils.scale_initializers_by_dimension(initializers, width_scale=None, depth_scale=None)

Scales the std of an initializer or list of initializers by the specified width and depth scalars. Unsupported initializers are ignored and a warning is printed to the user.

previous

cerebras.modelzoo.common.utils.model.mup_utils.process_lr_adjustment_params

next

cerebras.modelzoo.common.utils.model.mup_utils.LRAdjustmentGroup