cerebras.modelzoo.common.utils.model.mup_utils#

Functions

`is_mup`
`process_lr_adjustment_params`	Parses the model's supported lr adjustment groups and optionally overrides any user set scales :param model_lr_adjustment_groups: Keys are the :type model_lr_adjustment_groups: dict :param LR group name and the values are LRAdjustmentGroup instances: :param params_lr_adjustment_groups: Keys are the :type params_lr_adjustment_groups: dict :param LR group name and the values are the scale override value:
`scale_initializers_by_dimension`	Scales the std of an initializer or list of initializers by the specified width and depth scalars.

Classes

Stores data for a group of params that share a learning rate scalar.

cerebras.modelzoo.common.utils.model.lora.LoraConfig

cerebras.modelzoo.common.utils.model.mup_utils.is_mup