cerebras.modelzoo.common.run_cstorch_flow.GradScalerParams#
- class cerebras.modelzoo.common.run_cstorch_flow.GradScalerParams(loss_scaling_factor=1.0, initial_loss_scale=None, steps_per_increase=2000, min_loss_scale=None, max_loss_scale=None, max_gradient_norm=None, max_gradient_value=None)[source]#
Bases:
object
Dataclass for parsing grad scaler params from optimizer params.
Methods
Returns an instance of GradScalerParams from a dictionary.
Attributes
initial_loss_scale
loss_scaling_factor
max_gradient_norm
max_gradient_value
max_loss_scale
min_loss_scale
steps_per_increase