cerebras.modelzoo.common.run_cstorch_flow.GradScalerParams#

class cerebras.modelzoo.common.run_cstorch_flow.GradScalerParams(loss_scaling_factor=1.0, initial_loss_scale=None, steps_per_increase=2000, min_loss_scale=None, max_loss_scale=None, max_gradient_norm=None, max_gradient_value=None)[source]#

Bases: object

Dataclass for parsing grad scaler params from optimizer params.

Methods

from_dict

Returns an instance of GradScalerParams from a dictionary.

Attributes

initial_loss_scale

loss_scaling_factor

max_gradient_norm

max_gradient_value

max_loss_scale

min_loss_scale

steps_per_increase

classmethod from_dict(params)[source]#

Returns an instance of GradScalerParams from a dictionary.

Note that matching keys are popped from the dictionary.