cerebras.modelzoo.tools.checkpoint_converters.mup.Converter_sP_muP_post_CS23#
- class cerebras.modelzoo.tools.checkpoint_converters.mup.Converter_sP_muP_post_CS23[source]#
Bases:
cerebras.modelzoo.tools.checkpoint_converters.mup.Converter_sP_muP_pre_CS23
Transforms a CS 2.3 and onwards muP checkpoint into a CS sP checkpoint.
muP: Maximal Update Parametrization. sP: Standard Parametrization.
Methods
convert_all_keys
Attempts to convert the old key by matching against the list of conversion rules.
formats
get_converter_indices
is_mup
Copies value that exists at old_state_dict's old_key to new_state_dict's new_key.
scale_embedding_layernorm
scale_embeddings
scale_k_projection
scale_lm_head
supports_conversion
- convert_key(old_key, old_state_dict, new_state_dict, from_index, match_start=0, prefix='', action_fn_args=None, debug=False)#
Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occurred.
- static replaceKey(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args=None)#
Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.