cerebras.modelzoo.tools.checkpoint_converters.mup.Converter_T5_sP_muP#
- class cerebras.modelzoo.tools.checkpoint_converters.mup.Converter_T5_sP_muP[source]#
Bases:
cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseDictionaryConverter
Transforms a T5 CS muP checkpoint into a T5 CS sP checkpoint.
muP: Maximal Update Parametrization. sP: Standard Parametrization.
Methods
convert_all_keys
Attempts to convert the old key by matching against the list of conversion rules.
formats
get_converter_indices
is_mup
Copies value that exists at old_state_dict's old_key to new_state_dict's new_key.
scale_decoder_k_projection
scale_embeddings
scale_encoder_k_projection
scale_lm_head
scale_output_projection
scale_q_projection
scale_v_projection
supports_conversion
- convert_key(old_key, old_state_dict, new_state_dict, from_index, match_start=0, prefix='', action_fn_args=None, debug=False)#
Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occurred.
- static replaceKey(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args=None)#
Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.