cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseCheckpointConverter_UnpackedHF_PackedCS#

class cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseCheckpointConverter_UnpackedHF_PackedCS[source]#

Bases: cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseCheckpointConverter_HF_CS

Converter between a set of unpacked HF checkpoints and a single packed CS checkpoint.

Some CS models consist of separate components which we want to initialize from existing HF checkpoints. For example, initializing the image encoder and text decoder of a multimodal model. This converter class provides an abstraction for using existing HF <-> CS checkpoint converters.

In particular, we specify a list of BaseCheckpointConverter_HF_CS classes through converters() corresponding to each model component. Similarly, we specify another list of directory names through component_names() corresponding to the name of the subdirectory containing the model checkpoint.

During conversion, this converter applies the i-th component converter to the component checkpoint found in the i-th subdirectory name.

Methods

architectures

attempt_mup_to_sp

component_names

convert

convert_all_keys

convert_helper

convert_key

Attempts to convert the old key by matching against the list of conversion rules.

converter_note

converters

extract_model_dict

file_formats

formats

get_config_converter_class

get_converter_indices

get_mup_converter

Allows models to override the default muP converters with their own

init_output_checkpoint

load

post_checkpoint_convert

post_model_convert

pre_checkpoint_convert

pre_model_convert

Hook executes right before model conversion.

replaceKey

Copies value that exists at old_state_dict's old_key to new_state_dict's new_key.

save

supports_conversion

supports_mup_conversion

convert_key(old_key, old_state_dict, new_state_dict, from_index, match_start=0, prefix='', action_fn_args=None, debug=False)#

Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occurred.

get_mup_converter()#

Allows models to override the default muP converters with their own

pre_model_convert(old_state_dict, new_state_dict, configs, converter_indices, drop_unmatched_keys)#

Hook executes right before model conversion.

static replaceKey(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args=None)#

Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.