cerebras.modelzoo.tools.checkpoint_converters.dpr.Converter_DPR_BertWrapper#
- class cerebras.modelzoo.tools.checkpoint_converters.dpr.Converter_DPR_BertWrapper(encoder_params_key)[source]#
Bases:
cerebras.modelzoo.tools.checkpoint_converters.bert.Converter_BertModel_WithoutOptionalModel_HF_CS21
Methods
attempt_mup_to_sp
convert
convert_all_keys
Converts all keys in a checkpoint from converter_indices.direction format to the other format.
Attempts to convert the old key by matching against the list of conversion rules.
DPR checkpoints have pooler weights, but these are thrown away in the HF model code.
extract_model_dict
file_formats
formats
get_config_converter_class
get_converter_indices
Allows models to override the default muP converters with their own
init_output_checkpoint
load
position_embeddings_convert
post_checkpoint_convert
post_model_convert
pre_checkpoint_convert
pre_model_convert
Copies value that exists at old_state_dict's old_key to new_state_dict's new_key.
save
supports_conversion
supports_mup_conversion
- convert_pooler_factory_fn()[source]#
DPR checkpoints have pooler weights, but these are thrown away in the HF model code. Therefore we have to explicitly catch these weights but we return None to get rid of them.
- convert_helper(input_checkpoint, configs, converter_indices, output_checkpoint={}, drop_unmatched_keys=False, no_progress_bar=True, debug=False)#
Converts all keys in a checkpoint from converter_indices.direction format to the other format. Conversion will fail if at least one of the keys did not match on any conversion rules and drop_unmatched_keys is not enabled. Returns the newly converted checkpoint.
- convert_key(old_key, old_state_dict, new_state_dict, from_index, match_start=0, prefix='', action_fn_args=None, debug=False)#
Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occurred.
- get_mup_converter()#
Allows models to override the default muP converters with their own
- static replaceKey(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args=None)#
Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.