cerebras.modelzoo.tools.checkpoint_converters.llava.ConfigConverter_LLaMaProjector_HF_CS22#
- class cerebras.modelzoo.tools.checkpoint_converters.llava.ConfigConverter_LLaMaProjector_HF_CS22[source]#
Bases:
cerebras.modelzoo.tools.checkpoint_converters.llama.ConfigConverter_LLaMa_HF_CS21
Methods
assert_factory_fn
assert_rotary_dim
Determines whether muP -> sP conversion is should be attempted.
convert
convert_all_keys
convert_gqa
Converts all keys in a config from converter_indices.direction format to the other format.
Attempts to convert the old key by matching against the list of conversion rules.
convert_mm_projector_type
convert_mm_vision_select_feature
convert_nonlinearity
convert_pi
convert_tune_mm_mlp_adapter
file_formats
formats
get_converter_indices
Allows models to override the default muP converters with their own
load
post_config_convert
pre_config_convert
Copies value that exists at old_state_dict's old_key to new_state_dict's new_key.
save
supports_conversion
supports_mup_conversion
- attempt_mup_to_sp()#
Determines whether muP -> sP conversion is should be attempted. Some HF models (such as BTLM) should not attempt muP -> sP conversion since they can natively handle muP.
- convert_helper(config, converter_indices, drop_unmatched_keys=False, no_progress_bar=True, debug=False)#
Converts all keys in a config from converter_indices.direction format to the other format. Conversion will fail if at least one of the keys did not match on any conversion rules and drop_unmatched_keys is not enabled. Returns the newly converted config.
- convert_key(old_key, old_state_dict, new_state_dict, from_index, match_start=0, prefix='', action_fn_args=None, debug=False)#
Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occurred.
- get_mup_converter()#
Allows models to override the default muP converters with their own
- static replaceKey(old_key, new_key, old_state_dict, new_state_dict, from_index, action_fn_args=None)#
Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.