cerebras.modelzoo.tools.checkpoint_converters.base_converter#

Functions

convert_dataloader_checkpoint

Converts DataLoader state files saved in release 1.9 to DataLoader checkpoint format for the new map and iterable DataLoaders in MZ in release 2.0.

get_artifact_dir

no_ckpt_conversion_necessary

Some converters copy the input checkpoint to the output (i.e.

parse_format_version

select_mup_converter

update_ckpt_metadata

Classes

BaseCheckpointConverter

Converts between checkpoint state_dict formats.

BaseCheckpointConverter_CS18_CS19

Generic fallback converter class for cs-1.8 -> cs-1.9.

BaseCheckpointConverter_CS19_CS20

Generic fallback converter class for cs-1.9 -> cs-2.0.

BaseCheckpointConverter_CS20_CS21

Generic fallback converter class for cs-2.0 -> cs-2.1.

BaseCheckpointConverter_CS21_CS22

Generic fallback converter class for cs-2.1 -> cs-2.2.

BaseCheckpointConverter_CS22_CS23

Generic fallback converter class for cs-2.2 -> cs-2.3.

BaseCheckpointConverter_CS_CS

BaseCheckpointConverter_HF_CS

HF checkpoints contain model only while CS checkpoints package model, optimizer, and lr_scheduler into a single checkpoint.

BaseCheckpointConverter_UnpackedHF_PackedCS

Converter between a set of unpacked HF checkpoints and a single packed CS checkpoint.

BaseConfigConverter

BaseConfigConverter_CS18_CS19

Generic fallback config converter class for cs-1.8 -> cs-1.9.

BaseConfigConverter_CS19_CS20

Generic fallback config converter class for cs-1.9 -> cs-2.0.

BaseConfigConverter_CS20_CS21

Generic fallback config converter class for cs-2.0 -> cs-2.1.

BaseConfigConverter_CS21_CS22

Generic fallback config converter class for cs-2.1 -> cs-2.2.

BaseConfigConverter_CS22_CS23

Generic fallback config converter class for cs-2.2 -> cs-2.3.

BaseConfigConverter_CS_CS

CS packages model, optimizer, and lr_scheduler into a single config.

BaseConfigConverter_HF_CS

CS packages model, optimizer, and lr_scheduler into a single config.

BaseConfigConverter_UnpackedHF_PackedCS

Converter between a set of unpacked HF configs and a single packed CS configs.

BaseDictionaryConverter

A dictionary converter represents a pair of two dictionary formats that can be converted between each other.

ConversionRule

ConversionRule defines a "rule" which:

EquivalentSubkey

EquivalentSubkey defines the bidirectional relationship between subkeys of a model's checkpoint.

FallbackConfigConverter_CS_CS

Generic fallback config converter class.

FallbackConverter_CS_CS

Generic fallback converter class.

FormatIndices

Class for propagating src and tgt version to base converter.

FormatVersions

Exceptions

ConfigConversionError

Raised when a config cannot be converted.