cerebras.modelzoo.data_preparation.data_preprocessing.dpo_token_generator#

Classes

DPOTokenGenerator

Initialize the DPTokenGenerator class with dataset parameters, tokenizer, and token IDs.