cerebras.modelzoo.data_preparation.nlp.slimpajama.utils.cycle_documents

cerebras.modelzoo.data_preparation.nlp.slimpajama.utils.cycle_documents(dataset, process_id, n_process, dup_sh, short_sh)

previous

cerebras.modelzoo.data_preparation.nlp.slimpajama.utils

next

cerebras.modelzoo.data_preparation.nlp.slimpajama.utils.rm_if_exists