cerebras.modelzoo.data_preparation.raw_dataset_processor#

RawDatasetProcessor

This is Dataset process for processing Raw data set on the fly This contains methods for loading the dataset, tokenizing the dataset and all data transformations are handled as part of the collator function

config

Config classes of T5 data Configs

utils

previous

cerebras.modelzoo.data_preparation.nlp.write_csv_ner.InputFeatures

next

cerebras.modelzoo.data_preparation.raw_dataset_processor.RawDatasetProcessor