cerebras.modelzoo.data.nlp.t5.T5DynamicDataProcessor#
Processor for PyTorch T5 training.
Classes
Reads text files containing the input text tokens, adds extra ids for language modeling task on the fly. |
Processor for PyTorch T5 training.
Classes
Reads text files containing the input text tokens, adds extra ids for language modeling task on the fly. |