cerebras.modelzoo.data_preparation.nlp.pubmed.TextSharding#

Script to shard into separate train and test dataset files

Reference: https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT

Classes

NLTKSegmenter

Sharding