cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.utils.add_lm_args#

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.utils.add_lm_args(parser)[source]#

The language-modeling format is common enough (FIM is very similar) that we can re-use the arguments for it