cerebras.modelzoo.data_preparation.nlp.t5.utils.flat_map#

cerebras.modelzoo.data_preparation.nlp.t5.utils.flat_map(fn, dataset)[source]#

Map a function over an iterator and flatten the result.

Parameters

  • fn (callable) – The function to apply to each element.

  • dataset (iterator) – A stream of elements to map fn over.

Yields

A flattened version of fn applied to each element of dataset.

previous

cerebras.modelzoo.data_preparation.nlp.t5.utils.create_transformer_input_features

next

cerebras.modelzoo.data_preparation.nlp.t5.utils.get_raw_sequence_lengths