cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.utils.collect_stats#

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.utils.collect_stats(data_arr, args)[source]#

Collect statistics of the dataset.

Parameters
  • data_arr (numpy.ndarray) – Numpy array containing the dataset.

  • args (ValidationArgs) – Arguments for verifying HDF5 dataset.