cerebras.modelzoo.data_preparation.data_preprocessing.data_preprocessor#
This module implements a generic data preprocessor called DataPreprocessor. It internally uses DataFrame and DataReader to read and process data.
Functions
Ensures a directory exists, optionally handling a subdirectory. |
|
Format seconds into a human-readable string showing hours:minutes:seconds, minutes:seconds, or seconds. |
|
Returns available memory in bytes. |
|
Calculate and return the compression factor based on a file's extension. |
|
Update the progress bar based on the current progress. |
Classes
Initialize the class with given parameters. |