cerebras.modelzoo.data.vision.utils.create_worker_cache#
- cerebras.modelzoo.data.vision.utils.create_worker_cache(src_dir, force_overwrite=False)[source]#
Checks for the dir in the worker_cache (SSD) on the worker node corresponding to the src_dir. If the directory exists and is same as the src_dir, it returns the dir path on worker_cache. Otherwise writes the directory to the worker_cache and returns the dir path. Writing to the cache can take a while, depending on the size of the src_dir: Displays a progress bar (in the worker logs) which shows progress of the cache Forces cache overwrite irrespective of a cache hit, when force_overwrite is True.