cerebras.modelzoo.data.common.input_utils#
Functions
Checks if with the given sharding, at least one batch is generated. |
|
Returns (ClusterSpec, TaskSpec). The TaskSpec contains the following fields: |
|
Function to get distribute files with given number of examples such that each distributed task has access to exactly the same number of examples |
|
Returns True if DDP is enabled. |
|
Shards a list by splitting it into num_workers contiguous segments. Only the `worker_id`th shard is returned. If the length of the list is not divisible by the number of workers, the last worker will be assigned all remainder elements. |
|
Shards a list by assigning consecutive elements to alternating workers (i.e. |
|
Shards a list of chunks by distributing contiguous segments of each chunk across shards. |
|
Classes
Modified from: https://pytorch.org/docs/stable/_modules/torch/utils/data/distributed.html#DistributedSampler Sampler that restricts data loading to a subset of the dataset. |
|
Samples elements sequentially, starting from given start_index, |