cerebras.modelzoo.data.common.input_utils.shard_list_contiguous#
- cerebras.modelzoo.data.common.input_utils.shard_list_contiguous(input_list, worker_id, num_workers)[source]#
Shards a list by splitting it into num_workers contiguous segments. Only the `worker_id`th shard is returned. If the length of the list is not divisible by the number of workers, the last worker will be assigned all remainder elements.
- Parameters
input_list (list) – list to shard into contiguous segments
worker_id (int) – index of shard to return
num_workers (int) – number of shards to create
- Returns
A sublist of contiguous elements (worker_id’s shard)