cerebras.modelzoo.data.common.input_utils.cluster_config#
- cerebras.modelzoo.data.common.input_utils.cluster_config()[source]#
- Returns (ClusterSpec, TaskSpec). The TaskSpec contains the following fields:
rank: the global rank of the current worker
- local_rank: the rank of the current worker among workers who feed
the same system as the current worker
- wse_id: the index of the system that the current worker is
associated with
- The ClusterSpec contains the following fields:
tasks: a list of TaskSpecs for each task running on the cluster
rank: the rank of the current process’s task in the cluster
num_csx: the number of CSX systems in the cluster
num_workers_per_csx: the number of worker tasks per CSX
If the current job is running on GPU instead of CS system, then the ranks and world sizes in the returned TaskSpec will be set to the GPU rank and world size.