cerebras.modelzoo.common.utils.model.attention_utils.StridedSparseAttentionBuilder#
- class cerebras.modelzoo.common.utils.model.attention_utils.StridedSparseAttentionBuilder(num_heads, max_sequence_length, local_attn_ctx=4, stride=2, attention_type='unidirectional')[source]#
Bases:
cerebras.modelzoo.common.utils.model.attention_utils.SparseAttentionBuilder
Methods
build_mask
get_pytorch_mask
set_global_attention
set_local_attention