cerebras.modelzoo.data.nlp.bert.bert_utils.get_whole_word_span#

cerebras.modelzoo.data.nlp.bert.bert_utils.get_whole_word_span(tokens, start_index)[source]#

Returns the whole word start and end indices.

Param

list tokens: Tokens to process.

Param

int start_index: Start index.

Returns

tuple with start and end index of the word from the token list.