cerebras.modelzoo.data_preparation.data_preprocessing.hooks#

Functions

chat_read_hook

Transforms chat data for reading.

dpo_read_hook

Transforms data for the Direct Preference Optimization (DPO) task into a semantic data array format.

finetuning_image_captions_hook

Process finetuning image captions data into a semantic_data_array format.

finetuning_llava_hook

Transforms conversation data for finetuning LLaVA.

finetuning_llava_hook_prompt_completion

Transforms conversation data for finetuning LLaVA.

nlg_read_hook

Transforms natural language generation (NLG) data for reading.

pretraining_image_captions_hook

Transforms image and caption data for pretraining.

prompt_completion_chat_read_hook

Process prompt and completion data from a chat into a semantic_data_array format.

prompt_completion_text_read_hook

Process prompt and completion text data into a semantic_data_array format.

text_read_hook

Transforms text data for reading.