cerebras.modelzoo.data_preparation.data_preprocessing.hooks#
Functions
Transforms chat data for reading. |
|
Transforms data for the Direct Preference Optimization (DPO) task into a semantic data array format. |
|
Process finetuning image captions data into a semantic_data_array format. |
|
Transforms conversation data for finetuning LLaVA. |
|
Transforms conversation data for finetuning LLaVA. |
|
Transforms natural language generation (NLG) data for reading. |
|
Transforms image and caption data for pretraining. |
|
Process prompt and completion data from a chat into a semantic_data_array format. |
|
Process prompt and completion text data into a semantic_data_array format. |
|
Transforms text data for reading. |