cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultimodalSimpleHDF5Dataset#
- class cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultimodalSimpleHDF5Dataset(*args, **kwargs)[source]#
Bases:
cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5Dataset
Dataset class to handle image preprocessing in multimodal datasets.
This class is largely the same as the parent class MultimodalHDF5Dataset except with added support for multiple images and intermingling of text and images.
- Parameters
params (dict) – A dictionary containing parameters that HDF5Dataset accepts along with the following add-ons: - “img_data_dir” (str): the path to the directory containing the images. - “image_data_size” (list[int]): the final C x H x W shape of the image. - “transforms” (list[dict]): a specification of the torchvision transforms.
Methods
generate_sample
load_state_dict
map
preprocess_img
state_dict
Attributes
by_sample