cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5Dataset#

class cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5Dataset(*args, **kwargs)[source]#

Bases: cerebras.modelzoo.data.common.h5_map_dataset.dataset.HDF5Dataset

Dataset class to handle image preprocessing in multimodal datasets.

This class is largely the same as the parent class HDF5Dataset except with added image loading and preprocessing.

Parameters

params (dict) – A dictionary containing parameters that HDF5Dataset accepts along with the following add-ons: - “img_data_dir” (str): the path to the directory containing the images. - “image_data_size” (list[int]): the final C x H x W shape of the image. - “transforms” (list[dict]): a specification of the torchvision transforms.

Methods

generate_sample

load_state_dict

map

preprocess_img

state_dict

Attributes

by_sample