aiml.load_data package — Adversarial Insight ML documentation

# aiml.load_data package

## Submodules

## aiml.load_data.generate_parameter module

generate_parameter.py

This module provides a function for generating parameters based on input dataset.

aiml.load_data.generate_parameter.generate_parameter(input_shape, clip_values, nb_classes, dataset_test, dataloader_test)

  Generate the image shape, clip values, and number of classes based on the input dataset.

  Parameters:

    - input_shape (numpy array) – Image shape; if given (not None), it will remain unchanged.

    - clip_values (tuple) – Data range; if given (not None), it will remain unchanged.

    - nb_classes (int) – Number of classes; if given (not None), it will remain unchanged.

    - dataset_test – The input dataset for reference.

    - dataloader_test – DataLoader for the input dataset.

  Returns:

    A tuple containing (input_shape, clip_values, nb_classes).

  Return type:

    tuple

## aiml.load_data.load_model module

load_model.py

This script is responsible for loading the model.

aiml.load_data.load_model.load_model(model, device)

  Load a machine learning model.

  Parameters:

    - model (model or string) – If a string is provided, it will search for the target model by detectors.

    - device (string) – The device to use, either ‘cpu’ or ‘gpu’.

  Returns:

    The loaded machine learning model.

  Return type:

    model

## aiml.load_data.load_set module

load_set.py

This script is responsible for loading the test dataset for model evaluation.

aiml.load_data.load_set.load_test_set(test_set)

  Load a test dataset.

  Parameters:

    dataset (dataset or string) – Given a string, it will search for the target dataset.

  Returns:

    The loaded test dataset.

  Return type:

    dataset

aiml.load_data.load_set.load_train_set(train_set)

  Load a training dataset.

  Parameters:

    dataset (dataset or string) – Given a string, it will search for the target dataset.

  Returns:

    The loaded training dataset.

  Return type:

    dataset

## aiml.load_data.normalize_datasets module

normalize_datasets.py

This module contains functions for normalizing and denormalizing a dataset.

aiml.load_data.normalize_datasets.check_normalize(dataloader)

  Check if the data in a dataloader is normalized.

  Parameters:

    dataloader (torch.utils.data.DataLoader) – A PyTorch DataLoader containing the dataset.

  Returns:

    True if the data is normalized (mean close to 0, std close to 1), False otherwise.

  Return type:

    bool

aiml.load_data.normalize_datasets.denormalize(batch)

  Denormalize a batch of normalized data using mean and standard deviation values.

  Parameters:

    batch (torch.Tensor) – A batch of normalized data.

  Returns:

    The denormalized batch of data with the same shape as the input.

  Return type:

    torch.Tensor

aiml.load_data.normalize_datasets.get_mean_std(dataset)

  Get the mean and standard deviation of the dataset’s image channels.

  Parameters:

    dataset (dataset) – The dataset containing images.

  Returns:

    None.

aiml.load_data.normalize_datasets.get_transforms()

  Get a list of transformations for datasets, including normalization.

  Returns:

    A composition of transformations.

  Return type:

    torchvision.transforms.Compose

aiml.load_data.normalize_datasets.normalize_and_check_datasets(num_workers, batch_size_test, batch_size_train, dataset_test, dataset_train)

  Normalize and check the given test and optionally, training datasets for normalization.

  Parameters:

    - num_workers (int) – Number of workers for data loading.

    - batch_size_test (int) – Batch size for the test dataset.

    - batch_size_train (int) – Batch size for the training dataset (if provided).

    - test_dataset – The test dataset.

    - train_dataset (optional) – The training dataset (Default is None).

  Returns:

    If normalization is required, returns a tuple containing the normalized test and training datasets along with their data loaders. If no normalization is needed, returns the test dataset as-is.

  Return type:

    Tuple

aiml.load_data.normalize_datasets.normalize_datasets(dataset_test, dataset_train=None)

  Normalize the training and testing datasets.

  Parameters:

    - dataset_test (dataset) – The testing dataset.

    - dataset_train (dataset, optional) – The training dataset (Default is None).

  Returns:

    A tuple containing the normalized testing dataset and, if provided, the normalized training dataset.

  Return type:

    tuple

## Module contents