Trainers — Pytorch-Sconce master documentation

# Trainers

## Basic Trainer

class `sconce.trainer.` `Trainer`(*, model, validation_feed=None, training_feed=None, test_data_generator=None, training_data_generator=None, monitor=None)

  A Class that is used to train sconce models.

  Keyword Arguments:
  | model (`Model`) – the sconce model to be trained. See `sconce.models` for examples. training_data_generator (`DataGenerator`) – DEPRECATED, use training_feed argument instead. training_feed (`DataFeed`) – used during training to provide inputs and targets. test_data_generator (`DataGenerator`) – DEPRECATED, use validation_feed argument instead. validation_feed (`DataFeed`) – targets. These are never used for back-propagation. monitor (`Monitor`, optional) – the sconce monitor that records data during training. This data can be sent to external systems during training or kept until training completes allowing you to analyze training or make plots. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used.

  `checkpoint`(filename=None)

    Save model state and retain filename for a later call to `restore()`.

    Parameters: | filename (path, optional) – the filename to save the model state to.

  `load_model_state`(filename)

    Restore model state frome a file.

    Parameters: | filename (path) – the filename to where the model’s state was saved.

  `multi_train`(*, num_cycles, cycle_length=1, cycle_multiplier=2.0, **kwargs)

    Runs multiple training sessions one after another.

    Parameters: | num_cycles (int) – [1, inf) the number of cycles to train for. cycle_length (float) – (0.0, inf) the length (in epochs) of the first cycle. cycle_multiplier (float) – (0.0, inf) a factor used to determine the length of a cycle. The length of a cycle is equal to the length of the previous cycle (or `cycle_length` if it is the first cycle) multiplied by `cycle_multiplier`.
    Keyword Arguments:
    | **kwargs – are passed to the underlying `train()` method.

  `num_trainable_parameters`

    The number of trainable parameters that the models has.

  `restore`()

    Restore model to previously checkpointed state. See also `checkpoint()`.

  `save_model_state`(filename=None)

    Save model state to a file.

    Parameters: | filename (path, optional) – the filename to save the model state to. If `None`, a system dependent temporary location will be chosen.
    Returns: | the passed in filename, or the temporary filename chosen if `None` was passed in.
    Return type: | filename (path)

  `survey_learning_rate`(*, num_epochs=1.0, min_learning_rate=1e-12, max_learning_rate=10, monitor=None, batch_multiplier=1, stop_factor=10)

    Checkpoints a model, then runs a learning rate survey, before restoring the model back.

    Keyword Arguments:
    | num_epochs (float, optional) – (0.0, inf) the number of epochs to train the model for. min_learning_rate (float, optional) – (0.0, inf) the minimum learning rate used in the survey. max_learning_rate (float, optional) – (0.0, inf) the maximum learning rate used in the survey. monitor (`Monitor`, optional) – the sconce monitor that records data during the learning rate survey. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used. batch_multiplier (int, optional) – [1, inf) determines how often parameter updates will occur during training. If greater than 1, this simulates large batch sizes without increasing memory usage. For example, if the batch size were 100 and batch_multipler=10, the effective batch size would be 1,000, but the memory usage would be for a batch size of 100. stop_factor (float) – (1.0, inf) determines early stopping. If the training loss rises by more than this factor from it’s minimum value, the survey will stop.
    Returns: | the monitor used during this learning rate survey.
    Return type: | monitor (`Monitor`)

  `test`(*, monitor=None)

    Run all samples of self.validation_feed through the model in test (inference) mode.

    Parameters: | monitor (`Monitor`, optional) – the sconce monitor that records data during this testing. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used.
    Returns: | the monitor used during this testing.
    Return type: | monitor (`Monitor`)

    Note

    This method has been deprecated since 1.2.0. Please use the `validate()` method instead.

  `train`(*, num_epochs, monitor=None, test_to_train_ratio=None, validation_to_train_ratio=None, batch_multiplier=1)

    Train the model for a given number of epochs.

    Parameters: | num_epochs (float) – the number of epochs to train the model for. monitor (`Monitor`, optional) – a monitor to use for this training session. If `None`, then self.monitor will be used. test_to_train_ratio (float, optional) – [0.0, 1.0] determines how often (relative to training samples) that test samples are run through the model during training. If `None`, then the relative size of the training and test datasets is used. For example, for MNIST with 60,000 training samples and 10,000 test samples, the value would be 1/6th. batch_multiplier (int, optional) – [1, inf) determines how often parameter updates will occur during training. If greater than 1, this simulates large batch sizes without increasing memory usage. For example, if the batch size were 100 and batch_multipler=10, the effective batch size would be 1,000, but the memory usage would be for a batch size of 100.
    Returns: | the monitor used during training.
    Return type: | monitor (`Monitor`)

  `validate`(*, monitor=None)

    Run all samples of self.validation_feed through the model in test (inference) mode.

    Parameters: | monitor (`Monitor`, optional) – the sconce monitor that records data during this testing. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used.
    Returns: | the monitor used during this testing.
    Return type: | monitor (`Monitor`)

## AutoencoderTrainer

class `sconce.trainers.` `AutoencoderTrainer`(*, model, validation_feed=None, training_feed=None, test_data_generator=None, training_data_generator=None, monitor=None)

  Bases: `sconce.trainer.Trainer`, `sconce.trainers.autoencoder_trainer.AutoencoderMixin`

  `checkpoint`(filename=None)

    Save model state and retain filename for a later call to `restore()`.

    Parameters: | filename (path, optional) – the filename to save the model state to.

  `load_model_state`(filename)

    Restore model state frome a file.

    Parameters: | filename (path) – the filename to where the model’s state was saved.

  `multi_train`(*, num_cycles, cycle_length=1, cycle_multiplier=2.0, **kwargs)

    Runs multiple training sessions one after another.

    Parameters: | num_cycles (int) – [1, inf) the number of cycles to train for. cycle_length (float) – (0.0, inf) the length (in epochs) of the first cycle. cycle_multiplier (float) – (0.0, inf) a factor used to determine the length of a cycle. The length of a cycle is equal to the length of the previous cycle (or `cycle_length` if it is the first cycle) multiplied by `cycle_multiplier`.
    Keyword Arguments:
    | **kwargs – are passed to the underlying `train()` method.

  `num_trainable_parameters`

    The number of trainable parameters that the models has.

  `restore`()

    Restore model to previously checkpointed state. See also `checkpoint()`.

  `save_model_state`(filename=None)

    Save model state to a file.

    Parameters: | filename (path, optional) – the filename to save the model state to. If `None`, a system dependent temporary location will be chosen.
    Returns: | the passed in filename, or the temporary filename chosen if `None` was passed in.
    Return type: | filename (path)

  `survey_learning_rate`(*, num_epochs=1.0, min_learning_rate=1e-12, max_learning_rate=10, monitor=None, batch_multiplier=1, stop_factor=10)

    Checkpoints a model, then runs a learning rate survey, before restoring the model back.

    Keyword Arguments:
    | num_epochs (float, optional) – (0.0, inf) the number of epochs to train the model for. min_learning_rate (float, optional) – (0.0, inf) the minimum learning rate used in the survey. max_learning_rate (float, optional) – (0.0, inf) the maximum learning rate used in the survey. monitor (`Monitor`, optional) – the sconce monitor that records data during the learning rate survey. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used. batch_multiplier (int, optional) – [1, inf) determines how often parameter updates will occur during training. If greater than 1, this simulates large batch sizes without increasing memory usage. For example, if the batch size were 100 and batch_multipler=10, the effective batch size would be 1,000, but the memory usage would be for a batch size of 100. stop_factor (float) – (1.0, inf) determines early stopping. If the training loss rises by more than this factor from it’s minimum value, the survey will stop.
    Returns: | the monitor used during this learning rate survey.
    Return type: | monitor (`Monitor`)

  `test`(*, monitor=None)

    Run all samples of self.validation_feed through the model in test (inference) mode.

    Parameters: | monitor (`Monitor`, optional) – the sconce monitor that records data during this testing. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used.
    Returns: | the monitor used during this testing.
    Return type: | monitor (`Monitor`)

    Note

    This method has been deprecated since 1.2.0. Please use the `validate()` method instead.

  `train`(*, num_epochs, monitor=None, test_to_train_ratio=None, validation_to_train_ratio=None, batch_multiplier=1)

    Train the model for a given number of epochs.

    Parameters: | num_epochs (float) – the number of epochs to train the model for. monitor (`Monitor`, optional) – a monitor to use for this training session. If `None`, then self.monitor will be used. test_to_train_ratio (float, optional) – [0.0, 1.0] determines how often (relative to training samples) that test samples are run through the model during training. If `None`, then the relative size of the training and test datasets is used. For example, for MNIST with 60,000 training samples and 10,000 test samples, the value would be 1/6th. batch_multiplier (int, optional) – [1, inf) determines how often parameter updates will occur during training. If greater than 1, this simulates large batch sizes without increasing memory usage. For example, if the batch size were 100 and batch_multipler=10, the effective batch size would be 1,000, but the memory usage would be for a batch size of 100.
    Returns: | the monitor used during training.
    Return type: | monitor (`Monitor`)

  `validate`(*, monitor=None)

    Run all samples of self.validation_feed through the model in test (inference) mode.

    Parameters: | monitor (`Monitor`, optional) – the sconce monitor that records data during this testing. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used.
    Returns: | the monitor used during this testing.
    Return type: | monitor (`Monitor`)

## SingleClassImageClassifierTrainer

class `sconce.trainers.` `SingleClassImageClassifierTrainer`(*, model, validation_feed=None, training_feed=None, test_data_generator=None, training_data_generator=None, monitor=None)

  Bases: `sconce.trainer.Trainer`, `sconce.trainers.single_class_image_classifier_trainer.SingleClassImageClassifierMixin`

  A trainer with some methods that are handy when you’re training an image classifier model. Specifically a model that classifies images into a single class per image.

  New in 0.10.0 (Used to be called ClassifierTrainer)

  `checkpoint`(filename=None)

    Save model state and retain filename for a later call to `restore()`.

    Parameters: | filename (path, optional) – the filename to save the model state to.

  `load_model_state`(filename)

    Restore model state frome a file.

    Parameters: | filename (path) – the filename to where the model’s state was saved.

  `multi_train`(*, num_cycles, cycle_length=1, cycle_multiplier=2.0, **kwargs)

    Runs multiple training sessions one after another.

    Parameters: | num_cycles (int) – [1, inf) the number of cycles to train for. cycle_length (float) – (0.0, inf) the length (in epochs) of the first cycle. cycle_multiplier (float) – (0.0, inf) a factor used to determine the length of a cycle. The length of a cycle is equal to the length of the previous cycle (or `cycle_length` if it is the first cycle) multiplied by `cycle_multiplier`.
    Keyword Arguments:
    | **kwargs – are passed to the underlying `train()` method.

  `num_trainable_parameters`

    The number of trainable parameters that the models has.

  `plot_samples`(predicted_class, true_class=None, data_generator=None, feed=None, sort_by='rising predicted class score', num_samples=7, num_cols=7, figure_width=15, image_height=3, cache_results=True)

    Plot samples of the dataset where the given <predicted_class> was predicted by the model.

    Parameters: | predicted_class (int or string) – the class string or the index of the class that was predicted by the model. true_class (int or string) – the class string or the index of the class that the image actually belongs to. data_generator (`SingleClassImageDataGenerator`) – DEPRECATED, use feed argument instead. feed (`DataFeed`) – the DataFeed to use to find the samples. sort_by (string) – one of the sort_by strings, see note below. num_samples (int) – the number of sample images to plot. num_cols (int) – the number of columns to plot, one image per column. figure_width (float) – the size, in matplotlib-inches, for the width of the whole figure. image_height (float) – the size, in matplotlib-inches, for the height of a single image. cache_results (bool) – keep the results in memory to make subsequent plots faster. Beware, that on large datasets (like imagenet) this can cause your system to run out of memory.

    Note

    The sort_by strings supported are:
      - “rising predicted class score”: samples are plotted in order of the lowest predicted class score to the highest predicted class score.
      - “falling predicted class score”: samples are plotted in order of the higest predicted class score to the lowest predicted class score.
      - “rising true class score”: samples are plotted in order of the lowest true class score to the highest true class score.
      - “falling true class score”: samples are plotted in order of the higest true class score to the lowest true class score.
  `restore`()

    Restore model to previously checkpointed state. See also `checkpoint()`.

  `save_model_state`(filename=None)

    Save model state to a file.

    Parameters: | filename (path, optional) – the filename to save the model state to. If `None`, a system dependent temporary location will be chosen.
    Returns: | the passed in filename, or the temporary filename chosen if `None` was passed in.
    Return type: | filename (path)

  `survey_learning_rate`(*, num_epochs=1.0, min_learning_rate=1e-12, max_learning_rate=10, monitor=None, batch_multiplier=1, stop_factor=10)

    Checkpoints a model, then runs a learning rate survey, before restoring the model back.

    Keyword Arguments:
    | num_epochs (float, optional) – (0.0, inf) the number of epochs to train the model for. min_learning_rate (float, optional) – (0.0, inf) the minimum learning rate used in the survey. max_learning_rate (float, optional) – (0.0, inf) the maximum learning rate used in the survey. monitor (`Monitor`, optional) – the sconce monitor that records data during the learning rate survey. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used. batch_multiplier (int, optional) – [1, inf) determines how often parameter updates will occur during training. If greater than 1, this simulates large batch sizes without increasing memory usage. For example, if the batch size were 100 and batch_multipler=10, the effective batch size would be 1,000, but the memory usage would be for a batch size of 100. stop_factor (float) – (1.0, inf) determines early stopping. If the training loss rises by more than this factor from it’s minimum value, the survey will stop.
    Returns: | the monitor used during this learning rate survey.
    Return type: | monitor (`Monitor`)

  `test`(*, monitor=None)

    Run all samples of self.validation_feed through the model in test (inference) mode.

    Parameters: | monitor (`Monitor`, optional) – the sconce monitor that records data during this testing. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used.
    Returns: | the monitor used during this testing.
    Return type: | monitor (`Monitor`)

    Note

    This method has been deprecated since 1.2.0. Please use the `validate()` method instead.

  `train`(*, num_epochs, monitor=None, test_to_train_ratio=None, validation_to_train_ratio=None, batch_multiplier=1)

    Train the model for a given number of epochs.

    Parameters: | num_epochs (float) – the number of epochs to train the model for. monitor (`Monitor`, optional) – a monitor to use for this training session. If `None`, then self.monitor will be used. test_to_train_ratio (float, optional) – [0.0, 1.0] determines how often (relative to training samples) that test samples are run through the model during training. If `None`, then the relative size of the training and test datasets is used. For example, for MNIST with 60,000 training samples and 10,000 test samples, the value would be 1/6th. batch_multiplier (int, optional) – [1, inf) determines how often parameter updates will occur during training. If greater than 1, this simulates large batch sizes without increasing memory usage. For example, if the batch size were 100 and batch_multipler=10, the effective batch size would be 1,000, but the memory usage would be for a batch size of 100.
    Returns: | the monitor used during training.
    Return type: | monitor (`Monitor`)

  `validate`(*, monitor=None)

    Run all samples of self.validation_feed through the model in test (inference) mode.

    Parameters: | monitor (`Monitor`, optional) – the sconce monitor that records data during this testing. If `None`, a composite monitor consisting of a `StdoutMonitor` and a `DataframeMonitor` will be created for you and used.
    Returns: | the monitor used during this testing.
    Return type: | monitor (`Monitor`)

## ClassifierTrainer (deprecated)

class `sconce.trainers.` `ClassifierTrainer`(*args, **kwargs)

  Warning

  This class has been deprecated for `SingleClassImageClassifierTrainer` and will be removed soon. It will continue to work for now, but please update your code accordingly.