Schedules — Pytorch-Sconce master documentation

# Schedules

## Schedule

class `sconce.schedules.` `Schedule`

  The base class for all schedules in Sconce. It is only an interface, describing what must be implemented if you want to define a schedule.

  `get_value`(step, current_state)

    Returns the value one should set, based on this schedule.

    Parameters: | step (float) – (0.0, inf) the training step that is about to be completed. Fractional steps are possible (see batch_multiplier option on `sconce.trainer.Trainer.train()`). current_state (dict) – a dictionary describing the current training state.

  `set_num_steps`(value)
    This is the canonical method to set num_steps, taking care of setting any other state needed
      by the schedule as well.

## Cosine

class `sconce.schedules.` `Cosine`(initial_value, final_value)

  A Schedule where the hyperparameter follows a scaled and shifted cosine function from [0, pi]. It will begin at <initial_value> and end at <final_value>, after <num_steps>.

  Parameters: | initial_value (float) – the initial value of the hyperparameter. final_value (float) – the final value of the hyperparameter.

## Exponential

class `sconce.schedules.` `Exponential`(initial_value, final_value, stop_factor=None, loss_key='training_loss')

  A Schedule where the value, adjusts exponentially from <initial_value> to <final_value>, over <num_steps>.

  Parameters: | initial_value (float) – the initial value of the hyperparameter. final_value (float) – the final value of the hyperparameter. stop_factor (float, optional) – a StopTrainingError will be raised if <loss_key> rises to <stop_factor> above it’s observed minimum value so far. loss_key (string, optional) – the name of the quantity (in `current_state`) to observe for <stop_factor>.

## Linear

class `sconce.schedules.` `Linear`(initial_value, final_value)

  A Schedule where the value begins with value <initial_value>, then changes linearly to <final_value>.

  Parameters: | initial_value (float) – the initial value of the hyperparameter. final_value (float) – the final value of the hyperparameter.

## Step

class `sconce.schedules.` `Step`(initial_value, final_value, num_changes=1)

  A Schedule where the value starts at <initial_value> and changes <num_changes> times over the course of <num_steps> to a final value of <final_value>.

  Parameters: | initial_value (float) – the initial value of the hyperparameter. final_value (float) – the final value of the hyperparameter. num_changes (int, optional) – [1, <num_steps>-1] the number of times the hyperparameter’s value will change.

  Note

  The parameter <num_steps> is set during training based on the size of the batch_size and number of samples in the training_feed, and the batch_multiplier value.

## Triangle

class `sconce.schedules.` `Triangle`(initial_value, peak_value, peak_fraction=0.5)

  A Schedule where the value begins with value <initial_value>, then changes linearly to <peak_value>, and then back to <initial_value> by the end of <num_steps>. Peak value will occur after <peak_steps>.

  Parameters: | initial_value (float) – the initial value of the hyperparameter. peak_value (float) – the value of the hyperparameter at it’s peak (maximum or minimum). peak_fraction (float, optional) – (0.0, 1.0) used to determine the number of steps before the hyperparameter’s value will become <peak_value>.