---

title: Callbacks

keywords: fastai
sidebar: home_sidebar

summary: "API details."
description: "API details."
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: notebooks/02_callbacks.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
        
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>/Users/hmamin/incendio/incendio/callbacks.py:26: UserWarning: Accio not available.
  warnings.warn(&#39;Accio not available.&#39;)
</pre>
</div>
</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">%</span><span class="k">load_ext</span> autoreload
<span class="o">%</span><span class="k">autoreload</span> 2
<span class="o">%</span><span class="k">matplotlib</span> inline
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>/Users/hmamin/anaconda3/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.
  and should_run_async(code)
/Users/hmamin/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:18: UserWarning: Accio not available.
</pre>
</div>
</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="TorchCallback" class="doc_header"><code>class</code> <code>TorchCallback</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L35" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>TorchCallback</code>()</p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="BasicConfig" class="doc_header"><code>class</code> <code>BasicConfig</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L72" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>BasicConfig</code>(<strong><code>order</code></strong>=<em><code>0</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>Handles basic model tasks like putting the model on the GPU
and switching between train and eval modes.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="StatsHandler" class="doc_header"><code>class</code> <code>StatsHandler</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L100" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>StatsHandler</code>(<strong><code>order</code></strong>=<em><code>5</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>This updates metrics at the end of each epoch to account for
potentially varying batch sizes.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="MetricPrinter" class="doc_header"><code>class</code> <code>MetricPrinter</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L124" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>MetricPrinter</code>(<strong><code>pbar_metric</code></strong>=<em><code>'loss'</code></em>, <strong><code>batch_freq</code></strong>=<em><code>1</code></em>, <strong><code>order</code></strong>=<em><code>10</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>Prints metrics at the end of each epoch. This is one of the
default callbacks provided in BaseModel - it does not need to
be passed in explicitly.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="BatchMetricPrinter" class="doc_header"><code>class</code> <code>BatchMetricPrinter</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L182" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>BatchMetricPrinter</code>(<strong><code>batch_freq</code></strong>, <strong><code>n_prints</code></strong>=<em><code>inf</code></em>, <strong><code>order</code></strong>=<em><code>10</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>Prints mini batch metrics to help us see if a model is
learning early in training (helpful for debugging). We
remove the callback after the specified number of prints
so that it isn't called unnecessarily throughout the whole
training process.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="EarlyStopper" class="doc_header"><code>class</code> <code>EarlyStopper</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L211" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>EarlyStopper</code>(<strong><code>metric</code></strong>, <strong><code>goal</code></strong>:<code>('max', 'min')</code>, <strong><code>min_improvement</code></strong>=<em><code>0.0</code></em>, <strong><code>patience</code></strong>=<em><code>3</code></em>, <strong><code>order</code></strong>=<em><code>15</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="PerformanceThreshold" class="doc_header"><code>class</code> <code>PerformanceThreshold</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L279" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>PerformanceThreshold</code>(<strong><code>metric</code></strong>, <strong><code>goal</code></strong>:<code>('min', 'max')</code>, <strong><code>threshold</code></strong>, <strong><code>skip_epochs</code></strong>=<em><code>0</code></em>, <strong><code>split</code></strong>:<code>('train', 'val')</code>=<em><code>'val'</code></em>, <strong><code>order</code></strong>=<em><code>15</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="ModelCheckpoint" class="doc_header"><code>class</code> <code>ModelCheckpoint</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L313" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>ModelCheckpoint</code>(<strong><code>metric</code></strong>=<em><code>'loss'</code></em>, <strong><code>goal</code></strong>:<code>('max', 'min')</code>=<em><code>'min'</code></em>, <strong><code>fname</code></strong>=<em><code>'trainer.pkl'</code></em>, <strong><code>metric_fname</code></strong>=<em><code>'best_val_metrics.json'</code></em>, <strong><code>order</code></strong>=<em><code>25</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="MetricHistory" class="doc_header"><code>class</code> <code>MetricHistory</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L362" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>MetricHistory</code>(<strong><code>fname</code></strong>=<em><code>'history.csv'</code></em>, <strong><code>plot_fname</code></strong>=<em><code>'history.png'</code></em>, <strong><code>order</code></strong>=<em><code>90</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>Separate from StatsHandler in case we don't want to log outputs.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="S3Uploader" class="doc_header"><code>class</code> <code>S3Uploader</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L421" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>S3Uploader</code>(<strong><code>bucket</code></strong>, <strong><code>prefix</code></strong>, <strong><code>order</code></strong>=<em><code>95</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>Upload model and logs to S3 when training finishes.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="BotoS3Uploader" class="doc_header"><code>class</code> <code>BotoS3Uploader</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L442" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>BotoS3Uploader</code>(<strong><code>bucket</code></strong>, <strong><code>s3_dir</code></strong>=<em><code>''</code></em>, <strong><code>retain_tree</code></strong>=<em><code>True</code></em>, <strong><code>recurse</code></strong>=<em><code>True</code></em>, <strong><code>keep_fn</code></strong>=<em><code>None</code></em>, <strong><code>order</code></strong>=<em><code>95</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>Upload model and logs to S3 when training finishes. This version of the
callback does not rely on any GoGuardian packages.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">



<div class="output_text output_subarea output_execute_result">
<pre>&#39;builtins&#39;</pre>
</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="CometCallback" class="doc_header"><code>class</code> <code>CometCallback</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L490" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>CometCallback</code>(<strong><code>project_name</code></strong>, <strong><code>exp_name</code></strong>=<em><code>None</code></em>, <strong><code>tags</code></strong>=<em><code>()</code></em>, <strong><code>order</code></strong>=<em><code>100</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>Logs metrics to comet ML. Loss is logged per batch, everything else is
logged per epoch.

Note: This could use some more work to add more options to store other
things, but I want to use what I have so far on my self-supervised
learning experiments.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="CometGradientCallback" class="doc_header"><code>class</code> <code>CometGradientCallback</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L543" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>CometGradientCallback</code>(<strong><code>project_name</code></strong>, <strong><code>exp_name</code></strong>=<em><code>None</code></em>, <strong><code>tags</code></strong>=<em><code>()</code></em>, <strong><code>order</code></strong>=<em><code>100</code></em>) :: <a href="/incendio/callbacks#CometCallback"><code>CometCallback</code></a></p>
</blockquote>

<pre><code>Contains standard Comet.ml tracking but adds some information about
gradients. At the moment, this creates a single bar plot at the end of
training displaying the average gradient magnitudes by layer. Eventually,
we might want to update this to show how magnitudes change by epoch. The
only issue here is for a model with many layers (e.g. hundreds), this
becomes tricky to visualize in a useful way with the added time dimension.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="EC2Closer" class="doc_header"><code>class</code> <code>EC2Closer</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L582" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>EC2Closer</code>(<strong><code>timeout</code></strong>=<em><code>5</code></em>, <strong><code>order</code></strong>=<em><code>100</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="ModelUnfreezer" class="doc_header"><code>class</code> <code>ModelUnfreezer</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L602" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>ModelUnfreezer</code>(<strong><code>i2n</code></strong>, <strong><code>unfreeze_type</code></strong>:<code>('groups', 'layers')</code>=<em><code>'groups'</code></em>, <strong><code>mode</code></strong>:<code>('batch', 'epoch')</code>=<em><code>'epoch'</code></em>, <strong><code>order</code></strong>=<em><code>25</code></em>) :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

<pre><code>Gradually unfreeze a model during training.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="SchedulerMixin" class="doc_header"><code>class</code> <code>SchedulerMixin</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L660" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>SchedulerMixin</code>() :: <a href="/incendio/callbacks#TorchCallback"><code>TorchCallback</code></a></p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="CosineLRScheduler" class="doc_header"><code>class</code> <code>CosineLRScheduler</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L696" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>CosineLRScheduler</code>(<strong><code>warm</code></strong>=<em><code>0.3</code></em>, <strong><code>restarts</code></strong>=<em><code>False</code></em>, <strong><code>cycle_len</code></strong>=<em><code>5</code></em>, <strong><code>cycle_decay</code></strong>=<em><code>0.0</code></em>, <strong><code>min_lr</code></strong>=<em><code>None</code></em>, <strong><code>verbose</code></strong>=<em><code>False</code></em>, <strong><code>order</code></strong>=<em><code>10</code></em>) :: <a href="/incendio/callbacks#SchedulerMixin"><code>SchedulerMixin</code></a></p>
</blockquote>

<pre><code>Learning rate scheduler that makes updates each batch.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="AdaptiveSawtoothScheduler" class="doc_header"><code>class</code> <code>AdaptiveSawtoothScheduler</code><a href="https://github.com/hdmamin/incendio/tree/master/incendio/callbacks.py#L823" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>AdaptiveSawtoothScheduler</code>(<strong><code>add</code></strong>=<em><code>0.0001</code></em>, <strong><code>scale</code></strong>=<em><code>0.6</code></em>, <strong><code>patience</code></strong>=<em><code>5</code></em>, <strong><code>order</code></strong>=<em><code>10</code></em>) :: <a href="/incendio/callbacks#SchedulerMixin"><code>SchedulerMixin</code></a></p>
</blockquote>

<pre><code>Learning rate scheduler inspired by the sawtooth pattern often
used to manage TCP flow
(ex: https://witestlab.poly.edu/blog/tcp-congestion-control-basics/).
This uses a strategy called "additive increase, multiplicative decrease".
Basically, while the training loss is generally decreasing, we
gradually increase the learning rate. When things show signs of getting
worse, we dramatically decrease the LR and begin slowly climbing again.
The result looks something like a cyclical policy with restarts,
except that in this case the cycle lengths are dependent on training
rather than pre-defined. SGD w/ restarts typically also uses a sharp
increase and a gradual decrease, while this is closer to the opposite.

Unlike the standard AIMD algorithm, we decay the amount added if the
batch loss increases, even if we're still within the patience window.</code></pre>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Here is a sample training run with an adaptive sawtooth scheduler.
{% include image.html file="/incendio/adaptive_sawtooth_lrs.png" %}</p>

</div>
</div>
</div>
</div>
 

