---

title: TimeSeriesLoader


keywords: fastai
sidebar: home_sidebar

summary: "Data Loader for Time Series data"
description: "Data Loader for Time Series data"
nb_path: "nbs/data__tsloader.ipynb"
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/data__tsloader.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
        
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Inherited-DataLoader-from-pytorch">Inherited <code>DataLoader</code> from <code>pytorch</code><a class="anchor-link" href="#Inherited-DataLoader-from-pytorch"> </a></h2>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="TimeSeriesLoader" class="doc_header"><code>class</code> <code>TimeSeriesLoader</code><a href="https://github.com/Nixtla/nixtlats/tree/master/nixtlats/data/tsloader.py#L18" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>TimeSeriesLoader</code>(<strong>*<code>args</code></strong>, <strong>**<code>kwds</code></strong>) :: <code>DataLoader</code></p>
</blockquote>
<p>Data loader. Combines a dataset and a sampler, and provides an iterable over
the given dataset.</p>
<p>The :class:<code>~torch.utils.data.DataLoader</code> supports both map-style and
iterable-style datasets with single- or multi-process loading, customizing
loading order and optional automatic batching (collation) and memory pinning.</p>
<p>See :py:mod:<code>torch.utils.data</code> documentation page for more details.</p>
<p>Args:
    dataset (Dataset): dataset from which to load the data.
    batch_size (int, optional): how many samples per batch to load
        (default: <code>1</code>).
    shuffle (bool, optional): set to <code>True</code> to have the data reshuffled
        at every epoch (default: <code>False</code>).
    sampler (Sampler or Iterable, optional): defines the strategy to draw
        samples from the dataset. Can be any <code>Iterable</code> with <code>__len__</code>
        implemented. If specified, :attr:<code>shuffle</code> must not be specified.
    batch_sampler (Sampler or Iterable, optional): like :attr:<code>sampler</code>, but
        returns a batch of indices at a time. Mutually exclusive with
        :attr:<code>batch_size</code>, :attr:<code>shuffle</code>, :attr:<code>sampler</code>,
        and :attr:<code>drop_last</code>.
    num_workers (int, optional): how many subprocesses to use for data
        loading. <code>0</code> means that the data will be loaded in the main process.
        (default: <code>0</code>)
    collate_fn (callable, optional): merges a list of samples to form a
        mini-batch of Tensor(s).  Used when using batched loading from a
        map-style dataset.
    pin_memory (bool, optional): If <code>True</code>, the data loader will copy Tensors
        into CUDA pinned memory before returning them.  If your data elements
        are a custom type, or your :attr:<code>collate_fn</code> returns a batch that is a custom type,
        see the example below.
    drop_last (bool, optional): set to <code>True</code> to drop the last incomplete batch,
        if the dataset size is not divisible by the batch size. If <code>False</code> and
        the size of dataset is not divisible by the batch size, then the last batch
        will be smaller. (default: <code>False</code>)
    timeout (numeric, optional): if positive, the timeout value for collecting a batch
        from workers. Should always be non-negative. (default: <code>0</code>)
    worker_init_fn (callable, optional): If not <code>None</code>, this will be called on each
        worker subprocess with the worker id (an int in <code>[0, num_workers - 1]</code>) as
        input, after seeding and before data loading. (default: <code>None</code>)
    prefetch_factor (int, optional, keyword-only arg): Number of samples loaded
        in advance by each worker. <code>2</code> means there will be a total of
        2 * num_workers samples prefetched across all workers. (default: <code>2</code>)
    persistent_workers (bool, optional): If <code>True</code>, the data loader will not shutdown
        the worker processes after a dataset has been consumed once. This allows to
        maintain the workers <code>Dataset</code> instances alive. (default: <code>False</code>)</p>
<p>.. warning:: If the <code>spawn</code> start method is used, :attr:<code>worker_init_fn</code>
             cannot be an unpicklable object, e.g., a lambda function. See
             :ref:<code>multiprocessing-best-practices</code> on more details related
             to multiprocessing in PyTorch.</p>
<p>.. warning:: <code>len(dataloader)</code> heuristic is based on the length of the sampler used.
             When :attr:<code>dataset</code> is an :class:<code>~torch.utils.data.IterableDataset</code>,
             it instead returns an estimate based on <code>len(dataset) / batch_size</code>, with proper
             rounding depending on :attr:<code>drop_last</code>, regardless of multi-process loading
             configurations. This represents the best guess PyTorch can make because PyTorch
             trusts user :attr:<code>dataset</code> code in correctly handling multi-process
             loading to avoid duplicate data.</p>

<pre><code>         However, if sharding results in multiple workers having incomplete last batches,
         this estimate can still be inaccurate, because (1) an otherwise complete batch can
         be broken into multiple ones and (2) more than one batch worth of samples can be
         dropped when :attr:`drop_last` is set. Unfortunately, PyTorch can not detect such
         cases in general.

         See `Dataset Types`_ for more details on these two types of datasets and how
         :class:`~torch.utils.data.IterableDataset` interacts with
         `Multi-process data loading`_.

</code></pre>
<p>.. warning:: See :ref:<code>reproducibility</code>, and :ref:<code>dataloader-workers-random-seed</code>, and
             :ref:<code>data-loading-randomness</code> notes for random seed related questions.</p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Tests-WindowsDataset-and-TimeSeriesDataset">Tests WindowsDataset and TimeSeriesDataset<a class="anchor-link" href="#Tests-WindowsDataset-and-TimeSeriesDataset"> </a></h3>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">nixtlats.data.utils</span> <span class="kn">import</span> <span class="n">create_synthetic_tsdata</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">test_eq_batch_size</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="p">,</span> <span class="n">loader_class</span><span class="p">):</span>
    <span class="c1"># Check returns batch_size tensors</span>
    <span class="n">loader</span> <span class="o">=</span> <span class="n">loader_class</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="n">batch_size</span><span class="p">,</span> <span class="n">eq_batch_size</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
    <span class="n">sizes</span> <span class="o">=</span> <span class="p">[</span><span class="n">batch</span><span class="p">[</span><span class="s1">&#39;Y&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="o">==</span> <span class="n">batch_size</span> <span class="k">for</span> <span class="n">batch</span> <span class="ow">in</span> <span class="n">loader</span><span class="p">]</span>
    
    <span class="k">assert</span> <span class="nb">all</span><span class="p">(</span><span class="n">sizes</span><span class="p">),</span> <span class="s1">&#39;Unexpected batch sizes.&#39;</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">test_eq_batch_size_order</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="p">,</span> <span class="n">loader_class</span><span class="p">):</span>
    <span class="c1">#This test only works for TimeSeriesDataset class</span>
    <span class="n">loader</span> <span class="o">=</span> <span class="n">loader_class</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="n">batch_size</span><span class="p">,</span> <span class="n">eq_batch_size</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
    <span class="k">for</span> <span class="n">batch</span> <span class="ow">in</span> <span class="n">loader</span><span class="p">:</span>
        <span class="n">idxs</span> <span class="o">=</span> <span class="n">batch</span><span class="p">[</span><span class="s1">&#39;idxs&#39;</span><span class="p">]</span>
        <span class="n">dataset_batch</span> <span class="o">=</span> <span class="n">dataset</span><span class="p">[</span><span class="n">idxs</span><span class="o">.</span><span class="n">numpy</span><span class="p">()</span><span class="o">.</span><span class="n">tolist</span><span class="p">()]</span>
        <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">batch</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
            <span class="k">assert</span> <span class="n">t</span><span class="o">.</span><span class="n">equal</span><span class="p">(</span><span class="n">batch</span><span class="p">[</span><span class="n">key</span><span class="p">],</span> <span class="n">dataset_batch</span><span class="p">[</span><span class="n">key</span><span class="p">]),</span> <span class="p">(</span>
                <span class="sa">f</span><span class="s1">&#39;Batch and dataset batch differ, key </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s1">&#39;</span>
            <span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h4 id="Complete-timeseries-dataset">Complete timeseries dataset<a class="anchor-link" href="#Complete-timeseries-dataset"> </a></h4>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">Y_df</span><span class="p">,</span> <span class="n">X_df</span><span class="p">,</span> <span class="n">S_df</span> <span class="o">=</span> <span class="n">create_synthetic_tsdata</span><span class="p">(</span><span class="n">sort</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">TimeSeriesDataset</span><span class="p">(</span><span class="n">S_df</span><span class="o">=</span><span class="n">S_df</span><span class="p">,</span> <span class="n">Y_df</span><span class="o">=</span><span class="n">Y_df</span><span class="p">,</span> <span class="n">X_df</span><span class="o">=</span><span class="n">X_df</span><span class="p">,</span>
                            <span class="n">input_size</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span>
                            <span class="n">output_size</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">dataloader</span> <span class="o">=</span> <span class="n">TimeSeriesLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">12</span><span class="p">,</span> 
                              <span class="n">eq_batch_size</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>

<span class="k">for</span> <span class="n">batch</span> <span class="ow">in</span> <span class="n">dataloader</span><span class="p">:</span>
    <span class="n">batch</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">test_eq_batch_size</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="n">TimeSeriesLoader</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">test_eq_batch_size_order</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="n">TimeSeriesLoader</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h4 id="Windowed-timeseries-dataset">Windowed timeseries dataset<a class="anchor-link" href="#Windowed-timeseries-dataset"> </a></h4>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">Y_df</span><span class="p">,</span> <span class="n">X_df</span><span class="p">,</span> <span class="n">S_df</span> <span class="o">=</span> <span class="n">create_synthetic_tsdata</span><span class="p">(</span><span class="n">sort</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">WindowsDataset</span><span class="p">(</span><span class="n">S_df</span><span class="o">=</span><span class="n">S_df</span><span class="p">,</span> <span class="n">Y_df</span><span class="o">=</span><span class="n">Y_df</span><span class="p">,</span> <span class="n">X_df</span><span class="o">=</span><span class="n">X_df</span><span class="p">,</span>
                         <span class="n">input_size</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span>
                         <span class="n">output_size</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span>
                         <span class="n">sample_freq</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">dataloader</span> <span class="o">=</span> <span class="n">TimeSeriesLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">12</span><span class="p">,</span> 
                              <span class="n">eq_batch_size</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>

<span class="k">for</span> <span class="n">batch</span> <span class="ow">in</span> <span class="n">dataloader</span><span class="p">:</span>
    <span class="n">batch</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">test_eq_batch_size</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="n">TimeSeriesLoader</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Faster-implemention">Faster implemention<a class="anchor-link" href="#Faster-implemention"> </a></h2>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="FastTimeSeriesLoader" class="doc_header"><code>class</code> <code>FastTimeSeriesLoader</code><a href="https://github.com/Nixtla/nixtlats/tree/master/nixtlats/data/tsloader.py#L99" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>FastTimeSeriesLoader</code>(<strong><code>dataset</code></strong>:<a href="/nixtlats/data__tsdataset.html#TimeSeriesDataset"><code>TimeSeriesDataset</code></a>, <strong><code>batch_size</code></strong>:<code>int</code>=<em><code>32</code></em>, <strong><code>eq_batch_size</code></strong>:<code>bool</code>=<em><code>False</code></em>, <strong><code>shuffle</code></strong>:<code>bool</code>=<em><code>False</code></em>)</p>
</blockquote>
<p>A DataLoader-like object for a set of tensors that can be much faster than
TensorDataset + DataLoader because dataloader grabs individual indices of
the dataset and calls cat (slow).
Source: <a href="https://discuss.pytorch.org/t/dataloader-much-slower-than-manual-batching/27014/6">https://discuss.pytorch.org/t/dataloader-much-slower-than-manual-batching/27014/6</a></p>
<h2 id="Notes">Notes<a class="anchor-link" href="#Notes"> </a></h2><p>[1] Adapted from <a href="https://github.com/hcarlens/pytorch-tabular/blob/master/fast_tensor_data_loader.py">https://github.com/hcarlens/pytorch-tabular/blob/master/fast_tensor_data_loader.py</a>.</p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h4 id="FastTimeSeriesLoader.__iter__" class="doc_header"><code>FastTimeSeriesLoader.__iter__</code><a href="https://github.com/Nixtla/nixtlats/tree/master/nixtlats/data/tsloader.py#L147" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>FastTimeSeriesLoader.__iter__</code>()</p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h4 id="FastTimeSeriesLoader.__next__" class="doc_header"><code>FastTimeSeriesLoader.__next__</code><a href="https://github.com/Nixtla/nixtlats/tree/master/nixtlats/data/tsloader.py#L164" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>FastTimeSeriesLoader.__next__</code>()</p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h4 id="FastTimeSeriesLoader.__len__" class="doc_header"><code>FastTimeSeriesLoader.__len__</code><a href="https://github.com/Nixtla/nixtlats/tree/master/nixtlats/data/tsloader.py#L180" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>FastTimeSeriesLoader.__len__</code>()</p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Tests-WindowsDataset-and-TimeSeriesDataset">Tests WindowsDataset and TimeSeriesDataset<a class="anchor-link" href="#Tests-WindowsDataset-and-TimeSeriesDataset"> </a></h3>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h4 id="Complete-timeseries-dataset">Complete timeseries dataset<a class="anchor-link" href="#Complete-timeseries-dataset"> </a></h4>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">Y_df</span><span class="p">,</span> <span class="n">X_df</span><span class="p">,</span> <span class="n">S_df</span> <span class="o">=</span> <span class="n">create_synthetic_tsdata</span><span class="p">(</span><span class="n">sort</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">TimeSeriesDataset</span><span class="p">(</span><span class="n">S_df</span><span class="o">=</span><span class="n">S_df</span><span class="p">,</span> <span class="n">Y_df</span><span class="o">=</span><span class="n">Y_df</span><span class="p">,</span> <span class="n">X_df</span><span class="o">=</span><span class="n">X_df</span><span class="p">,</span>
                            <span class="n">input_size</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span>
                            <span class="n">output_size</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">dataloader</span> <span class="o">=</span> <span class="n">FastTimeSeriesLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">12</span><span class="p">,</span> 
                                  <span class="n">eq_batch_size</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>

<span class="k">for</span> <span class="n">batch</span> <span class="ow">in</span> <span class="n">dataloader</span><span class="p">:</span>
    <span class="n">batch</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">test_eq_batch_size</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="n">FastTimeSeriesLoader</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">test_eq_batch_size_order</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="n">FastTimeSeriesLoader</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h4 id="Windowed-timeseries-dataset">Windowed timeseries dataset<a class="anchor-link" href="#Windowed-timeseries-dataset"> </a></h4>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">Y_df</span><span class="p">,</span> <span class="n">X_df</span><span class="p">,</span> <span class="n">S_df</span> <span class="o">=</span> <span class="n">create_synthetic_tsdata</span><span class="p">(</span><span class="n">sort</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">WindowsDataset</span><span class="p">(</span><span class="n">S_df</span><span class="o">=</span><span class="n">S_df</span><span class="p">,</span> <span class="n">Y_df</span><span class="o">=</span><span class="n">Y_df</span><span class="p">,</span> <span class="n">X_df</span><span class="o">=</span><span class="n">X_df</span><span class="p">,</span>
                         <span class="n">input_size</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span>
                         <span class="n">output_size</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span>
                         <span class="n">sample_freq</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">dataloader</span> <span class="o">=</span> <span class="n">FastTimeSeriesLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">12</span><span class="p">,</span> 
                                  <span class="n">eq_batch_size</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">for</span> <span class="n">batch</span> <span class="ow">in</span> <span class="n">dataloader</span><span class="p">:</span>
    <span class="n">batch</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">test_eq_batch_size</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="n">FastTimeSeriesLoader</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Performance-comparison">Performance comparison<a class="anchor-link" href="#Performance-comparison"> </a></h3>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">dataloader</span> <span class="o">=</span> <span class="n">TimeSeriesLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">12</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">fast_dataloader</span> <span class="o">=</span> <span class="n">FastTimeSeriesLoader</span><span class="p">(</span><span class="n">dataset</span><span class="o">=</span><span class="n">dataset</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">12</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">%</span><span class="k">timeit</span> -n 50 -r 3  [batch for batch in dataloader]
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">%</span><span class="k">timeit</span> -n 50 -r 3 [batch for batch in fast_dataloader]
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

</div>
 

