--- title: N-BEATS: Neural Basis Expansion Analysis keywords: fastai sidebar: home_sidebar summary: "API details." description: "API details." nb_path: "nbs/models_nbeats__nbeats.ipynb" ---
{% raw %}
{% endraw %} {% raw %}

    # from google.colab import drive
    # drive.mount('/content/drive')
    # os.chdir('./drive/MyDrive/nixtlats')
    # print(os.getcwd())

    Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
    /content/drive/MyDrive/nixtlats

{% endraw %} {% raw %}

    # !pip install torchinfo
    # !pip install fastcore
    # !pip install s3fs
    # !pip install patool

{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

## `class` `IdentityBasis`[source]

> `IdentityBasis`(`backcast_size`:`int`, `forecast_size`:`int`) :: `Module`

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

    import torch.nn as nn
    import torch.nn.functional as F

    class Model(nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.conv1 = nn.Conv2d(1, 20, 5)
            self.conv2 = nn.Conv2d(20, 20, 5)

        def forward(self, x):
            x = F.relu(self.conv1(x))
            return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:`to`, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

## `class` `TrendBasis`[source]

> `TrendBasis`(`degree_of_polynomial`:`int`, `backcast_size`:`int`, `forecast_size`:`int`) :: `Module`

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

    import torch.nn as nn
    import torch.nn.functional as F

    class Model(nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.conv1 = nn.Conv2d(1, 20, 5)
            self.conv2 = nn.Conv2d(20, 20, 5)

        def forward(self, x):
            x = F.relu(self.conv1(x))
            return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:`to`, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

## `class` `SeasonalityBasis`[source]

> `SeasonalityBasis`(`harmonics`:`int`, `backcast_size`:`int`, `forecast_size`:`int`) :: `Module`

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

    import torch.nn as nn
    import torch.nn.functional as F

    class Model(nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.conv1 = nn.Conv2d(1, 20, 5)
            self.conv2 = nn.Conv2d(20, 20, 5)

        def forward(self, x):
            x = F.relu(self.conv1(x))
            return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:`to`, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}
{% endraw %} {% raw %}

## `class` `ExogenousBasisInterpretable`[source]

> `ExogenousBasisInterpretable`() :: `Module`

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

    import torch.nn as nn
    import torch.nn.functional as F

    class Model(nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.conv1 = nn.Conv2d(1, 20, 5)
            self.conv2 = nn.Conv2d(20, 20, 5)

        def forward(self, x):
            x = F.relu(self.conv1(x))
            return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:`to`, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

## `class` `ExogenousBasisWavenet`[source]

> `ExogenousBasisWavenet`(`out_features`, `in_features`, `num_levels`=`4`, `kernel_size`=`3`, `dropout_prob`=`0`) :: `Module`

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

    import torch.nn as nn
    import torch.nn.functional as F

    class Model(nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.conv1 = nn.Conv2d(1, 20, 5)
            self.conv2 = nn.Conv2d(20, 20, 5)

        def forward(self, x):
            x = F.relu(self.conv1(x))
            return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:`to`, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

## `class` `ExogenousBasisTCN`[source]

> `ExogenousBasisTCN`(`out_features`, `in_features`, `num_levels`=`4`, `kernel_size`=`2`, `dropout_prob`=`0`) :: `Module`

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

    import torch.nn as nn
    import torch.nn.functional as F

    class Model(nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.conv1 = nn.Conv2d(1, 20, 5)
            self.conv2 = nn.Conv2d(20, 20, 5)

        def forward(self, x):
            x = F.relu(self.conv1(x))
            return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:`to`, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}
{% endraw %} {% raw %}

#### `init_weights`[source]

> `init_weights`(`module`, `initialization`)

{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}

    from torchinfo import summary

    model = _NBEATS(n_time_in=168,
                    n_time_out=24,
                    n_s=1,
                    n_x=10,
                    n_s_hidden=100,
                    n_x_hidden=30,
                    n_polynomials=2,
                    n_harmonics=4,
                    stack_types=['trend', 'seasonality', 'exogenous_wavenet'],
                    n_blocks=[1, 1, 1],
                    n_layers=[2, 2, 2],
                    n_theta_hidden=3 * [[128, 128]],
                    dropout_prob_theta=0,
                    activation='SELU',
                    initialization='lecun_normal',
                    batch_normalization=True,
                    shared_weights=True)

    # inputs: S, Y, X, insample_mask
    # S.shape (n_batch,n_s)
    # Y.shape (n_batch,n_time_in+n_time_out)
    # X.shape (n_batch,n_x,n_time_in+n_time_out)
    # insample_mask.shape (n_batch,n_time_in+n_time_out)
    # summary(model, input_size=[(256, 1), (256, 168+24), (256, 10, 168+24), (256, 168+24)])

{% endraw %}

# N-BEATS model wrapper

{% raw %}
{% endraw %} {% raw %}

## `class` `NBEATS`[source]

> `NBEATS`(`n_time_in`, `n_time_out`, `n_x`, `n_x_hidden`, `n_s`, `n_s_hidden`, `shared_weights`, `activation`, `initialization`, `stack_types`, `n_blocks`, `n_layers`, `n_harmonics`, `n_polynomials`, `n_theta_hidden`, `batch_normalization`, `dropout_prob_theta`, `learning_rate`, `lr_decay`, `lr_decay_step_size`, `weight_decay`, `loss_train`, `loss_hypar`, `loss_valid`, `frequency`, `random_seed`, `seasonality`) :: `LightningModule`

Helper class that provides a standard way to create an ABC using inheritance.

{% endraw %} {% raw %}
{% endraw %}

# N-BEATS Usage Example

### Load Data

{% raw %}

    import multiprocessing

    import pandas as pd
    from nixtlats.data.datasets.epf import EPF#, EPFInfo
    from nixtlats.data.tsloader import TimeSeriesLoader

    import pylab as plt
    from pylab import rcParams
    plt.style.use('seaborn-whitegrid')
    plt.rcParams['font.family'] = 'serif'

    FONTSIZE = 19

    # Load and plot data
    Y_df, X_df, S_df = EPF.load_groups(directory='./data', groups=['NP'])

    fig = plt.figure(figsize=(15, 6))
    plt.plot(Y_df.ds, Y_df.y.values, color='#628793', linewidth=0.4)
    plt.ylabel('Price [EUR/MWh]', fontsize=19)
    plt.xlabel('Date', fontsize=15)
    # plt.savefig('./results/NP.png', bbox_inches = 'tight')
    plt.show()

{% endraw %}

### Declare Model and Data Parameters

{% raw %}

    mc = {}
    mc['model'] = 'nbeats'
    mc['mode'] = 'simple'
    mc['activation'] = 'SELU'

    mc['n_time_in'] = 24*7
    mc['n_time_out'] = 24*7
    mc['n_x_hidden'] = 8
    mc['n_s_hidden'] = 0

    # mc['input_size_multiplier'] = 7
    # mc['output_size'] = 24

    mc['stack_types'] = ['trend', 'seasonality', 'exogenous_wavenet']
    mc['n_blocks'] = [1, 1, 1]
    mc['n_layers'] = [2, 2, 2]
    # mc['stack_types'] = ['trend', 'seasonality']
    # mc['n_blocks'] = [1, 1]
    # mc['n_layers'] = [2, 2]

    mc['n_hidden'] = 128
    mc['shared_weights'] = False
    mc['n_harmonics'] = 4
    mc['n_polynomials'] = 2

    # Optimization and regularization parameters
    mc['initialization'] = 'lecun_normal'
    mc['learning_rate'] = 0.0007
    mc['batch_size'] = 128
    mc['lr_decay'] = 0.5
    mc['lr_decay_step_size'] = 2
    mc['max_epochs'] = 1#_000
    mc['max_steps'] = 20#_000
    mc['early_stop_patience'] = 20
    mc['eval_freq'] = 500
    mc['batch_normalization'] = False
    mc['dropout_prob_theta'] = 0.51
    mc['dropout_prob_exogenous'] = 0.44
    mc['l1_theta'] = 0
    mc['weight_decay'] = 0
    mc['loss_train'] = 'MAE'
    mc['loss_hypar'] = 0.5
    mc['loss_valid'] = mc['loss_train']
    mc['random_seed'] = 1

    # Data Parameters
    mc['len_sample_chunks'] = None
    mc['idx_to_sample_freq'] = 1
    mc['val_idx_to_sample_freq'] = 24 * 7
    mc['n_val_weeks'] = 52
    mc['window_sampling_limit'] = 500_000
    mc['normalizer_y'] = None
    mc['normalizer_x'] = 'median'
    mc['complete_inputs'] = False
    mc['frequency'] = 'H'
    mc['seasonality'] = 24

    # # Within decomposition
    mc['learning_rate'] = 0.0005
    mc['batch_size'] = 256
    mc['weight_decay'] = 0.00006
    mc['n_harmonics'] = 2
    mc['n_polynomials'] = 4
    mc['dropout_prob_theta'] = 0
    mc['dropout_prob_exogenous'] = 0

    print(65*'=')
    print(pd.Series(mc))
    print(65*'=')

    mc['n_theta_hidden'] = len(mc['stack_types']) * [ [int(mc['n_hidden']), int(mc['n_hidden'])] ]

    =================================================================
    model                                                      nbeats
    mode                                                       simple
    activation                                                   SELU
    n_time_in                                                     168
    n_time_out                                                    168
    n_x_hidden                                                      8
    n_s_hidden                                                      0
    stack_types               [trend, seasonality, exogenous_wavenet]
    n_blocks                                                [1, 1, 1]
    n_layers                                                [2, 2, 2]
    n_hidden                                                      128
    shared_weights                                              False
    n_harmonics                                                     2
    n_polynomials                                                   4
    initialization                                       lecun_normal
    learning_rate                                              0.0005
    batch_size                                                    256
    lr_decay                                                      0.5
    lr_decay_step_size                                              2
    max_epochs                                                      1
    max_steps                                                      20
    early_stop_patience                                            20
    eval_freq                                                     500
    batch_normalization                                         False
    dropout_prob_theta                                              0
    dropout_prob_exogenous                                          0
    l1_theta                                                        0
    weight_decay                                                6e-05
    loss_train                                                    MAE
    loss_hypar                                                    0.5
    loss_valid                                                    MAE
    random_seed                                                     1
    len_sample_chunks                                            None
    idx_to_sample_freq                                              1
    val_idx_to_sample_freq                                        168
    n_val_weeks                                                    52
    window_sampling_limit                                      500000
    normalizer_y                                                 None
    normalizer_x                                               median
    complete_inputs                                             False
    frequency                                                       H
    seasonality                                                    24
    dtype: object
    =================================================================

{% endraw %}

### Instantiate Loaders and Model

{% raw %}

    from nixtlats.experiments.utils import create_datasets

    train_dataset, val_dataset, test_dataset, scaler_y = create_datasets(mc=mc,
                                                                         S_df=S_df, Y_df=Y_df, X_df=X_df,
                                                                         f_cols=['Exogenous1', 'Exogenous2'],
                                                                         ds_in_val=294*24,
                                                                         ds_in_test=728*24,
                                                                         n_uids=None, n_val_windows=None,freq=None,
                                                                         is_val_random=False)

    train_loader = TimeSeriesLoader(dataset=train_dataset,
                                    batch_size=int(mc['batch_size']),
                                    #num_workers=int(min(multiprocessing.cpu_count(), 3)),
                                    shuffle=True)

    val_loader = TimeSeriesLoader(dataset=val_dataset,
                                  batch_size=int(mc['batch_size']),
                                  #num_workers=int(min(multiprocessing.cpu_count(), 3)),
                                  shuffle=False)

    test_loader = TimeSeriesLoader(dataset=test_dataset,
                                   batch_size=int(mc['batch_size']),
                                   #num_workers=int(min(multiprocessing.cpu_count(), 3)),
                                   shuffle=False)

    mc['n_x'], mc['n_s'] = train_dataset.get_n_variables()

    /usr/local/lib/python3.7/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm
    INFO:root:Train Validation splits

    INFO:root:                              ds
                                 min                 max
    unique_id sample_mask
    NP        0           2016-03-08 2018-12-24 23:00:00
              1           2013-01-01 2016-03-07 23:00:00
    INFO:root:
    Total data 			52416 time stamps
    Available percentage=100.0, 	52416 time stamps
    Insample  percentage=53.21, 	27888 time stamps
    Outsample percentage=46.79, 	24528 time stamps

    INFO:root:Train Validation splits

    INFO:root:                              ds
                                 min                 max
    unique_id sample_mask
    NP        0           2013-01-01 2018-12-24 23:00:00
              1           2016-03-08 2016-12-26 23:00:00
    INFO:root:
    Total data 			52416 time stamps
    Available percentage=100.0, 	52416 time stamps
    Insample  percentage=13.46, 	7056 time stamps
    Outsample percentage=86.54, 	45360 time stamps

    INFO:root:Train Validation splits

    INFO:root:                              ds
                                 min                 max
    unique_id sample_mask
    NP        0           2013-01-01 2016-12-26 23:00:00
              1           2016-12-27 2018-12-24 23:00:00
    INFO:root:
    Total data 			52416 time stamps
    Available percentage=100.0, 	52416 time stamps
    Insample  percentage=33.33, 	17472 time stamps
    Outsample percentage=66.67, 	34944 time stamps

{% endraw %} {% raw %}

    model = NBEATS(n_time_in=int(mc['n_time_in']),
                   n_time_out=int(mc['n_time_out']),
                   n_x=mc['n_x'],
                   n_s=mc['n_s'],
                   n_s_hidden=int(mc['n_s_hidden']),
                   n_x_hidden=int(mc['n_x_hidden']),
                   shared_weights=mc['shared_weights'],
                   initialization=mc['initialization'],
                   activation=mc['activation'],
                   stack_types=mc['stack_types'],
                   n_blocks=mc['n_blocks'],
                   n_layers=mc['n_layers'],
                   n_theta_hidden=mc['n_theta_hidden'],
                   n_harmonics=int(mc['n_harmonics']),
                   n_polynomials=int(mc['n_polynomials']),
                   batch_normalization = mc['batch_normalization'],
                   dropout_prob_theta=mc['dropout_prob_theta'],
                   learning_rate=float(mc['learning_rate']),
                   lr_decay=float(mc['lr_decay']),
                   lr_decay_step_size=float(mc['lr_decay_step_size']),
                   weight_decay=mc['weight_decay'],
                   loss_train=mc['loss_train'],
                   loss_hypar=float(mc['loss_hypar']),
                   loss_valid=mc['loss_valid'],
                   frequency=mc['frequency'],
                   seasonality=int(mc['seasonality']),
                   random_seed=int(mc['random_seed']))

{% endraw %}

### Train Model

{% raw %}

    from pytorch_lightning.callbacks import EarlyStopping

    early_stopping = EarlyStopping(monitor="val_loss",
                                   min_delta=1e-4,
                                   patience=mc['early_stop_patience'],
                                   verbose=False,
                                   mode="min")

    trainer = pl.Trainer(max_epochs=mc['max_epochs'],
                         max_steps=mc['max_steps'],
                         gradient_clip_val=1.0,
                         progress_bar_refresh_rate=10,
                         log_every_n_steps=500,
                         check_val_every_n_epoch=1,
                         callbacks=[early_stopping])

    trainer.fit(model, train_loader, val_loader)

    GPU available: True, used: False
    TPU available: False, using: 0 TPU cores
    /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py:597: UserWarning: GPU available but not used. Set the gpus flag in your trainer `Trainer(gpus=1)` or script `--gpus=1`.
      "GPU available but not used. Set the gpus flag in your trainer"

      | Name  | Type    | Params
    ----------------------------------
    0 | model | _NBEATS | 1.6 M
    ----------------------------------
    1.5 M     Trainable params
    113 K     Non-trainable params
    1.6 M     Total params
    6.436     Total estimated model params size (MB)
    /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/data_loading.py:103: UserWarning: The dataloader, val dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
      f'The dataloader, {name}, does not have many workers which may be a bottleneck.'

    /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/data_loading.py:103: UserWarning: The dataloader, train dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
      f'The dataloader, {name}, does not have many workers which may be a bottleneck.'

    /usr/local/lib/python3.7/dist-packages/pytorch_lightning/callbacks/model_checkpoint.py:611: LightningDeprecationWarning: Relying on `self.log('val_loss', ...)` to set the ModelCheckpoint monitor is deprecated in v1.2 and will be removed in v1.4. Please, create your own `mc = ModelCheckpoint(monitor='your_monitor')` and use it as `Trainer(callbacks=[mc])`.
      "Relying on `self.log('val_loss', ...)` to set the ModelCheckpoint monitor is deprecated in v1.2"

{% endraw %}

### Make Predictions

{% raw %}

    model.return_decomposition = True
    outputs = trainer.predict(model, val_loader)

    print("outputs[0][0].shape", outputs[0][0].shape)
    print("outputs[0][1].shape", outputs[0][1].shape)
    print("outputs[0][2].shape", outputs[0][2].shape)

    /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/data_loading.py:103: UserWarning: The dataloader, predict dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
      f'The dataloader, {name}, does not have many workers which may be a bottleneck.'

    outputs[0][0].shape torch.Size([42, 168])
    outputs[0][1].shape torch.Size([42, 168])
    outputs[0][2].shape torch.Size([42, 4, 168])

{% endraw %}