
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/04_eddy-lifetime_fit_gelu.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_04_eddy-lifetime_fit_gelu.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_04_eddy-lifetime_fit_gelu.py:


================================================
Example 4: Changing MLP Architecture and Fitting
================================================

This example is nearly identical to the Synthetic Data fit, however we use
a different neural network architecture in hopes of obtaining a better spectra fitting.
The same set-up for fitting the Kaimal spectra is used here as in Examples 2 and 3.
The only difference here is in the neural network architecture.
Although certain combinations of activation functions, such as ``GELU`` result in considerably
improved spectra fitting and terminal loss values, the resulting eddy lifetime functions may
be non-physical.

.. GENERATED FROM PYTHON SOURCE LINES 16-21

Import packages
---------------

First, we import the packages we need for this example. Additionally, we choose to use
CUDA if it is available.

.. GENERATED FROM PYTHON SOURCE LINES 21-36

.. code-block:: Python

    import torch
    import torch.nn as nn

    from drdmannturb.parameters import (
        LossParameters,
        NNParameters,
        PhysicalParameters,
        ProblemParameters,
    )
    from drdmannturb.spectra_fitting import CalibrationProblem, OnePointSpectraDataGenerator

    device = "cuda" if torch.cuda.is_available() else "cpu"

    if torch.cuda.is_available():
        torch.set_default_tensor_type("torch.cuda.FloatTensor")







.. GENERATED FROM PYTHON SOURCE LINES 37-40

Set up physical parameters and domain associated with the Kaimal spectrum.
We perform the spectra fitting over the :math:`k_1` space :math:`[10^{{-1}}, 10^2]`
with 20 points.

.. GENERATED FROM PYTHON SOURCE LINES 40-53

.. code-block:: Python


    zref = 40  # reference height
    ustar = 1.773  # friction velocity

    # Scales associated with Kaimal spectrum
    L = 0.59 * zref  # length scale
    Gamma = 3.9  # time scale
    sigma = 3.2 * ustar**2.0 / zref ** (2.0 / 3.0)  # magnitude (σ = αϵ^{2/3})

    print(f"Physical Parameters: {L,Gamma,sigma}")

    k1 = torch.logspace(-1, 2, 20) / zref





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Physical Parameters: (23.599999999999998, 3.9, 0.8600574364289042)




.. GENERATED FROM PYTHON SOURCE LINES 54-67

%%
Now, we construct our ``CalibrationProblem``.

Compared to Examples 2 and 3, we are using
a more complicated neural network architecture. This time, specifically, our
network will have 4 layers of width 10, 20, 20, 10 respectively, and we
use both ``GELU`` and ``RELU`` activations. We have
prescribed more Wolfe iterations.
Finally, this task is considerably more difficult than before since the exponent of
the eddy lifetime function :math:`\nu` is to be learned. Much more training
may be necessary to obtain a close fit to the eddy lifetime function. Interestingly,
learning this parameter results in models that more accurately describe the spectra of
Mann turbulence than using the Mann model itself.

.. GENERATED FROM PYTHON SOURCE LINES 67-84

.. code-block:: Python


    pb = CalibrationProblem(
        nn_params=NNParameters(
            nlayers=4,
            # Specifying the activations is done as in Examples 2 and 3.
            hidden_layer_sizes=[10, 20, 20, 10],
            activations=[nn.ReLU(), nn.GELU(), nn.GELU(), nn.ReLU()],
        ),
        prob_params=ProblemParameters(nepochs=25, wolfe_iter_count=20),
        loss_params=LossParameters(alpha_pen2=1.0, beta_reg=1.0e-5),
        phys_params=PhysicalParameters(
            L=L, Gamma=Gamma, sigma=sigma, ustar=ustar, domain=k1
        ),
        logging_directory="runs/synthetic_fit_deep_arch",
        device=device,
    )








.. GENERATED FROM PYTHON SOURCE LINES 85-89

Data Generation
---------------
In the following cell, we construct our :math:`k_1` data points grid and
generate the values. ``Data`` will be a tuple ``(<data points>, <data values>)``.

.. GENERATED FROM PYTHON SOURCE LINES 89-91

.. code-block:: Python

    Data = OnePointSpectraDataGenerator(data_points=k1, zref=zref, ustar=ustar).Data








.. GENERATED FROM PYTHON SOURCE LINES 92-96

Training
--------
Now, we fit our model. ``CalibrationProblem.calibrate()`` takes the tuple ``Data``
which we just constructed and performs a typical training loop.

.. GENERATED FROM PYTHON SOURCE LINES 96-100

.. code-block:: Python

    optimal_parameters = pb.calibrate(data=Data)

    pb.print_calibrated_params()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ========================================
    Initial loss: 0.04514402146682933
    ========================================

      0%|          | 0/25 [00:00<?, ?it/s]
      4%|▍         | 1/25 [00:04<01:58,  4.93s/it]
      8%|▊         | 2/25 [00:08<01:40,  4.38s/it]
     12%|█▏        | 3/25 [00:12<01:32,  4.21s/it]
     16%|█▌        | 4/25 [00:17<01:27,  4.19s/it]
     20%|██        | 5/25 [00:21<01:22,  4.12s/it]
     24%|██▍       | 6/25 [00:25<01:17,  4.09s/it]
     28%|██▊       | 7/25 [00:29<01:13,  4.06s/it]
     32%|███▏      | 8/25 [00:33<01:09,  4.09s/it]
     36%|███▌      | 9/25 [00:37<01:05,  4.12s/it]
     40%|████      | 10/25 [00:41<01:01,  4.08s/it]
     44%|████▍     | 11/25 [00:45<00:57,  4.11s/it]
     48%|████▊     | 12/25 [00:49<00:52,  4.07s/it]
     52%|█████▏    | 13/25 [00:53<00:48,  4.05s/it]
     56%|█████▌    | 14/25 [00:57<00:44,  4.08s/it]
     60%|██████    | 15/25 [01:01<00:40,  4.06s/it]
     64%|██████▍   | 16/25 [01:05<00:36,  4.09s/it]
     68%|██████▊   | 17/25 [01:09<00:32,  4.06s/it]
     72%|███████▏  | 18/25 [01:14<00:28,  4.09s/it]
     76%|███████▌  | 19/25 [01:18<00:24,  4.11s/it]
     80%|████████  | 20/25 [01:22<00:20,  4.17s/it]
     84%|████████▍ | 21/25 [01:26<00:16,  4.12s/it]
     88%|████████▊ | 22/25 [01:30<00:12,  4.08s/it]
     92%|█████████▏| 23/25 [01:34<00:08,  4.10s/it]
     96%|█████████▌| 24/25 [01:39<00:04,  4.21s/it]Spectra Fitting Concluded with loss below tolerance. Final loss: 0.0009987765556514398

     96%|█████████▌| 24/25 [01:43<00:04,  4.30s/it]
    ========================================
    Spectra fitting concluded with final loss: 0.0009987765556514398
    ========================================
    Optimal calibrated L        :  22.4707 
    Optimal calibrated Γ        :   4.9734 
    Optimal calibrated αϵ^{2/3} :   0.8399 
    ========================================




.. GENERATED FROM PYTHON SOURCE LINES 101-104

Plotting
--------
Lastly, we'll use built-in plotting utilities to see the fit result.

.. GENERATED FROM PYTHON SOURCE LINES 104-106

.. code-block:: Python

    pb.plot()




.. image-sg:: /auto_examples/images/sphx_glr_04_eddy-lifetime_fit_gelu_001.png
   :alt: One-point spectra, Eddy lifetime
   :srcset: /auto_examples/images/sphx_glr_04_eddy-lifetime_fit_gelu_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 107-111

This plots the loss function terms as specified, each multiplied by the
respective coefficient hyperparameter. The training logs can be accessed from the logging directory
with Tensorboard utilities, but we also provide a simple internal utility for a single
training log plot.

.. GENERATED FROM PYTHON SOURCE LINES 111-112

.. code-block:: Python

    pb.plot_losses(run_number=0)



.. image-sg:: /auto_examples/images/sphx_glr_04_eddy-lifetime_fit_gelu_002.png
   :alt: Total Loss, Regularization, 2nd Order Penalty, MSE Loss
   :srcset: /auto_examples/images/sphx_glr_04_eddy-lifetime_fit_gelu_002.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (1 minutes 44.262 seconds)


.. _sphx_glr_download_auto_examples_04_eddy-lifetime_fit_gelu.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: 04_eddy-lifetime_fit_gelu.ipynb <04_eddy-lifetime_fit_gelu.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: 04_eddy-lifetime_fit_gelu.py <04_eddy-lifetime_fit_gelu.py>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
