.. _adding-notebooks:

Adding or editing tutorials as Jupyter notebooks
------------------------------------------------

Under the ``doc/source/`` folder of the SciPy tree you can find a few
documents written in MyST Markdown format. The content in these files is
executed when the SciPy documentation is built (locally or on CI) and any
outputs generated by the execution are rendered in the final HTML files. In
addition, they can be made interactive in the SciPy documentation website by
using the
`jupyterlite-sphinx <https://jupyterlite-sphinx.readthedocs.io/en/stable/>`__
extension.

If you have a document written in Jupyter notebook format (an ``.ipynb`` file)
and would like to submit it as part of the SciPy documentation, there are two
options: you can convert it into a MyST Markdown file, and work with a ``.md``
file only, or you can pair your ``.ipynb`` file with a ``.md`` file and work
with both. Note that ``.ipynb`` files *should not* be submitted to the SciPy
documentation.

In this document, we address conversion to a MyST Markdown file. For more
information about MyST-NB, Jupytext and pairing notebooks, you can also consult
the `Pairing tutorial on NumPy Tutorials <https://numpy.org/numpy-tutorials/content/pairing.html>`__.
For even more information, please consult the
`MyST-NB documentation <https://myst-nb.readthedocs.io/en/stable/authoring/text-notebooks.html>`__.

How to convert a ``.ipynb`` file to a ``.md`` file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you don't need to keep the ``.ipynb`` file, and want to work with MyST
Markdown only, follow the steps below.

1. Install the jupytext_ tool, using ``pip install jupytext`` or
   ``conda install jupytext -c conda-forge``;
2. On your terminal, run ``jupytext notebook.ipynb --to myst``, where
   ``notebook.ipynb`` should be replaced with the file you want to convert.

.. _jupytext: https://jupytext.readthedocs.io

Now, the resulting ``.md`` file (in MyST Markdown format) should contain a
preamble similar to the one below, indicating that its contents will be executed
when the SciPy documentation is built:

::

   ---
   jupytext:
      text_representation:
         extension: .md
         format_name: myst
         format_version: 0.13
         jupytext_version: 1.14.0
   kernelspec:
      display_name: Python 3 (ipykernel)
      language: python
      name: python3
   ---

You don't need to edit this preamble, as it is autogenerated.

Links, formatting and images
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When converting a notebook to a MyST Markdown file, you may need to adjust some
of the content to match the MyST Markdown syntax.

- **External links:** MyST Markdown uses the standard Markdown syntax for links.
  For example, ``[link text](https://www.example.com)``.
- **Internal links:** For internal cross-references such as links to SciPy
  classes or functions, as well as intersphinx links, you can use the following
  syntax: ``{role}`link text <reference>```, where ``role`` can be ``ref``,
  ``class``, ``func`` or any other role you would use with reST. For example, to
  link to the ``scipy.stats.bartlett`` function, use
  ``{func}`scipy.stats.bartlett```.
- **Formatting:** MyST Markdown supports standard Markdown formatting, such as
  bold text, italic text, and code blocks. For example, to make text bold, you
  can use double asterisks: ``**bold text**``. To make text monospace/formatted
  as code, you can use *single* backticks: ```code```.
- **Images:** If your notebook contains static images, you can include them in
  the MyST Markdown file by using the following syntax:
  ``![alt text](path/to/image.png)``. Images are usually stored in the same
  folder as the MyST Markdown file, but you can also use relative paths to
  reference images in other folders. Note that outputs of executed cells should
  not be included in version control, as they will be automatically generated
  when the notebook is executed during the SciPy documentation build.
- **Linking to the MyST Markdown pages:** MyST Markdown supports adding link
  anchors, which can be used to link to specific pages or document sections from
  other documents. To add an anchor to a page, add ``(anchor_name)=`` to the
  desired place in the document to be referenced. From other markdown pages, you
  can link to this anchor using the following syntax: ``{ref}`anchor_name```.
  From other reST pages, you can link to this anchor using the following syntax:
  ``:ref:`anchor_name```.

Making the notebook interactive
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you want to enable interactivity for your notebook when it is rendered in
the SciPy documentation website, you need to add the following snippet at the
beginning of the markdown file:

.. code-block:: markdown

    ```{eval-rst}
    .. notebooklite:: <filename>.md
       :new_tab: True
    ```

where ``<filename>.md`` should be replaced with the name of the file you are
working with. This will create a button that allows users to open the notebook
in a new tab and interact with it. See
`Bartlett's test for equal variances <https://scipy.github.io/devdocs/tutorial/stats/hypothesis_bartlett.html>`__
(and `its source <https://github.com/scipy/scipy/blob/main/doc/source/tutorial/stats/hypothesis_bartlett.md?plain=1>`__)
for an example.

.. note::

    For simplicity, we don't want certain notebook cells to be visible in the
    rendered documentation. We can hide them by adding ``"jupyterlite_sphinx_strip"``
    to the tags section of the corresponding cell metadata. To do that in the
    Markdown file, add the following snippet right before the cell you want to hide:

    .. code-block:: markdown

        +++ {"tags": ["jupyterlite_sphinx_strip"]}

    Note that the `+++` notation should come in pairs, surrounding the cell you want to hide.

Visit the `NotebookLite directive documentation <https://jupyterlite-sphinx.readthedocs.io/en/stable/directives/notebooklite.html>`__
for more details and options.

Opening MyST Markdown files in the Jupyter Notebook application
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you have the ``jupytext`` tool installed, you can open MyST Markdown ``.md``
files in the Jupyter Notebook application and execute them, just as you would
with a ``.ipynb`` file.

Converting ``.rst`` files to MyST Markdown
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To convert a reStructuredText (``.rst``) file to MyST Markdown (``.md``), you
can follow the formatting instructions above. You can also use tools such as
`RST-to-MyST <https://rst-to-myst.readthedocs.io>`__ to automate the conversion,
but please note some review may be necessary to ensure the conversion is
correct. In particular, some reStructuredText directives may need to be wrapped
in the ``{eval-rst}`` MyST directive, as follows::

   Some Markdown here

   ```{eval-rst}
   .. some_rest_directive_here::
     :some_option: some_value
   ```

   Some more Markdown here

.. _MyST-NB: https://myst-nb.readthedocs.io/
