# SSB Publish Markdown

[![PyPI](https://img.shields.io/pypi/v/ssb-pubmd.svg)][pypi status]
[![Status](https://img.shields.io/pypi/status/ssb-pubmd.svg)][pypi status]
[![Python Version](https://img.shields.io/pypi/pyversions/ssb-pubmd)][pypi status]
[![License](https://img.shields.io/pypi/l/ssb-pubmd)][license]

[![Documentation](https://github.com/statisticsnorway/ssb-pubmd/actions/workflows/docs.yml/badge.svg)][documentation]
[![Tests](https://github.com/statisticsnorway/ssb-pubmd/actions/workflows/tests.yml/badge.svg)][tests]
[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=statisticsnorway_ssb-pubmd&metric=coverage)][sonarcov]
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=statisticsnorway_ssb-pubmd&metric=alert_status)][sonarquality]

[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)][pre-commit]
[![Black](https://img.shields.io/badge/code%20style-black-000000.svg)][black]
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

[pypi status]: https://pypi.org/project/ssb-pubmd/
[documentation]: https://statisticsnorway.github.io/ssb-pubmd
[tests]: https://github.com/statisticsnorway/ssb-pubmd/actions?workflow=Tests
[sonarcov]: https://sonarcloud.io/summary/overall?id=statisticsnorway_ssb-pubmd
[sonarquality]: https://sonarcloud.io/summary/overall?id=statisticsnorway_ssb-pubmd
[pre-commit]: https://github.com/pre-commit/pre-commit
[black]: https://github.com/psf/black
[poetry]: https://python-poetry.org/

## Features

- Write SSB articles using [Quarto Markdown](https://quarto.org/docs/authoring/markdown-basics.html) (qmd) files.
- Create SSB components using Python:
  - Highchart
  - Factbox
- Insert components in article using simple Markdown syntax.

## Requirements

- A VSCode or Jupyter service in Dapla Lab.
- An [SSB project](https://manual.dapla.ssb.no/statistikkere/ssb-project.html).

## Installation

```console
poetry add ssb-pubmd
```

Latest development version:

```console
poetry add ssb-pubmd@latest --source testpypi --allow-prereleases
```

## Usage

```console
poetry run ssb-pubmd create "My article title"
```

Follow further instructions in template file, and see the [Reference Guide] for details on creating components.

## Contributing

### Setup

1. Install [uv](https://docs.astral.sh/uv/getting-started/installation/) (python and package manager):

   ```console
   curl -LsSf https://astral.sh/uv/install.sh | sh
   ```

   (See link for installation on Windows.)

2. Clone this repo.

   ```console
   git clone https://github.com/statisticsnorway/ssb-pubmd.git
   ```

### Run tests

From the root folder (where `pyproject.toml` is), run:

```console
uv run pytest
```

### Testing the whole integration

- When you push changes to a branch, the package is published to [Test PyPI](https://test.pypi.org/) by the [release workflow](.github/workflows/release.yml).
- The test package can then be installed from a Dapla service and tested directly.

#### Steps

Use the following steps to test the most recently pushed change (to any branch):

1. Open a Jupyter service in [Dapla Dev](https://lab.dapla-dev.ssb.no/).
2. Open a Jupyter terminal and run the following commands:

   ```console
   ssb-project create my-project
   cd my-project
   poetry source add testpypi https://test.pypi.org/simple/ -p explicit
   poetry add "pandas<3"
   poetry add ssb-pubmd@latest --source testpypi --allow-prereleases
   poetry run ssb-pubmd create my-article
   ```

3. You can then continue testing in the same Dapla service. If you push new changes, you need to fetch these changes in the Dapla service with the following command:

   ```console
   poetry update ssb-pubmd
   ```

   Note that it takes about a minute (sometimes longer) for the PyPI Test package to be updated after pushing a change.


### Architecture

Dependency graph:

```mermaid
graph LR
   subgraph driving[Interface]
      cli[cli]
   end
   subgraph core[Core]
      project[create_project]
      article[render_article]
      docprocessor[Document processor]
      docpublisher[Document publisher]
   end
   subgraph driven[Adapters]
      cmsclient[Cms client]
      storage[Storage]
   end
   cmsservice[Cms service]

   cli --> project
   cli --> article --> docpublisher

   article --> docprocessor

   docpublisher --> cmsclient
   docpublisher --> storage

   cmsclient --> cmsservice

   classDef empty width:0px,height:0px;
```

Note: _Cms Service_ refers to [statisticsnorway/ssbno-app-pubmd](https://github.com/statisticsnorway/ssbno-app-pubmd).

General document flow:

1. The user asks to render their notebook file through the command-line interface.
2. The core method `render_article` uses the document processor to parse the file into a general `Document` object.
3. The `Document` object is passed into the document publisher, which extracts the content and passes it to the CMS client.
4. The CMS client sends the content to the CMS service and returns the parsed response.
5. The document processor stores the response, so that the id and path can be reused in future requests (to modify existing CMS content instead of creating new content).

Flow of components:

1. Components are defined programmatically by the user in the notebook file.
2. When the notebook is executed by the document processor, the `create_component` function (from [`__init__.py`](src/ssb_pubmd/__init__.py)) stores the component data in a temporary storage file, which is then added to the `Document` object. The temporary storage file is then deleted, to ensure sensitive data is not stored on disk.
3. The document publisher first extracts all the components from the `Document` objects and sends these to the Cms Client. During this process, the `Document` object is also modified so that components in the document tree are replaced with Cms references.
4. Finally the whole document is extracted (as html) and sent to the Cms client, with correct component references in the html.

### Updates to Mimir content types

This package exposes a Python interface to [mimir](https://github.com/statisticsnorway/mimir) content types; see the source file [interface/models.py](src/ssb_pubmd/interface/models.py).
When mimir types change, the corresponding Python interface can be updated following these steps.

#### Adding a new field

There are two simple steps of adding an extra field to an exposed content type:

1. Decide how the extra field should be exposed to the user.
   - The exposed models are defined in [interface/models.py](src/ssb_pubmd/interface/models.py).
   - Example of adding a required string field:

     ```py
     from pydantic import BaseModel
     class MyModel(BaseModel):
          extra_field: str
     ```

   - If the string field should be optional, a default value is given:

     ```py
     from pydantic import BaseModel
     class MyModel(BaseModel):
          extra_field: str | None = None
          another_field: str = "default-string"
     ```

   - See existing models for more examples. Remember to add a description of the field in the documentation string.

2. Add the extra field to the payload (that will be sent to the CMS).
   - The payload is created in the source file [adapters/cms_client/mimir.py](src/ssb_pubmd/adapters/cms_client/mimir.py).
   - If we have added a field to the `Highchart` model, we would add it to the payload in the method `_create_highchart_payload`.
   - The payload must match the Mimir content type. For instance, `Highchart` has an `xlabel` field which should be given as `xAxisLabel` in the payload.

     ```py
     payload["xAxisTitle"] = data.xlabel
     ```

   - The easiest way to see how the payload should look is to use the Content Viewer in Content Studio, or the generated Enonic types.

## License

Distributed under the terms of the [MIT license][license],
_SSB Publish Markdown_ is free and open source software.

## Issues

If you encounter any problems,
please [file an issue] along with a detailed description.

## Credits

This project was generated from [Statistics Norway]'s [SSB PyPI Template].

[statistics norway]: https://www.ssb.no/en
[pypi]: https://pypi.org/
[ssb pypi template]: https://github.com/statisticsnorway/ssb-pypitemplate
[file an issue]: https://github.com/statisticsnorway/ssb-pubmd/issues
[pip]: https://pip.pypa.io/

<!-- github-only -->

[license]: https://github.com/statisticsnorway/ssb-pubmd/blob/main/LICENSE
[contributor guide]: https://github.com/statisticsnorway/ssb-pubmd/blob/main/CONTRIBUTING.md
[reference guide]: https://statisticsnorway.github.io/ssb-pubmd/reference.html
