# Pack - how `east pack` works

Packing is the process of copying the build artifacts into a specific directory, usually for the
purpose of creating a release. All the files are also renamed to include the version tag, which is
automatically determined by `git` or provided via the `--tag` option to the `east pack` command.

The `east pack` command uses the `pack` key from the `east.yml` file to determine which files to
copy from build directories generated by
[Twister](https://docs.zephyrproject.org/latest/develop/test/twister.html).

Details are provided below, after the example.

## Example

We have an `app`, a `sample/blinky` and a `scripts/some_script/some_script.sh` file we would like to
pack. We build for 2 versions of our custom board (`v1.0.0` and `v1.4.0`) and the `nrf52840DK`.

After running Twister, we get the following `twister-out` folder structure:

```text
> tree twister-out -L 4
twister-out
├── custom_board@1.0.0_nrf52840
│   └── zephyr
│       └── app
│           ├── app.debug
│           ├── app.prod
│           ├── app.rtt
│           └── app.uart
├── custom_board@1.4.0_nrf52840
│   └── zephyr
│       ├── app
│       │   ├── app.debug
│       │   ├── app.prod
│       │   ├── app.rtt
│       │   └── app.uart
│       └── samples
│           └── blinky
├── nrf52840dk_nrf52840
│   └── zephyr
│       ├── app
│       │   ├── app.debug
│       │   ├── app.prod
│       │   ├── app.rtt
│       │   └── app.uart
│       └── samples
│           └── blinky
├── testplan.json
├── twister.json
├── twister.log
├── twister_report.xml
├── twister_suite_report.xml
└── twister.xml
```

Our `east.yml` file looks like this:

```yaml
pack:
  artifacts:
    - $APP_DIR/zephyr/zephyr.hex
    - $APP_DIR/zephyr/zephyr.bin
    - merged.hex
  build_configurations:
    - name: sample.blinky
      overwrite_artifacts:
        - $APP_DIR/zephyr/zephyr.hex
  extra:
    - scripts/update_docker_versions.sh
    - scripts/renode/install.sh
```

When we run `east pack`, the following `package` folder is generated:

```text
> east pack --tag v1.0.0
> tree package
package
├── app.debug
│   ├── custom_board@1.0.0_nrf52840
│   │   ├── app.debug-merged-v1.0.0.hex
│   │   ├── app.debug-zephyr-v1.0.0.bin
│   │   └── app.debug-zephyr-v1.0.0.hex
│   ├── custom_board@1.4.0_nrf52840
│   │   ├── app.debug-merged-v1.0.0.hex
│   │   ├── app.debug-zephyr-v1.0.0.bin
│   │   └── app.debug-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       ├── app.debug-merged-v1.0.0.hex
│       ├── app.debug-zephyr-v1.0.0.bin
│       └── app.debug-zephyr-v1.0.0.hex
├── app.debug-v1.0.0.zip
├── app.prod
│   ├── custom_board@1.0.0_nrf52840
│   │   ├── app.prod-merged-v1.0.0.hex
│   │   ├── app.prod-zephyr-v1.0.0.bin
│   │   └── app.prod-zephyr-v1.0.0.hex
│   ├── custom_board@1.4.0_nrf52840
│   │   ├── app.prod-merged-v1.0.0.hex
│   │   ├── app.prod-zephyr-v1.0.0.bin
│   │   └── app.prod-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       ├── app.prod-merged-v1.0.0.hex
│       ├── app.prod-zephyr-v1.0.0.bin
│       └── app.prod-zephyr-v1.0.0.hex
├── app.prod-v1.0.0.zip
├── app.rtt
│   ├── custom_board@1.0.0_nrf52840
│   │   ├── app.rtt-merged-v1.0.0.hex
│   │   ├── app.rtt-zephyr-v1.0.0.bin
│   │   └── app.rtt-zephyr-v1.0.0.hex
│   ├── custom_board@1.4.0_nrf52840
│   │   ├── app.rtt-merged-v1.0.0.hex
│   │   ├── app.rtt-zephyr-v1.0.0.bin
│   │   └── app.rtt-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       ├── app.rtt-merged-v1.0.0.hex
│       ├── app.rtt-zephyr-v1.0.0.bin
│       └── app.rtt-zephyr-v1.0.0.hex
├── app.rtt-v1.0.0.zip
├── app.uart
│   ├── custom_board@1.0.0_nrf52840
│   │   ├── app.uart-merged-v1.0.0.hex
│   │   ├── app.uart-zephyr-v1.0.0.bin
│   │   └── app.uart-zephyr-v1.0.0.hex
│   ├── custom_board@1.4.0_nrf52840
│   │   ├── app.uart-merged-v1.0.0.hex
│   │   ├── app.uart-zephyr-v1.0.0.bin
│   │   └── app.uart-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       ├── app.uart-merged-v1.0.0.hex
│       ├── app.uart-zephyr-v1.0.0.bin
│       └── app.uart-zephyr-v1.0.0.hex
├── app.uart-v1.0.0.zip
├── extra
│   ├── install-v1.0.0.sh
│   └── update_docker_versions-v1.0.0.sh
├── extra-v1.0.0.zip
├── sample.blinky
│   ├── custom_board@1.4.0_nrf52840
│   │   └── sample.blinky-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       └── sample.blinky-zephyr-v1.0.0.hex
└── sample.blinky-v1.0.0.zip
```

## Details

To understand how `east pack` works, it is important to know how to use Twister and how to specify
build configurations with `sample.yaml` files.

For more details on how to use Twister to generate build artifacts, see our
[Zephyr template repo](https://github.com/IRNAS/irnas-zephyr-template) and read the docs there.

<!-- prettier-ignore -->
> [!NOTE]
> Each configuration built by Twister is called a **test-suite** by the Twister documentation,
> but will be referred to as a **build configuration** in this document.
> The term **test-suite** makes sense in the context of Twister, since its goal is to run tests.
> Here, we use Twister to generate build artifacts.
> Refer to the [Twister](https://docs.zephyrproject.org/latest/develop/test/twister.html)
> documentation for more details.

`east pack` has 3 inputs:

- The Twister output folder (and its corresponding `twister.json` file).
- The `east.yml` file in the root of the project.
- The `tag`, which specifies the version tag to use when naming the files.

When executed it performs the following steps:

1. For each build configuration found in `twister.json`:
   1. Create output directory `package/<build-configuration>/<board>/`.
   2. Determine which files to copy for this build configuration from `east.yml`.
   3. Copy the files and rename them according to the Twister artifact naming scheme (See below).
2. If any extra files are specified in `east.yml`:
   1. Create output directory `package/extra/`.
   2. For each extra file:
      1. Copy the file to `package/extra/`.
      2. Rename the file according to the extra artifact naming scheme (See below).
3. Zip top level directories in `package/` and place the zip files into `package/`.

### `east.yml` pack key

The `pack` key in `east.yml` specifies which files to copy for each of the build configurations. It
has the following sub-keys:

- `pack.artifacts` - a list of files to copy for all build configurations.
- `pack.build_configurations` - a list of specific build configurations. This allows you to
  overwrite the default artifacts or specify additional artifacts for a specific build
  configuration. Each item in the list must have a `name` and either `artifacts` or
  `overwrite_artifacts`:
  - `name` - the name of the build configurations. This **must match** the name of the build
    configuration as defined in the `sample.yaml` file.
  - `artifacts` - a list of files to copy for this specific build configuration in addition to files
    in `pack.artifacts`.
  - `overwrite_artifacts` - a list of files to copy for this specific build configuration, instead
    of the default artifacts specified in `pack.artifacts`.
- `pack.extra` - a list of additional files to copy that are not part of the Twister output, but are
  created by other means.

### File paths and the `$APP_DIR` variable

The files specified in the `artifacts` and `build_configurations` keys are **relative** to the build
folder of the build configuration (the build folders generated by Twister are the same as the ones
generated by `west build`).

Thus, specifying `merged.hex` means that `build/merged.hex` must exist and will be copied.

The build folder contains sub-folders with different names, depending on what is being built:

- When using sysbuild, the application files are always placed in a sub-folder named the same as the
  folder where the base `CMakeLists.txt` is located. For example, if the base `CMakeLists.txt` is in
  the `app/` folder, the application files will be in `build/app/`.
- When not using sysbuild, the application files are placed directly in `build/`.

Whether sysbuild is enabled and what the folder is named is different for each build configuration.
For ease of use, the `$APP_DIR` variable is used to refer to the application files build directory.

For example:

- if we have a sysbuild application in the `app/` folder, the `$APP_DIR` variable will be `app`.
- if we have a sysbuild application in the `samples/blinky/` folder, the `$APP_DIR` variable will be
  `blinky`.
- if we have a non-sysbuild application in the `app/` folder, the `$APP_DIR` variable will be `.`.

This is useful when specifying files in the `artifacts` folder, since they are common for all build
configurations, which have differently named sub-folders in their build folders.

### The destination of the copied files

All the specified files are copied to the `package` folder in the root of the project (or some other
folder as specified by the `--pack-path` option).

Extra files are copied to `package/extra`.

Files from Twister are copied to `package/<build-configuration>/<board>/` for each build
configuration and board combination (as specified in the `sample.yaml` files).

### File renaming and versioning

When copying files, `east pack` will rename them to include the version tag, and information about
the build configuration and board. The tag is either provided with the `--tag` option or is
determined from the current git tag (`git describe --tags --always --long --dirty=+`).

The files are renamed according to the file rename scheme, specified separately for Twister and
extra artifacts - the specification can be found [below](#artifact-naming-scheme).

To simplify:

- Twister artifacts will be renamed to `<build-configuration>-<filename>-<version>.<ext>`.
- Extra artifacts will be renamed to `<filename>-<version>.<ext>`.

### Zip file creation

After the files are copied, `east pack` will create zip files:

- For each build configuration, a zip file will be created in the `package` folder with the name
  `<build-configuration>-<version>.zip`.
- If there are extra files, `extra-<version>.zip` will be created.

These zip files are intended to be used for releases.

## Artifact naming scheme

### Twister artifacts naming scheme

Artifact renaming schema:

```text
<project_name>-[dirs-]<filename>-<version_str>.<ext>
```

Where:

| Field          | Description                                                                                                                                     |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `project_name` | The name of the project, more specifically the name of the project's build directory as output by Twister, e.g., app.prod, samples.blinky, etc. |
| `dirs`         | Directories in the path to the artifact file. Optional, see below.                                                                              |
| `filename`     | Unmodified filename of the artifact.                                                                                                            |
| `version_str`  | Version string.                                                                                                                                 |
| `ext`          | Unmodified file extension of the artifact.                                                                                                      |

The `dirs` part is optional and is determined by the path to the artifact file and its application
build directory. The intention is to use `dirs` part only for the cases where artifacts don't reside
in the commonly used locations.

The `dirs` part is omitted completely if the artifact is located directly in:

- The project's build directory, or
- the `zephyr` directory in Sysbuild's default applications's build directory, eg. under
  "$APP_DIR/zephyr".

If the artifact is located in the `zephyr` directory of some non-default application project, the
`dirs` part contains only the name of that non-default application project. For example, for
`"mcuboot/zephyr/zephyr.hex"`, `dirs` is just `"mcuboot"`.

For all other cases, the `dirs` part contains the full path to the artifact file, relative to the
project's build directory, with slashes replaced by dots.

---

#### Example

Given the following input:

- $APP_DIR = "blinky"
- project_name = "blinky.prod"
- version_str = "v1.0.0"

Twister artifacts will be renamed as follows:

| Artifact path                           | Renamed artifact name                                      |
| --------------------------------------- | ---------------------------------------------------------- |
| blinky/zephyr/zephyr.hex                | blinky.prod-zephyr-v1.0.0.hex                              |
| mcuboot/zephyr/zephyr.hex               | blinky.prod-mcuboot-zephyr-v1.0.0.hex                      |
| blinky/Kconfig/Kconfig.dts              | blinky.prod-blinky.Kconfig-Kconfig-v1.0.0.dts              |
| dfu_application.zip                     | blinky.prod-dfu_application-v1.0.0.zip                     |
| merged.hex                              | blinky.prod-merged-v1.0.0.hex                              |
| blinky/zephyr/arch/cmake_install.cmake  | blinky.prod-blinky.zephyr.arch-cmake_install-v1.0.0.cmake  |
| mcuboot/zephyr/arch/cmake_install.cmake | blinky.prod-mcuboot.zephyr.arch-cmake_install-v1.0.0.cmake |

### Extra artifacts naming scheme

Artifact renaming schema:

```text
<filename>-<version_str>.<ext>
```

Where:

| Field         | Description                                |
| ------------- | ------------------------------------------ |
| `filename`    | Unmodified filename of the artifact.       |
| `version_str` | Version string.                            |
| `ext`         | Unmodified file extension of the artifact. |

---

#### Example

Given the following input:

- `version_str` = "v1.0.0"

Extra artifacts will be renamed as follows:

| Artifact path                     | Renamed artifact name            |
| --------------------------------- | -------------------------------- |
| scripts/install/install.py        | install-v1.0.0.py                |
| scripts/update_docker_versions.sh | update_docker_versions-v1.0.0.sh |
| some/dir/generated.zip            | generated-v1.0.0.zip             |

## nrfutil flash pack

Build configurations with `nrfutil_flash_pack: True` in `east.yml` get a self-contained flash
package generated alongside the normal pack output. This package allows users to flash firmware
using only the `nrfutil` binary, without needing east, west, or Zephyr installed.

### Configuration

Set `nrfutil_flash_pack: True` on a build configuration in `east.yml`:

```yaml
pack:
  build_configurations:
    - name: app.prod
      nrfutil_flash_pack: True
```

### What gets generated

For each build configuration with `nrfutil_flash_pack: True`, the following files are added to the
build output directory inside `package/`:

```text
package/app.prod/
├── <batch files and firmware files>
├── README.md
├── linux/
│   ├── flash.sh
│   ├── erase.sh
│   ├── reset.sh
│   ├── recover.sh
│   └── nrfutil_setup.sh
└── windows/
    ├── flash.bat
    ├── erase.bat
    ├── reset.bat
    ├── recover.bat
    └── nrfutil_setup.bat
```

The batch JSON files and firmware binaries are discovered automatically by running
`west flash --dry-run` on each board's build output.

The README.md file contains instructions on how to use the flash package, and the `nrfutil_setup`
scripts check for the required tools and install the correct version of `nrfutil` if necessary.
