# Pack - how `east pack` works

Packing is the process of copying the build artifacts into a specific directory, usually for the
purpose of creating a release. All the files are also renamed to include the version tag, which is
automatically determined by `git` or provided via the `--tag` option to the `east pack` command.

The `east pack` command uses the `pack` key from the `east.yml` file to determine which files to
copy from build directories generated by
[Twister](https://docs.zephyrproject.org/latest/develop/test/twister.html).

Details are provided below, after the example.

## Example

We have an `app`, a `sample/blinky` and a `scripts/some_script/some_script.sh` file we would like to
pack. We build for 2 versions of our custom board (`v1.0.0` and `v1.4.0`) and the `nrf52840DK`.

After running Twister, we get the following `twister-out` folder structure:

```text
> tree twister-out -L 4
twister-out
├── custom_board@1.0.0_nrf52840
│   └── zephyr
│       └── app
│           ├── app.debug
│           ├── app.prod
│           ├── app.rtt
│           └── app.uart
├── custom_board@1.4.0_nrf52840
│   └── zephyr
│       ├── app
│       │   ├── app.debug
│       │   ├── app.prod
│       │   ├── app.rtt
│       │   └── app.uart
│       └── samples
│           └── blinky
├── nrf52840dk_nrf52840
│   └── zephyr
│       ├── app
│       │   ├── app.debug
│       │   ├── app.prod
│       │   ├── app.rtt
│       │   └── app.uart
│       └── samples
│           └── blinky
├── testplan.json
├── twister.json
├── twister.log
├── twister_report.xml
├── twister_suite_report.xml
└── twister.xml
```

Our `east.yml` file looks like this:

```yaml
pack:
  artifacts:
    - $APP_DIR/zephyr/zephyr.hex
    - $APP_DIR/zephyr/zephyr.bin
    - merged.hex
  build_configurations:
    - name: sample.blinky
      overwrite_artifacts:
        - $APP_DIR/zephyr/zephyr.hex
  extra:
    - scripts/update_docker_versions.sh
    - scripts/renode/install.sh
```

When we run `east pack`, the following `package` folder is generated:

```text
> east pack --tag v1.0.0
> tree package
package
├── app.debug
│   ├── custom_board@1.0.0_nrf52840
│   │   ├── app.debug-merged-v1.0.0.hex
│   │   ├── app.debug-zephyr-v1.0.0.bin
│   │   └── app.debug-zephyr-v1.0.0.hex
│   ├── custom_board@1.4.0_nrf52840
│   │   ├── app.debug-merged-v1.0.0.hex
│   │   ├── app.debug-zephyr-v1.0.0.bin
│   │   └── app.debug-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       ├── app.debug-merged-v1.0.0.hex
│       ├── app.debug-zephyr-v1.0.0.bin
│       └── app.debug-zephyr-v1.0.0.hex
├── app.debug-v1.0.0.zip
├── app.prod
│   ├── custom_board@1.0.0_nrf52840
│   │   ├── app.prod-merged-v1.0.0.hex
│   │   ├── app.prod-zephyr-v1.0.0.bin
│   │   └── app.prod-zephyr-v1.0.0.hex
│   ├── custom_board@1.4.0_nrf52840
│   │   ├── app.prod-merged-v1.0.0.hex
│   │   ├── app.prod-zephyr-v1.0.0.bin
│   │   └── app.prod-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       ├── app.prod-merged-v1.0.0.hex
│       ├── app.prod-zephyr-v1.0.0.bin
│       └── app.prod-zephyr-v1.0.0.hex
├── app.prod-v1.0.0.zip
├── app.rtt
│   ├── custom_board@1.0.0_nrf52840
│   │   ├── app.rtt-merged-v1.0.0.hex
│   │   ├── app.rtt-zephyr-v1.0.0.bin
│   │   └── app.rtt-zephyr-v1.0.0.hex
│   ├── custom_board@1.4.0_nrf52840
│   │   ├── app.rtt-merged-v1.0.0.hex
│   │   ├── app.rtt-zephyr-v1.0.0.bin
│   │   └── app.rtt-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       ├── app.rtt-merged-v1.0.0.hex
│       ├── app.rtt-zephyr-v1.0.0.bin
│       └── app.rtt-zephyr-v1.0.0.hex
├── app.rtt-v1.0.0.zip
├── app.uart
│   ├── custom_board@1.0.0_nrf52840
│   │   ├── app.uart-merged-v1.0.0.hex
│   │   ├── app.uart-zephyr-v1.0.0.bin
│   │   └── app.uart-zephyr-v1.0.0.hex
│   ├── custom_board@1.4.0_nrf52840
│   │   ├── app.uart-merged-v1.0.0.hex
│   │   ├── app.uart-zephyr-v1.0.0.bin
│   │   └── app.uart-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       ├── app.uart-merged-v1.0.0.hex
│       ├── app.uart-zephyr-v1.0.0.bin
│       └── app.uart-zephyr-v1.0.0.hex
├── app.uart-v1.0.0.zip
├── extra
│   ├── install-v1.0.0.sh
│   └── update_docker_versions-v1.0.0.sh
├── extra-v1.0.0.zip
├── sample.blinky
│   ├── custom_board@1.4.0_nrf52840
│   │   └── sample.blinky-zephyr-v1.0.0.hex
│   └── nrf52840dk_nrf52840
│       └── sample.blinky-zephyr-v1.0.0.hex
└── sample.blinky-v1.0.0.zip
```

## Details

To understand how `east pack` works, it is important to know how to use Twister and how to specify
build configurations with `sample.yaml` files.

For more details on how to use Twister to generate build artifacts, see our
[Zephyr template repo](https://github.com/IRNAS/irnas-zephyr-template) and read the docs there.

<!-- prettier-ignore -->
> [!NOTE]
> Each configuration built by Twister is called a **test-suite** by the Twister documentation,
> but will be referred to as a **build configuration** in this document.
> The term **test-suite** makes sense in the context of Twister, since its goal is to run tests.
> Here, we use Twister to generate build artifacts.
> Refer to the [Twister](https://docs.zephyrproject.org/latest/develop/test/twister.html)
> documentation for more details.

`east pack` has 3 inputs:

- The Twister output folder (and its corresponding `twister.json` file).
- The `east.yml` file in the root of the project.
- The `tag`, which specifies the version tag to use when naming the files.

When executed it performs the following steps:

1. For each build configuration found in `twister.json`:
   1. Create output directory `package/<build-configuration>/<board>/`.
   2. Determine which files to copy for this build configuration from `east.yml`.
   3. Copy the files and rename them according to the Twister artifact naming scheme (See below).
2. If any extra files are specified in `east.yml`:
   1. Create output directory `package/extra/`.
   2. For each extra file:
      1. Copy the file to `package/extra/`.
      2. Rename the file according to the extra artifact naming scheme (See below).
3. Zip top level directories in `package/` and place the zip files into `package/`.

### `east.yml` pack key

The `pack` key in `east.yml` specifies which files to copy for each of the build configurations. It
has the following sub-keys:

- `pack.artifacts` - a list of files to copy for all build configurations.
- `pack.build_configurations` - a list of specific build configurations. This allows you to
  overwrite the default artifacts or specify additional artifacts for a specific build
  configuration. Each item in the list must have a `name` and either `artifacts` or
  `overwrite_artifacts`:
  - `name` - the name of the build configurations. This **must match** the name of the build
    configuration as defined in the `sample.yaml` file.
  - `artifacts` - a list of files to copy for this specific build configuration in addition to files
    in `pack.artifacts`.
  - `overwrite_artifacts` - a list of files to copy for this specific build configuration, instead
    of the default artifacts specified in `pack.artifacts`.
- `pack.extra` - a list of additional files to copy that are not part of the Twister output, but are
  created by other means.

### File paths and the `$APP_DIR` variable

The files specified in the `artifacts` and `build_configurations` keys are **relative** to the build
folder of the build configuration (the build folders generated by Twister are the same as the ones
generated by `west build`).

Thus, specifying `merged.hex` means that `build/merged.hex` must exist and will be copied.

The build folder contains sub-folders with different names, depending on what is being built:

- When using sysbuild, the application files are always placed in a sub-folder named the same as the
  folder where the base `CMakeLists.txt` is located. For example, if the base `CMakeLists.txt` is in
  the `app/` folder, the application files will be in `build/app/`.
- When not using sysbuild, the application files are placed directly in `build/`.

Whether sysbuild is enabled and what the folder is named is different for each build configuration.
For ease of use, the `$APP_DIR` variable is used to refer to the application files build directory.

For example:

- if we have a sysbuild application in the `app/` folder, the `$APP_DIR` variable will be `app`.
- if we have a sysbuild application in the `samples/blinky/` folder, the `$APP_DIR` variable will be
  `blinky`.
- if we have a non-sysbuild application in the `app/` folder, the `$APP_DIR` variable will be `.`.

This is useful when specifying files in the `artifacts` folder, since they are common for all build
configurations, which have differently named sub-folders in their build folders.

### The destination of the copied files

All the specified files are copied to the `package` folder in the root of the project (or some other
folder as specified by the `--pack-path` option).

Extra files are copied to `package/extra`.

Files from Twister are copied to `package/<build-configuration>/<board>/` for each build
configuration and board combination (as specified in the `sample.yaml` files).

### File renaming and versioning

When copying files, `east pack` will rename them to include the version tag, and information about
the build configuration and board. The tag is either provided with the `--tag` option or is
determined from the current git tag (`git describe --tags --always --long --dirty=+`).

The files are renamed according to the file rename scheme, specified separately for Twister and
extra artifacts - the specification can be found [below](#artifact-naming-scheme).

To simplify:

- Twister artifacts will be renamed to `<build-configuration>-<filename>-<version>.<ext>`.
- Extra artifacts will be renamed to `<filename>-<version>.<ext>`.

### Zip file creation

After the files are copied, `east pack` will create zip files:

- For each build configuration, a zip file will be created in the `package` folder with the name
  `<build-configuration>-<version>.zip`.
- If there are extra files, `extra-<version>.zip` will be created.

These zip files are intended to be used for releases.

## Artifact naming scheme

### Twister artifacts naming scheme

Artifact renaming schema:

```text
<project_name>-[dirs-]<filename>-<version_str>.<ext>
```

Where:

| Field          | Description                                                                                                                                     |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `project_name` | The name of the project, more specifically the name of the project's build directory as output by Twister, e.g., app.prod, samples.blinky, etc. |
| `dirs`         | Directories in the path to the artifact file. Optional, see below.                                                                              |
| `filename`     | Unmodified filename of the artifact.                                                                                                            |
| `version_str`  | Version string.                                                                                                                                 |
| `ext`          | Unmodified file extension of the artifact.                                                                                                      |

The `dirs` part is optional and is determined by the path to the artifact file and its application
build directory. The intention is to use `dirs` part only for the cases where artifacts don't reside
in the commonly used locations.

The `dirs` part is omitted completely if the artifact is located directly in:

- The project's build directory, or
- the `zephyr` directory in Sysbuild's default applications's build directory, eg. under
  "$APP_DIR/zephyr".

If the artifact is located in the `zephyr` directory of some non-default application project, the
`dirs` part contains only the name of that non-default application project. For example, for
`"mcuboot/zephyr/zephyr.hex"`, `dirs` is just `"mcuboot"`.

For all other cases, the `dirs` part contains the full path to the artifact file, relative to the
project's build directory, with slashes replaced by dots.

---

#### Example

Given the following input:

- $APP_DIR = "blinky"
- project_name = "blinky.prod"
- version_str = "v1.0.0"

Twister artifacts will be renamed as follows:

| Artifact path                           | Renamed artifact name                                      |
| --------------------------------------- | ---------------------------------------------------------- |
| blinky/zephyr/zephyr.hex                | blinky.prod-zephyr-v1.0.0.hex                              |
| mcuboot/zephyr/zephyr.hex               | blinky.prod-mcuboot-zephyr-v1.0.0.hex                      |
| blinky/Kconfig/Kconfig.dts              | blinky.prod-blinky.Kconfig-Kconfig-v1.0.0.dts              |
| dfu_application.zip                     | blinky.prod-dfu_application-v1.0.0.zip                     |
| merged.hex                              | blinky.prod-merged-v1.0.0.hex                              |
| blinky/zephyr/arch/cmake_install.cmake  | blinky.prod-blinky.zephyr.arch-cmake_install-v1.0.0.cmake  |
| mcuboot/zephyr/arch/cmake_install.cmake | blinky.prod-mcuboot.zephyr.arch-cmake_install-v1.0.0.cmake |

### Extra artifacts naming scheme

Artifact renaming schema:

```text
<filename>-<version_str>.<ext>
```

Where:

| Field         | Description                                |
| ------------- | ------------------------------------------ |
| `filename`    | Unmodified filename of the artifact.       |
| `version_str` | Version string.                            |
| `ext`         | Unmodified file extension of the artifact. |

---

#### Example

Given the following input:

- `version_str` = "v1.0.0"

Extra artifacts will be renamed as follows:

| Artifact path                     | Renamed artifact name            |
| --------------------------------- | -------------------------------- |
| scripts/install/install.py        | install-v1.0.0.py                |
| scripts/update_docker_versions.sh | update_docker_versions-v1.0.0.sh |
| some/dir/generated.zip            | generated-v1.0.0.zip             |

## nrfutil flash pack

Build configurations with `nrfutil_flash_pack: True` in `east.yml` get a self-contained flash
package generated alongside the normal pack output. This package allows users to flash firmware
using only the `nrfutil` binary, without needing east, west, or Zephyr installed.

### Configuration

Set `nrfutil_flash_pack: True` on a build configuration in `east.yml`:

```yaml
pack:
  build_configurations:
    - name: app.prod
      nrfutil_flash_pack: True
```

### What gets generated

For each build configuration with `nrfutil_flash_pack: True`, the following files are added to the
build output directory inside `package/`:

```text
package/app.prod/
├── <batch files and firmware files>
├── README.md
├── linux/
│   ├── flash.sh
│   ├── erase.sh
│   ├── reset.sh
│   ├── recover.sh
│   └── nrfutil_setup.sh
└── windows/
    ├── flash.bat
    ├── erase.bat
    ├── reset.bat
    ├── recover.bat
    └── nrfutil_setup.bat
```

The batch JSON files and firmware binaries are discovered automatically by running
`west flash --dry-run` on each board's build output.

The README.md file contains instructions on how to use the flash package, and the `nrfutil_setup`
scripts check for the required tools and install the correct version of `nrfutil` if necessary.

## Projects with several twister-out folders

In some cases it is required for a project to run several Twister runs, due to the clashing command
line arguments. For example, Twister run for the production projects needs different arguments than
the one that generates radio test images.

Such setup would create several `twister-out` folders, e.g., `twister-out-prod` and
`twister-out-radio`. In such case a single `east.yml` can't be used to specify the packing
configuration for both Twister runs, since `east pack` always makes sure that the artifacts for the
build configurations specified in `east.yml` are present in the `twister-our` dir, and thus it can
only be used with one.

For this use case the `east pack` command can be run with the `--east-yml-path`, `--pack-path` and
`--twister-out-path` options, providing a way to specify the path to the `twister-out` folder, the
output `package` folder and most importantly, a different `east.yml` file.
