# ESP-IDF Size Tool

## Description

The **esp-idf-size** tool provides developers with the means to explore
statically allocated RAM used by the final binary firmware image generated by
[ESP-IDF](https://github.com/espressif/esp-idf) SDK for Espressif devices.  It
utilizes the link map file, along with the ELF file and
*project_description.json* (if available), and memory type descriptions for
Espressif chips to create a generic memory map dictionary. For more information
please see the [`Memory Map Dictionary`](#memory-map-dictionary) section. This dictionary is
used to display static memory allocation in various formats and levels of
detail. Additionally, it enables the comparison of two memory maps to identify
unexpected changes in sizes.

**Please be aware that this documentation relates to the refactored version,
which must be explicitly invoked.**

## Installation

The **esp_idf_size** package can be installed via pip:

```sh
pip install esp-idf-size
```

## Basic Usage

```sh
python -m esp_idf_size --ng [<options>] MAP_FILE
python -m esp_idf_size.ng [<options>] MAP_FILE
```

or set the **ESP_IDF_SIZE_NG** environment variable to switch to the refactored version

```sh
export ESP_IDF_SIZE_NG=1
python -m esp_idf_size [<options>] MAP_FILE
```

## Options

**esp-idf-size** provides options to customize the output in various ways.

### Reporting Options

By default, esp-idf-size displays a basic summary of static memory usage in a
table format. The level of detail can be adjusted using the following options.

- **--archives**

    Print per-archive sizes.

- **--files**

    Print per-file sizes.

-  **--archive-details** *ARCHIVE_NAME* or **--archive_details** *ARCHIVE_NAME*

    Print detailed symbols per *ARCHIVE_NAME*

esp-idf-size can also show archives' dependencies or reverse dependencies

-  **--archive-dependencies**

    Print archives' dependencies or reverse dependencies.

### Formating Options

By default, the **table** format is used for all reports under **Reporting Options**.
All formats are generated from the generic memory map dictionary, except for
the **--archive-dependencies** option, which uses the cross-reference table from the
link map file.

-  **--format** *FORMAT*

    Display report in a designated *FORMAT*.

    - **table** or **text**: This is the default format.
        ```
        $ python -m esp_idf_size --ng build/hello_world.map
                                     Memory Type Usage Summary
        ┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
        ┃ Memory Type/Section   ┃ Used [bytes] ┃ Used [%] ┃ Remain [bytes] ┃ Total [bytes] ┃
        ┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
        │ Flash Code            │        80666 │     2.41 │        3261638 │       3342304 │
        │    .text              │        80666 │     2.41 │                │               │
        │ IRAM                  │        51835 │    39.55 │          79237 │        131072 │
        │    .text              │        50807 │    38.76 │                │               │
        │    .vectors           │         1027 │     0.78 │                │               │
        │ Flash Data            │        38224 │     0.91 │        4156048 │       4194272 │
        │    .rodata            │        37968 │     0.91 │                │               │
        │    .appdesc           │          256 │     0.01 │                │               │
        │ DRAM                  │        11236 │     6.22 │         169500 │        180736 │
        │    .data              │         8988 │     4.97 │                │               │
        │    .bss               │         2248 │     1.24 │                │               │
        │ RTC SLOW              │           24 │     0.29 │           8168 │          8192 │
        │    .rtc_slow_reserved │           24 │     0.29 │                │               │
        └───────────────────────┴──────────────┴──────────┴────────────────┴───────────────┘
        Total image size: 179712 bytes (.bin may be padded larger)
        ```
    - **tree**: Tree based format.
        ```
        $ python -m esp_idf_size --ng --format tree build/hello_world.map
        Memory Types
        ├── Flash Code 80666 / 3342304
        │   └── .text 80666
        │       ├── libc.a 42033
        │       │   ├── libc_a-vfprintf.o 13564
        │       │   ├── libc_a-vfiprintf.o 9516
        │       │   ├── libc_a-svfiprintf.o 9394
        │       │   ├── libc_a-dtoa.o 3601
        │       │   ├── libc_a-mprec.o 2626
        │       │   ├── libc_a-fseeko.o 730
        │       │   ├── libc_a-fvwrite.o 687
        │       │   ├── libc_a-refill.o 281
        │       │   ├── libc_a-makebuf.o 247
        │       │   ├── libc_a-fopen.o 232
        │       │   ├── libc_a-reent.o 188
        │       │   ├── libc_a-wsetup.o 174
        │       │   ├── libc_a-puts.o 138
        │       │   ├── libc_a-flags.o 128
        │       │   ├── libm_a-s_frexp.o 99
        │       │   ├── libc_a-printf.o 94
        │       │   ├── libc_a-mbtowc_r.o 64
        │       │   ├── libc_a-vprintf.o 61
        │       │   ├── libc_a-fwalk.o 55
        │       │   ├── libc_a-fseek.o 49
        │       │   ├── libc_a-localeconv.o 47
        │       │   ├── libc_a-sysfstat.o 24
        │       │   ├── libc_a-sysgettod.o 24
        │       │   └── libc_a-errno.o 10
        │       ├── libesp_system.a 7470
        │       │   ├── task_wdt.c.obj 1865
        │       │   ├── panic.c.obj 850
        ...
        ```
    - **csv**: CSV version of the **table** format.
        ```
        $ python -m esp_idf_size --ng --format csv build/hello_world.map
        "Memory Type/Section","Used [bytes]","Used [%]","Remain [bytes]","Total [bytes]"
        "Flash Code","80666","2.41","3261638","3342304"
        ".text","80666","2.41","",""
        "IRAM","51835","39.55","79237","131072"
        ".text","50807","38.76","",""
        ".vectors","1027","0.78","",""
        "Flash Data","38224","0.91","4156048","4194272"
        ".rodata","37968","0.91","",""
        ".appdesc","256","0.01","",""
        "DRAM","11236","6.22","169500","180736"
        ".data","8988","4.97","",""
        ".bss","2248","1.24","",""
        "RTC SLOW","24","0.29","8168","8192"
        ".rtc_slow_reserved","24","0.29","",""
        ```
    - **json2**: JSON format of the specified report
        ```
        $ python -m esp_idf_size --ng --format json2 build/hello_world.map
        {
            "version": "1.0",
            "layout": [
                {
                    "name": "Flash Code",
                    "total": 3342304,
                    "used": 80666,
                    "free": 3261638,
                    "parts": {
                        ".text": {
                            "size": 80666
                        }
                    }
                },
                {
                    "name": "IRAM",
                    "total": 131072,
                    "used": 51835,
                    "free": 79237,
                    "parts": {
                        ".text": {
                            "size": 50807
                        },
                        ".vectors": {
                            "size": 1027
                        }
                    }
                },

        ...
        ```
    - **raw** : Show the unprocessed Python dictionary with the memory map.

        This format ignores **Reporting Options**, with the exception of **--archive-dependencies**.
        ```
        $ python -m esp_idf_size --ng --format raw build/hello_world.map
        {
            "version": "1.0",
            "target": "esp32",
            "target_diff": "",
            "image_size": 179712,
            "image_size_diff": 0,
            "project_path": "/home/fhrbata/work/esp-idf/examples/get-started/hello_world/build/hello_world.map",
            "project_path_diff": "",
            "memory_types": {
                "DRAM": {
                    "size": 180736,
                    "size_diff": 0,
                    "used": 11236,
                    "used_diff": 0,
                    "sections": {
                        ".dram0.bss": {
                            "abbrev_name": ".bss",
                            "size": 2248,
                            "size_diff": 0,
                            "archives": {
                                "esp-idf/esp_app_format/libesp_app_format.a": {
                                    "abbrev_name": "libesp_app_format.a",
                                    "size": 10,
                                    "size_diff": 0,
                                    "object_files": {
                                        "esp_app_desc.c.obj": {
                                            "abbrev_name": "esp_app_desc.c.obj",
                                            "size": 10,
                                            "size_diff": 0,
                                            "symbols": {
                                                "app_elf_sha256_str": {
                                                    "abbrev_name": "app_elf_sha256_str",
                                                    "size": 10,
                                                    "size_diff": 0
                                                }
                                            }
                                        }
                                    }
                                },
        ...
        ```
    - **dot** : Show archives' dependencies in DOT format

        It can serve as input to generate a dependencies graph for archives, such as by utilizing [Graphviz][1].
        This format is supported only for **--archive-dependencies** report.
        ```
        $ python -m esp_idf_size --ng --filter main --archives-deps --format dot -o deps.dot build/hello_world.map
        $ dot -Tpdf -o deps.pdf deps.dot
        ```

Items in the report can be modified using the following options.

-  **--show-unused**

    Display unused memory types and sections, which are omitted by default.

-  **--show-unchanged**

    Show unchanged items for **--diff** operation, which are omitted by default.

- **--no-abbrev**

    Do not abbreviate section and file names.

-  **--unify**

    Use abbreviated names with aggregated size information. For example *.dram0.bss*
    and *.dram1.bss* sections will be reported under one *.bss* section. *Archives*,
    *object files* and *symbols* will be aggregated too. This can be useful for the
    **--diff** option when comparing project built with different ESP-IDF versions.


### Sorting Options

Applies only to **table** and **csv** formats, except when
**--archive-dependencies** is used; otherwise, it is ignored.

- **-s** *COLUMN* or **--sort** *COLUMN*

    Sort **table** or **csv** format rows based on specified column number,
    starting from 0. Column can be specified also as negative number, where
    *-1* means last column. Default is *1* and column *0*, containing row
    description, cannot be used.  The name of the column can be utilized in
    place of its numerical identifier.

- **--sort-reverse**

    Sort entries in reversed order. By default descending order is used.

- **--sort-diff**

    Sort entries based on diff value instead of size.

### Filtering Options

-  **-F** *PATTERN*, **--filter** *PATTERN*

    Use the provided *PATTERN* to filter *archives*, *object files*, or
    *symbols*(specified by **Reporting Options**) in **table**, **csv** or **dot** formats.
    The pattern can include following wildcards:

    - **\***: Matches any sequence of characters.
    - **?**: Matches any single character.
    - **[seq]**: Matches any character in the sequence.
    - **[!seq]**: Matches any character not in the sequence.

    This option can be used multiple times, functioning as a logical *OR*.

### Comparison Options

- **--diff** *REFERENCE_MAP_FILE***

    Compare sizes with another project specified by *REFERENCE_MAP_FILE*.


### Archives' dependencies options

- **--dep-symbols**, **--dep-syms**

    Include dependency symbols for the **--archive-dependencies** option. This
    means symbols used by one archive from another. In the case of
    **--dep-reverse**, it refers to symbols used by other archives from the
    specified archive.

- **--dep-reverse**, **--dep-rev**

    Use reverse dependencies for the **--archive-dependencies** option.
    This will show the reverse dependencies of archives, instead of archives
    dependencies.

### General Options

- **-h** or **--help**

    Display brief help message and exit.

- **--doc**

    Display more comprehensive documentation.

-  **-q** or **--quiet**

    Suppress all output.

-  **-d** or **--debug**

    Print debug information. Messages are printed to stderr.

-  **-o** *OUTPUT_FILE* or **--output-file** *OUTPUT_FILE*

    Print output to the specified file instead of stdout.

-  **--no-color**

    Disable ANSI color escape sequences.

-  **--force-terminal**

    Enable terminal control codes even if out is not attached to terminal. This
    option is ignored if used along with the **--output-file** option.

-  **--lto** or **--no-lto**

    Enable or disable usage of *DWARF* debugging information to identify archives
    for symbols without archive.  Intended to be used if *LTO* is enabled. If not
    specified, detect *LTO* usage from *sdkconfig.json*, if available. Note *LTO*
    is currently not supported by ESP-IDF.


### Memory Map Dictionary

Dictionary with memory map containing metadata and a *memory_types* key that organizes
the memory details into a tree structure. Memory types are defined in the chip information
YAML file, which describes the memory partitioning and classifies all memory into these
types.

Memory type sizes are determined by memory regions, as specified in the link map file using
the MEMORY command in the linker script. These regions are divided according to memory type
definitions, enabling precise tracking of size and usage for each memory type. For instance,
the *iram0_0_seg* memory region for the esp32s3 might encompass multiple memory types defined
in esp32s3.yaml (*IRAM* and *DRAM_1*).

Each memory type includes a tree hierarchy: sections, which contain archives; archives, which
contain object files; and object files, which contain symbols. Every object in this hierarchy
holds information about its size within the memory type.

Example:
```python
{
    'version': '1.0',
    'target': 'esp32s3',
    'target_diff': '',
    'image_size': 207313,
    'image_size_diff': 0,
    'project_path': '/home/fhrbata/work/esp-idf/examples/get-started/hello_world',
    'project_path_diff': '',
    'memory_types': {
        'IRAM': {
            'size': 16384,
            'size_diff': 0,
            'used': 16383,
            'used_diff': 0,
            'sections': {
                '.iram0.text': {
                    'abbrev_name': '.text',
                    'size': 15356,
                    'size_diff': 0,
                    'archives': {
                        'esp-idf/esp_system/libesp_system.a': {
                            'abbrev_name': 'libesp_system.a',
                            'size': 3335,
                            'size_diff': 0,
                            'object_files': {
                                'cpu_start.c.obj': {
                                    'abbrev_name': 'cpu_start.c.obj',
                                    'size': 1157,
                                    'size_diff': 0,
                                    'symbols': {
                                        '.iram1.0.literal': {
                                            'abbrev_name': '.iram1.0.literal',
                                            'size': 68,
                                            'size_diff': 0},
```


The `*_diff` values are set to zero by default. However, when the memory map is compared
with another referenced memory map, these fields will contain the difference between
the memory map and referenced memory map values.


### API

The `esp-idf-size` Python module provides a simple API for loading and
comparing memory maps. For comprehensive documentation, consult the pydocs for
the `esp_idf_size` Python module. Below are some basic usage examples.

1. **Display memory type usage**

```python
    import sys
    from esp_idf_size import memorymap

    try:
        memmap = memorymap.get(sys.argv[1])
    except memorymap.MemMapException as e:
        print(e, file=sys.stderr)

    for mem_type_name, mem_type_info in memmap['memory_types'].items():
        used = mem_type_info['used']
        size = mem_type_info['size']
        print(f'{mem_type_name}: used: {used}, size: {size}')
```

2. **Display the differences in memory type usage between two projects**

```python
    import sys
    from esp_idf_size import memorymap

    try:
        memmap_cur = memorymap.get(sys.argv[1])
        memmap_ref = memorymap.get(sys.argv[2])
    except memorymap.MemMapException as e:
        print(e, file=sys.stderr)

    memmap_diff = memorymap.diff(memmap_cur, memmap_ref)
    memorymap.unify(memmap_diff)

    for mem_type_name, mem_type_info in memmap_diff['memory_types'].items():
        size_diff = mem_type_info['size_diff']
        used_diff = mem_type_info['used_diff']
        print(f'{mem_type_name}: used_diff: {used_diff}, size_diff: {size_diff}')
```

[1]: https://graphviz.org/
