# DeltaTwin Manifest

## Abstract

## Status of this document

## Table of Content
- [DeltaTwin Manifest](#deltatwin-manifest)
- [Abstract](#abstract)
- [Status of this document](#status-of-this-document)
- [1. Introduction](#1-introduction)
  - [1.1 Changelog](#11-changelog)
- [2. Data model](#2-data-model)
  - [2.1 Data concepts](#21-data-concepts)
  - [2.2 Syntax](#22-syntax)
    - [2.2.1 Map](#221-map)
  - [2.3 Identifiers](#23-identifiers)
- [3. Dependency discovering](#3-dependency-mechanism)
- [4. Manifest](#4-manifest)
  - [4.1 License](#41-licence)
    - [4.1.1 Copyright](#411-copyright)
  - [4.2 Parameter](#42-parameter)
    - [4.2.2 Resource](#421-resource)
    - [4.2.1 Input](#422-input)
    - [4.2.2 InputModel](#423-inputmodel)
    - [4.2.3 Output](#424-output)
    - [4.2.3 OutputModel](#425-outputmodel)
  - [4.3 Model](#43-model)
- [5. Complete example](#5-complete-example) 

## 1. Introduction

### 1.1 Changelog
- Add `owner` section
- Removes `artifacts` section.
- Adds `inputs` section representing available inputs can be provided during the
`delta run start` command.
- Adds `outputs` section representing outputs generated by the DeltaTwin once
the `delta run start` command is completed.
- Removes `dependencies` section to manage dependencies between DeltaTwin.
- Adds `deltaVersion` to manage DeltaTwin manifest schema versions and parsers.
- Updates `models` section to support multiple model declarations.
- Renames `resources` into `internal_resources`. 

## 2. Data Model
### 2.1 Data concepts
An object is a data structure equivalent to the "object" type in *JSON*,
consisting of an unordered set of name/value pairs (referred to here as fields)
and where the name is a string and the value is a string, number, boolean,
array, or object.

### 2.2 Syntax
A DeltaTwin manifest must consist of an object represented by a *JSON* syntax.
An implementation may formally validate the structure of a DeltaTwin manifest
document using the specific schema located at
`delta/manifest/v1_1/manifest-schema.json`.

#### 2.2.1 Map
Map are used in this schema to allow a less verbose way to describe complex
types required an identifier. It also allows to avoid different object with the
same identifier.<br/>
For example:<br/>
using an array
```json
{
  "inputs": [
    {
      "name": "foo",
      "type": "int"
    },
    {
      "name": "bar",
      "type": "string"
    }
  ]
}
```
using a map
```json
{
  "inputs": {
    "foo": {
      "type": "int"
    },
    "bar": {
      "type": "string"
    }
  }
}
```

### 2.3 Identifiers
If an object contains a `name` field, that is used to uniquely identify the
object in that document. The value of the `name` field must be unique over the
entire document and matches this following pattern `[a-z][A-Za-z0-9_-]*`.

### 2.4 Type
| Symbol  | Description                                        |
|:--------|:---------------------------------------------------|
| boolean | a binary value                                     |
| integer | a integer                                          |
| number  | a decimal number                                   |
| string  | a char sequence                                    |
| array   | an array                                           |
| Data    | a Data object                                      |
| stdout  | a Data representing the standard output of a model |

### 2.5 Data
A `Data` is a data structure able to represent any data.

| field    | type   | description                                                                    |
|:---------|:-------|:-------------------------------------------------------------------------------|
| url      | string | The `Data` URL                                                                 |
| path     | string | The local host path where the `Data` is available when a DeltaTwin is executed |
| checksum | string | The hash code for validating `Data` integrity                                  |
| url      | string | The `Data` size                                                                |
| topic    | string | Topic URI from _Gael System Knowledge Base_                                    |


## 3. Dependency mechanism
Dependencies are structured following pattern <name>==<version> where: 
- name: represents the DeltaTwin name (or identifier)
- version: represents the version of the DeltaTwin

### fetching and store mechanism
- A DeltaTwin must be fetched from the dedicated API (TBD)
- A fetched DeltaTwin must be stored under the dedicated run execution storage
  - Example: DeltaTwin `foobar` version `1.0.0` must be stored in the folder
  `$RUN_EXEC_STORAGE/deltatwins/foobar/1.0.0/`

## 4. Manifest
This defines the schema of the DeltaTwin manifest document.

| field              | required | type                               | description                                                                    |
|:-------------------|:---------|:-----------------------------------|:-------------------------------------------------------------------------------|
| deltaVersion       | &empty;  | string                             | DeltaTwin manifest schema version                                              |
| name               | &check;  | string                             | Unique DeltaTwin name                                                          |
| owner              | &check;  | string                             | DeltaTwin owner                                                                |
| description        | &check;  | string                             | DeltaTwin description                                                          |
| short_description        | &empty;  | string                       | DeltaTwin short  description                                                          |
| license            | &check;  | [License](#41-license)             | DeltaTwin license                                                              |
| internal_resources | &empty;  | map<id, [Resource](#422-resource)> | Defines DeltaTwin resource parameters                                          |
| inputs             | &empty;  | map<id, [Input](#422-input)>       | Defines DeltaTwin input parameters                                             |
| outputs            | &empty;  | map<id, [Output](#424-output)>     | Defines DeltaTwin output parameters.                                           |
| models             | &empty;  | map<id, [Model](#43-model)>        | Declares models used by the DeltaTwin                                          |
| dependencies       | &empty;  | array\<string>                     | Declares dependencies to others DeltaTwin, using the DeltaTwin name or Git URL |

### 4.1 Licence
| field       | required | type                               | description                                                                                                        |
|:------------|:---------|:-----------------------------------|:-------------------------------------------------------------------------------------------------------------------|
| name        | &check;  | string                             | License name applied to the DeltaTwin                                                                              |
| description | &check;  | string                             | Description of the license applied to the Delta Twin                                                               |
| url         | &check;  | string                             | URL to the license                                                                                                 |
| copyrights  | &check;  | array<[Copyright](#441-copyright)> | Declare who owns the intellectual property of this delta twin.<br/>It includes the effective date of the ownership |

#### 4.1.1 Copyright
| field   | required | type         | description                                                                       |
|:--------|:---------|:-------------|:----------------------------------------------------------------------------------|
| company | &check;  | string       | The owner company name                                                            |
| years   | &check;  | array\<int>  | List of the effective dates of the ownership (unique value between 1970 and 2100) |

### 4.2 Parameter
A `Parameter` represents the base structure of an input or an output of a
`DeltaTwin` or a `Model`.

| field       | required | type   | description           |
|:------------|:---------|:-------|:----------------------|
| name        | &check;  | string | parameter name        |
| type        | &check;  | string | parameter type        |
| description | &empty;  | string | parameter description |


#### 4.2.1 Resource
A Resource represents a static data required by a process (Model or DeltaTwin).
A Resource inherits from [Parameter](#42-parameter):

| field | required | type                                            | description       |
|:------|:---------|:------------------------------------------------|:------------------|
| value | &check;  | boolean, integer, number, string, array, object | parameter value   |

#### 4.2.2 Input
An `Input` represents a provided data. An `Input` inherits from [Parameter](#42-parameter)

| field | required | type                                            | description     |
|:------|:---------|:------------------------------------------------|:----------------|
| value | &empty;  | boolean, integer, number, string, array, object | parameter value |

#### 4.2.3 InputModel
An `InputModel` is an `Input` containing information to build a command line.</br>
The binding behavior to build a command line depends on the data type:

- boolean: If true, add `prefix` to the command line, otherwise add nothing
- number: Add `prefix` and the number to the command line
- string: Add `prefix` and the string to the command line
- array: TBD
- object: TBD
- Data: Add `prefix` and value of `Data.path` to the command line
- null: Add nothing

| field | required | type                                            | description     |
|:------|:---------|:------------------------------------------------|:----------------|
| value | &empty;  | boolean, integer, number, string, array, object | parameter value |


#### 4.2.4 Output
An `Output` represents an output of a process (`Model` or `DeltaTwin`). An
`Output` inherits from [Parameter](#42-parameter)

| field | required | type | description |
|:------|:---------|:-----|:------------|
|       |          |      |             |

#### 4.2.5 OutputModel
An `Output` represents an output of a process (`Model` or `DeltaTwin`). An
`Output` inherits from [Parameter](#42-parameter)

| field | required | type   | description                                                |
|:------|:---------|:-------|:-----------------------------------------------------------|
| glob  | false    | string | Find files or directories relative to the output directory |

### 4.3 Model
A model represent a basic unit of computation which accepts input data,
performs some computation, and produces output data.

| field      | required | type                               | description             |
|:-----------|:---------|:-----------------------------------|:------------------------|
| name       | &check;  | string                             | model name              |
| path       | &check;  | string                             | model location          |
| type       | &check;  | string                             | model runner type       |
| parameters | &check;  | string                             | model runner parameters |
| inputs     | &empty;  | Map<string, [Input](#421-input)>   | model inputs            |
| outputs    | &empty;  | Map<string, [Output](#423-output)> | model outputs           |

## 5. Complete example

```json
{
  "name": "foobar",
  "description": "Foobar DeltaTwin description",
  "license": {
    "name": "LGPLv3",
    "url": "https://www.gnu.org/licenses/lgpl-3.0.txt",
    "copyrights": [
      { "company": "Gael Systems", "years": [2023] }
    ]
  },
  "resources": {
    "r0": {
      "type": "integer",
      "value": 42
    },
    "r1": {
      "type": "Data",
      "description": "A Sentinel-2 L1C Product",
      "value": "https://vision.odata.gael.fr/odata/v1/Products(142d244e-e2db-470c-ae87-f1cae3652882)/$value"
    }
  },
  "inputs": {
    "in_0": {
      "type": "Data",
      "description": "First DeltaTwin input"
    },
    "in_1": {
      "type": "boolean",
      "description": "Second DeltaTwin input with a default value",
      "value": false
    }
  },
  "outputs": {
    "out_0": {
      "type": "Data",
      "description": "DeltaTwin output"
    }
  },
  "models": {
    "json-formatter": {
      "path": "models/jsonFormatter",
      "type": "python",
      "parameters": {
        "image": "python:3.10",
        "requirements": ["rich"],
        "pyFile": "script.py",
        "command": "python -m json.tool $(inputs.sortedKeys) $(inputs.ident) $(inputs.infile)"
      },
      "inputs": {
        "infile": {
          "type": "Data",
          "description": "a JSON file to be validated or pretty-printed"
        },
        "ident": {
          "type": "integer",
          "description": "separate items with newlines and use this number of spaces for indentation",
          "value": 4
        },
        "sortedKeys": {
          "type": "boolean",
          "description": "sort the output of dictionaries alphabetically by key",
          "value": false
        }
      },
      "outputs": {
        "out": {
          "type": "stdout",
          "description": "formatted JSON content"
        }
      }
    }
  },
  "dependencies": [
    "another_delta_twin==1.0.0"
  ]
}
```