# Node Type Specification

This documentation outlines the configuration and code generation process for Node Type using JSON. The JSON schema is defined in `/src/editor/types/ConfigTypes.ts`

There are two primary types of nodes: Exec Nodes and Value Nodes.

* Exec Nodes: Exec Nodes are packed with `exec` type inputs and outputs. They are the backbone of the program flow in our visual environment. They control the sequence of execution and dictate how the program progresses from one operation to another. for example sequence, for-loop, and function.

- Value Nodes: Value Nodes are responsible for handling data within the program. They don't have any exec type, provide values, and perform operations on data. Like, make literal data.

## Code Generation

To facilitate code generation for a node:

1. The required import statements (`externalImports`).
2. Specify the text-based programming language target for code generation (**codeGenerator**).

    The **code generator** is a typescript function that should do:

    - Translate the node's logic into executable code.

    - Attach metadata to outputs when dealing with image data.

    - Ensure the correct execution order of the subsequent code when dealing with Exec Node.

## Node Example

### Exec Node Example

The function definition is as follows:

> [torchvision.io.read_image(*path: [str]*, *mode: [ImageReadMode] = ImageReadMode.UNCHANGED*) → [Tensor]](https://pytorch.org/docs/stable/tensors.html#torch.Tensor))
>
> Reads a JPEG or PNG image into a 3 dimensional RGB or grayscale Tensor. Optionally converts the image to the desired format. The values of the output tensor are uint8 in [0, 255].
>
> Parameters:
>
> - **path** ([*str*](https://docs.python.org/3/library/stdtypes.html#str)) – path of the JPEG or PNG image.
> - **mode** ([*ImageReadMode*](https://pytorch.org/vision/master/generated/torchvision.io.ImageReadMode.html#torchvision.io.ImageReadMode)) – the read mode used for optionally converting the image. Default: `ImageReadMode.UNCHANGED`. See `ImageReadMode` class for more information on various available modes.
>
> Returns:
>
> output (Tensor[image_channels, image_height, image_width])

#### Generating a `externalImports`

In order to execute the python code generated by the following `code generator`,  the `externalImports` should be

```json
"from torchvision import io\nfrom torchvision.io import ImageReadMode",
```

#### Generating a `codeGenerator` for Python

More detail about the metadata definition, please check the [Knowledge Graph](Knowledge Graph.md).

For the purpose of demonstration, we ignore the different templates for different input `modes`

```typescript
/**
 * Generates the Python code for an exec node given its inputs and outputs.
 * The function can be adapted to handle more complex scenarios and various metadata types.
 *
 * @param inputs An array of strings representing literals or variable names e.g., '1', '"str"'. 
 * @param outputs An array of strings for variable names or subsequent Python code.
 * @param node The node for which code is being generated.
 * @param generator An object providing context and utility functions for code generation.
 *
 * @returns  A string containing the generated Python code snippet.
 */
function code(inputs, outputs, node, generator) {  
 // 1. Read an image and assign it to an output variable.
 // 2. Create a metadata dictionary and attach when dealing with image data.
 // 3. Prepare the execution sequence for the subsequent code.
  const code = `${outputs[1]} = io.read_image(${inputs[1]}, ${inputs[2]})
${outputs[1]} = {
  'value': ${outputs[1]},
  'dataType': 'torch.tensor',
  'metadata': {
    'colorChannel': 'rgb',
    'channelOrder': 'channelFirst',
    'isMiniBatched': false,
    'intensityRange': '0-255',
    'device': 'cpu'
  }
}
${outputs[0]}`;
  return code;
}

// Convert the code generation function to a string format for JSON configuration.
const codeGeneratorFunction = code; // Assign the function for conversion
const jsonFormattedString = JSON.stringify(codeGeneratorFunction.toString());
console.log(jsonFormattedString);
```

#### Specification in JSON

**Note: make sure the key in inputs and outputs is unique.**

```json
{
    "read_image": {
        "type": "read_image",
        "category": "function",
        "title": "read_image",
        "tooltip": "Reads a JPEG or PNG image into a 3 dimensional RGB or grayscale Tensor. Optionally converts the image to the desired format. The values of the output tensor are uint8 in [0, 255].",
        "externalImports": "from torchvision import io\nfrom torchvision.io import ImageReadMode",
      	"codeGenerator": "function code(inputs, outputs, node, generator) {\r\n  // Begin Python code generation\r\n  const code = `${outputs[1]} = io.read_image(${inputs[1]}, ${inputs[2]})\r\n${outputs[1]} = {\r\n  'value': ${outputs[1]},\r\n  'dataType': 'torch.tensor',\r\n  'metadata': {\r\n    'colorChannel': 'rgb',\r\n    'channelOrder': 'channelFirst',\r\n    'isMiniBatched': False,\r\n    'intensityRange': '0-255',\r\n    'device': 'cpu'\r\n  }\r\n}\r\n${outputs[0]}`;\r\n  return code;\r\n}",
        "inputs": {
            "execIn": {
                "title": "execIn",
                "tooltip": "execIn",
                "dataType": "exec",
                "showWidget": false,
                "showTitle": false
            },
            "path": {
                "title": "path",
                "dataType": "string",
                "tooltip": "path(str) - path of the JPEG or PNG image."
            },
            "mode": {
                "title": "mode",
                "dataType": "imageio.ImageReadMode",
                "default": "ImageReadMode.UNCHANGED",
                "tooltip": "mode(ImageReadMode) - The read mode used for optionally converting the image. Default: ImageReadMode.UNCHANGED."
            }
        },
        "outputs": {
            "execOut": {
                "title": "execOut",
                "tooltip": "execOut",
                "dataType": "exec",
                "showWidget": false,
                "showTitle": false
            },
            "image": {
                "title": "image",
                "dataType": "image",
                "defaultValue": {
                    "dataType": "torch.tensor"
                },
                "tooltip": "{dataType: torch.tensor, value, layout: [chw], colorMode: [rgb, grayscale], intensityRange: 0-255' device: cpu}"
            }
        }
    }
}
```

![image-20231118221949728](screenshots/exec_node.png)

### Value Node Example

We will create a node for the plus operation.

#### Generating a `externalImports`

the default plus is supported, no need to import anything.

#### Generating a `codeGenerator` for Python

```typescript
/**
 * Generates the Python code for a value node given its inputs and outputs.
 *
 * @param inputs An array of strings representing literals or variable names e.g., '1', '"str"'. 
 * @param outputs An array of strings for variable names or subsequent Python code.
 * @param node The node for which code is being generated.
 * @param generator An object providing context and utility functions for code generation.
 *
 * @returns  A string containing the generated Python code snippet.
 */
function code(inputs, outputs, node, generator) {
 // Perfom plus operation.
 // *NOT NEED deal with execution sequence for the subsequent code, so the index starts   from the 0.
  const code = `${outputs[0]} = ${inputs.filter(s => s.length > 0).join(' + ')}`;
  return code;
}

// Convert the code generation function to a string format for JSON configuration.
const codeGeneratorFunction = code; // Assign the function for conversion
const jsonFormattedString = JSON.stringify(codeGeneratorFunction.toString());
console.log(jsonFormattedString);
```

#### Specification in JSON

```json
{
  "+": {
    "type": "+",
    "category": "math",
    "title": "+",
    "tooltip": "Addition of objects",
    "enableAddNewOne": true,
    "dataType": "anyDataType",
    "codeGenerator": "function code(inputs, outputs, node, generator) {\n  return `${outputs[0]} = ${inputs.filter(s => s.length > 0).join(' + ')}`;\n}",
    "inputs": {
      "in1": {
        "dataType": "anyDataType",
        "title": "in",
        "showTitle": false
      },
      "in2": {
        "dataType": "anyDataType",
        "showTitle": false
      }
    },
    "outputs": {
      "out": {
        "dataType": "anyDataType",
        "title": "out",
        "showTitle": false
      }
    }
  }
}
```

![image-20231118222818801](screenshots/value_node.png)
