# Normalized Data Model Approach

As already mentioned before in the nomenclatures, the normalized format is the more intricate of
two possible versions. Putting this more intricate version into a JSON-Schema
model requires deeply nested objects, which is the main drawback.

## Example ContextEntity Schema

The mentioned temperature sensor as ContextEntity might look something like this.

```json
{
  "id": "sensor123",
  "type": "TemperatureSensor",
  "temperature": {
    "value": 22.5,
    "type": "Number",
    "metadata": {
      "unit": {
        "type": "Unit",
        "value": {
          "name": "degree Celsius",
          "code": "CEL",
          "description": "Refer ISO 80000-5 (Quantities and units — Part 5: Thermodynamics)",
          "symbol": "°C",
          "conversion_factor": "1 x K"
        }
      }
    }
  },
  "manufacturer": {
    "value": "CompanyABC",
    "type": "Text"
  },
  "refLocation": {
    "value": "LivingRoom:001",
    "type": "Relationship"
  }
}
```

Bringing this into a normalized JSON-Schema, requires a little more work
in advance, but makes it very easy to validate data at a later point
against the schema, making it usually worth the time.

# Modelling a normalized JSON-Schema

Lets being by defining the general top level fields each JSON-Schema should have:

```json
{
  "$schema": "http://json-schema.org/schema#",
  "modelTags": "",
  "$id": "UniqueIdentifier12345678",
  "$schemaVersion": "1.0.0",
  "title": "",
  "description": "A minimal working example based on the key value representation",
  "type": "object",
  "properties": {
    ...
  }
}
```

Important fields are the `$schema` field that allows defining a specific
JSON-Schema version after the # symbol, while the `$id` field should be a
unique identifier for each schema. In the easiest case this is a uuid-4
or just the URL if the schema is available via the web. The `$schemaVersion`
field allows to handle different versions of the schema.

## Mandatory fields

Each ContextEntity on the Context Broker has some mandatory fields. Namely
`id` and `type`. These must be defined in the properties part of our
JSON-Schema above. They are defined very simply as keyvalue fields.
The `type` should be defined using the `const` keyword, so the 
JSON-Schema is locked to a specific Entity type. This is later validated 
when creating an instance.
```json
{
  "id": {
    "type": "string",
    "format": "uri"
  },
  "type": {
    "const": "TemperatureSensor",
    "type": "string"
  }
}
```

## Attributes

In the next step we can start defining our attributes of the temperature sensor, 
starting with the temperature itself. All attributes are defined on the same 
level as previously `id` and `type`. At this point the difference to the 
normalized format becomes evident, since each attribute is a whole object with 
properties `value`, `type` and possibly `metadata` in itself.

```json
{
  "temperature": {
    "description": "Temperature Measurement of the sensor in Celcius",
    "type": "object",
    "properties": {
      "value": {
        "minimum": -30,
        "maximum": 60,
        "type": "number"
      },
      "type": {
        "const": "Number",
        "type": "string"
      },
      "metadata": {...}
    }
  }
}
```
This JSON-Schema generalizes the specific sensor we have seen at the top 
into a JSON-Schema. We will take a look at possible metadata later in [Metadata](#Metadata).

Since our sensor is not just a measurement but some static information 
as well, we define this as attribute as well. In our exemplary case this 
is shows as `manufacturer` and `refLocation` where the 'ref' at the 
beginning indicates that the value is the `id` of another entity. So a Reference.
The according schema for the attributes looks like:

```json
{
  "manufacturer": {
    "description": "Manufacturer of the sensor",
    "type": "object",
    "properties": {
      "value": {
        "type": "string",
        "enum": [
          "CompanyABC",
          "CompanyXYZ"
        ]
      },
      "type": {
        "const": "Text",
        "type": "string"
      }
    }
  },
  "refLocation": {
    "description": "Manufacturer of the sensor",
    "type": "object",
    "properties": {
      "value": {
        "type": "string",
        "format": "uri"
      },
      "type": {
        "const": "Relationship",
        "type": "string"
      }
    }
  }
}
```

### Metadata
In general metadata is defined on an attribute level and for the normalized 
format it takes the form of attributes, defining `value` and `type` for each 
metadata entry of an attribute. 

The normalized data model supports two very specific metadata fields. 
1. unit
2. itoAttributeType

#### Units
The unit field is self-explanatory and the validation of is handled in 
the background by [FiLiP](https://github.com/RWTH-EBC/FiLiP) when 
creating either a `Device` or `ContextEntity`. It therefore
governs how units are to be modelled. Three different ways of defining 
units is possible:
1. Defining `unit` which is a StructuredValue with the fields `code`, `name`
and `type`. [FiLiP](https://github.com/RWTH-EBC/FiLiP). When creating the 
Object, [FiLiP](https://github.com/RWTH-EBC/FiLiP) then adds additional 
information.
2. Defining `unitCode` which is a simple three or four-letter Code defined 
in ISO 80000-5 (e.g. CEL for degree Celsius)
3. Defining `unitText` is the human-readable equivalent of the code

As an example we define the unit for temperature.
```json
{
  "unit": {
    "description": "Refer ISO 80000-5 (Quantities and units \u2014 Part 5: Thermodynamics)",
    "type": "object",
    "properties": {
      "type": {
        "const": "Unit"
      },
      "value": {
        "type": "object",
        "properties": {
          "code": {
            "const": "CEL"
          },
          "name": {
            "const": "degree Celsius"
          },
          "type": {
            "const": "unit"
          }
        }
      }
    }
  }
}
```

#### iotAttributeType
To understand why we need this metadata first we first take a look at 
differences between `Device` and `ContextEntity`. When creating an 
entity to be used with the Orion Context Broker we only 
need to distinguish between attributes and commands. In simplified terms 
attributes are written to the Context Broker, while commands are being sent 
out by the Context Broker.
We identify commands by the `type` field in JSON-Schema of the attribute.
```json
{
  "attributeName": {
    "type": "object",
    "properties": {
      "value": {...},
      "type": {
        "const": "command",
        "type": "string"
      }
    }
  }
}
```

The moment we want to create a `Device` to be used with the IoT-Agent we 
suddenly have different kinds of attributes, namely `DeviceAttribute`, 
`StaticDeviceAttribute`, `DeviceCommand`, and `LazyDeviceAttribute` to 
distinguish. In theory, we could use just `DeviceCommand` and `DeviceAttribute`,
but it would create unnecessary MQTT-Subscriptions and problems down the line.
Therefore, we somehow need to convey the information what kind of Attribute 
we want to create when creating a `Device`. To have all information 
baked into the JSON-Schema model there is the possibility to write this 
information into metadata on an attribute to attribute basis.

```json
{
  "iotAttributeType": {
    "type": "object",
    "properties": {
      "type": {
        "const": "Text"
      },
      "value": {
        "const": "DeviceCommand",
        "type": "string"
      }
    }
  }
}
```

## Working with $ref
At some point there will be quite a lot of redundant information in our JSON-Schema.
The `type` of an attribute in FIWARE is usually something like `Text`, `Number` 
or `List`. The same goes for metadata `unit` and `iotAttributeType`.  There 
is only a limited amount that is used over and over again. Fortunately 
JSON-Schema supports defining these things in its own schema and referencing it
from other schemas. This is done using the `$ref` keyword.

You can find the 
[FIWARE types](https://gitlab.fz-juelich.de/ese-ict-platform/fiware/smart-data-models/-/blob/main/BaseModels/v1/FiwareTypes/schema.json?ref_type=heads), 
[units](https://gitlab.fz-juelich.de/ese-ict-platform/fiware/smart-data-models/-/blob/main/BaseModels/v1/BaseUnits/schema.json?ref_type=heads) 
and [iotAttributeTypes](https://gitlab.fz-juelich.de/ese-ict-platform/fiware/smart-data-models/-/blob/main/BaseModels/v1/IoTAttributeTypes/schema.json?ref_type=heads) 
already defined in a 
[centralized location](https://gitlab.fz-juelich.de/ese-ict-platform/fiware/smart-data-models/).

This simplifies the definition of our attributes drastically and allows 
to make changes at one point instead of having to change all the attributes 
and schemas by hand. This can easily save dozens of lines of code per 
attribute if used correctly. A resulting simplified attribute definition 
looks as follows:

```json
{
  "temperature": {
    "description": "Temperature Measurement of the sensor in Celcius",
    "type": "object",
    "properties": {
      "value": {
        "minimum": -30,
        "maximum": 60,
        "type": "number"
      },
      "type": {
        "$ref": "https://gitlab.fz-juelich.de/ese-ict-platform/fiware/smart-data-models/-/raw/main/BaseModels/v1/FiwareTypes/schema.json?ref_type=heads#definitions/fiwareNumber"
      },
      "metadata": {
        "type": "object",
        "properties": {
          "iotAttributeType": {
            "$ref": "https://gitlab.fz-juelich.de/ese-ict-platform/fiware/smart-data-models/-/raw/main/BaseModels/v1/IoTAttributeTypes/schema.json?ref_type=heads#definitions/deviceAttribute"
          },
          "unit": {
            "$ref": "https://gitlab.fz-juelich.de/ese-ict-platform/fiware/smart-data-models/-/raw/main/BaseModels/v1/BaseUnits/schema.json?ref_type=heads#definitions/degreeCelcius"
          }
        }
      }
    }
  }
}
```
