# YamlOOP Documentation

A comprehensive guide to using the YamlOOP library for object-oriented programming features in YAML files.

## Table of Contents

1. [Installation](#installation)
2. [Overview & Design Philosophy](#overview)
3. [Variable Features](#variable-features)
4. [Inheritance Features](#inheritance-features)
5. [Other Features](#other-features)
6. [Error Handling](#error-handling)

## Installation

```bash
pip install yaml_oop
```

## Overview

YamlOOP is a Python library that allows templating of YAML files using **inline declarations** and **depth-first search (DFS)** traversal. This is accomplished through three key features: **Variables**, **File Inheritance**, and **Declarative Rules**. These declaration rules include Object-Oriented Programming (OOP) rules such as OOP's abstract, sealed, private, etc and variable replacement rules such as default, optional, global, etc. The DFS approach allows declarative rules to persist to child elements, even across inherited files. 

YamlOOP aims for yaml readability by avoiding {{ Helm-style }} templating and in-line programming logic. YamlOOP's in-line declarations use a minimal format that always places declaration in the beginning of an element. YamlOOP eliminates templating logic in Python and and calls YAML files the same way you would pyyaml's load():

```python
import yaml
from yaml_oop import oopify, YAMLOOPException

# Define variables in Python code
injected_variables = {
    "<var1>": "value1"
}

# Process YAML with injected variables
try:
  result = oopify(
      file_path="root_config.yaml",
      directory="/path/to/config",
      Loader=yaml.SafeLoader,
      variables=injected_variables
  )
except YamlOOPException as e:
   print(f"Error: {e}")
```

Processing behavior is defined within the Yaml file via in-line declarations.

```yaml
# config.yaml
(base_config): config_path/base_config.yaml
(override) base_key1: <variable1>
---
# base_config.yaml
(abstract) base_key1:
base_key2: value2
---
# yaml_oop output
base_key1: value1
base_key2: value2
```

### Logical Flow

YamlOOP only processes valid YAML files and outputs Python types that can also be dumped as a valid YAML.

1. **Load YAML** into Python objects (dict/list/scalars) via Pyyaml
2. Convert **config-level declarations** (like `(override_file)`) into top-level key declarations
3. Run **variable substitution** using Depth-First Search repeatedly until no more variable subsitutions can occur
4. Run **base-config expansion**: whenever a node contains `(base_config)`, repeat steps 1-4 and apply inheritance rules via DFS
5. Strip declaration markers from output keys / variable maps
6. Returns a Python objects (dict/list/scalars)

---

## Declaration Syntax Rules (Important)

Declarations are inline prefixes like:

- `(override) key: value`
- `(sealed): <variable>`
- `(append) my_list: [...]`

**Spacing matters**: declarations are removed using a literal pattern like `"(override) "` (declaration + a trailing space). Always write them as:

✅ `(override) key`  
❌ `(override)key`

Multiple declarations can modify the functionality of an element

'(override) (sealed) (optional) key: <variable1>_some_string_<variable2>'

---

## Variables: `(variables)`

Variables substitute scalar string values across the YAML file.

Variable can be added as Python arguments:

```python
injected_variables = {
    "<name>": "my_service",
    "<port>": 8080
}

result = oopify(
    file_path="root_config.yaml",
    directory="/path/to/config",
    Loader=yaml.SafeLoader,
    variables=injected_variables
)
```

And/or as `(base_config)` instantiation parameters:

```yaml
(base_config):
  - (path): /path/to/config/config.yaml
  - (variables):
      <name>: my_service
      <port>: 8080
```

And/or as a `(variables)` definition in the root of your dict YAML:

```yaml
(variables):
  <name>: my_service
  <port>: 8080
```

If the root of your YAML is a list, declare `(variables)` as a list item instead:

```yaml
- (variables):
    <var1>: my_service
    <var2>: 8080
```

Then reference them anywhere in scalar (sub)string values:

```yaml
service:
  name: <name>
  port: <port>
```

It's recommended to use distinct variable tokens (e.g., `<variable>` with angle bracket) to avoid accidental substring replacement.

---

### Variable Replacement Behavior

A scalar variable can be substituted into a YAML string value:

Example:

```yaml
(variables):
  <int-var>: 1
  <string-var>: value

full_replacement: <int-var>
substring_replacement: prefix-<string-var>
multiple_substring_replacement: prefix-<string-var>-<string-var>
```

Renders to:

```yaml
full_replacement: 1
substring_replacement: prefix-value
multiple_substring_replacement: prefix-value-value
```

If a variable value is a **dict or list**, it replaces the **entire YAML node**, not a substring.

- ✅ Allowed: `key: <some_dict_or_list>`
- ❌ Not allowed: `key: "prefix-<some_dict_or_list>"`

Example:

```yaml
(variables):
  <variable_dict>:
    replaced_key1: replaced_value1
    replaced_key2:
      - replaced_item1
  <var1>: val1
  <var2>: val2

root:
  key: <variable_dict>
  key2: <var1>-string-<var2>
```

Renders to:

```yaml
root:
  key:
    replaced_key1: replaced_value1
    replaced_key2:
      - replaced_item1
  key2: val1-string-val2
```

---

### Variable Precedence

Variables are applied in this order (earlier sources have higher priority):

1. **`variables=` passed to `oopify()`** *(only for the root YAML load)*
2. **Instantiation variables** from the `(variables)` block under the `(base_config)` that loaded the current YAML
3. **`(global)` variables**
4. **The local `(variables)` block at the root of the current YAML**

Example: Instantiation variables can `(override)` the **local root `(variables)` block**.

Variable scope is limited to the YAML file that it is applied to with the exception of `(global)` and `(carryover)` variables which can persist through nested `(base_config)` calls.

---

### Global Variables: `(global)`

`(global)` variables are visible to the local YAML scope and all nested `(base_config)` YAMLs.

Example:

```yaml
# Sub YAML
(variables):
  (global) <env>: prod
config:
  (base_config):
    (path): path/to/baselevel1.yaml
    (variables):
      (global) <api>: v1
---
# Base YAML Level 1
# path/to/baselevel1.yaml
(base_config): path/to/baselevel2.yaml
---
# Base YAML Level 2
# path/to/baselevel2.yaml
api: <api>
env: <env>

```

Renders to:

```yaml
config:
  api: v1
  env: prod
```

---

### Variable Inheritance Keywords: `(override)`, `(abstract)`, `(sealed)`

These keywords modify the behavior of variables defined under `(variables)`: `(override)`, `(abstract)`, and `(sealed)`.

`(override)`: When `(override)` is applied to a `(variables)` key in a higher-priority scope, a lower-priority source with the same key will have its value overridden
`(abstract)`: When `(abstract)` is applied to a `(variable)` key in a lower-priority scope, the higher-priority source **must** implement the same key. Otherwise, `oopify()` will throw `AbstractKeyNotImplementedException`.
`(sealed)`: When a `(sealed)` variable exists in a base/template YAML, lower-priority sources may not override it. Otherwise, `oopify()` will throw `KeySealedException`.

These keywords persist to all child elements, including '(base_config)' YAMLs within those child elements.

Example:

```yaml
# Base YAML
# path/to/base.yaml
(variables):
  (abstract) <required_var>:
  (sealed) <locked_var>: base_sealed
  <shared_var>: base_value

dict:
  required: <required_var>
  locked: <locked_var>
  shared: <shared_var>
---
# Sub YAML (sub.yaml)
(base_config): 
  (path): path/to/base.yaml
  (variables):
    <required_var>: provided_by_sub
    # (override) <locked_var>: sub_locked   # Would throw KeySealedException
    (override) <shared_var>: sub_overriden     # Allowed: sub replaces base for this variable
```

Renders to:

```yaml
dict:
  required: provided_by_sub
  locked: base_sealed
  shared: sub_overriden

```

---

### Optional Variable Keyword: `(optional)`

For `(optional)` keys, if its value is not a variable reference then the key is removed. `(optional)` values must be a scalar or `oopify()` will throw `InvalidDeclarationException`.

Example:

```yaml
(variables):
  <defined>: ok

dict:
  (optional) keep_me: <defined>
  (optional) drop_me: <missing>
list:
  - (optional) <defined>
  - (optional) <missing>
```

Renders to:

```yaml
dict:
  keep_me: ok
list:
  - ok
```

---

### Default Variable Keyword: `(default)` and the delimiter ` | `

For `(default)` keys, its value must contain the delimiter ` | ` (white space on both sides of pipe). 
If the value substring to the left of the delimiter ` | ` is a variable reference, then the entire value is replaced by the variable.
If left substring is not a variable reference, then the entire value is replaced by the right substring.

Example:

```yaml
(variables):
  <val1>: replaced_val1

dict:
  (default) key1: <val1> | default_val1
  (default) key2: <val2> | default_val2
list:
  - (default) <val1> | default_val1
  - (default) <val2> | default_val2
```

Renders to:

```yaml
dict:
  key1: replaced_val1
  key2: default_val2
list:
  - replaced_val1
  - default_val2
```

The default type is parsed as a Python literal when possible (e.g., `2` becomes an int). This behavior is governed by the underlying dependency, pyyaml.

---

### Enable Variable Keyword: `(enable)`

`(enable)` keeps its parent block if its value == `True`. Otherwise, `(enable)` will delete its parent block.
When the parent block is a dict, `(enable)` keeps or deletes its parent key and the entire dict.
When the parent block is a list, and `(enable)` is the entirety of a list item, keeps or deletes the entire list.
When the parent block is a list, and `(enable)` is a key within a dict within a list item, keeps or deletes only the item.

Example:

```yaml
(variables):
  <enable_dict>: true
  <enable_list>: true
  <enable_list_item>: true

root:
  enabled_dict:
    (enable): <enable_dict>
    key1: value1
    key2: value2
  disabled_dict:
    (enable): <missing_or_false>
    key3: value3
    key4: value4
  enabled_list:
    - (enable): <enable_list>
    - item1
    - item2
  disabled_list:
    - (enable): <missing_or_false>
    - itemA
    - itemB
  list_of_dicts:
    - (enable): <enable_list_item>
      enabled_key1: value1
      enabled_key2: value2
    - (enable): <missing_or_false>
      disabled_keyA: valueA
      disabled_keyB: valueB
```

Renders to:

```yaml
root:
  enabled_dict:
    key1: value1
    key2: value2
  enabled_list:
    - item1
    - item2
  list_of_dicts:
    - enabled_key1: value1
      enabled_key2: value2
```

---

### Carryover Variables: `(carryover)`

A `(carryover)` variable is a valueless key used inside `(base_config)` variables to pull a variable from the sub YAML scope into the base YAML scope.

Example:

```yaml
# Sub YAML
(variables):
  <var>: sub_value 
(base_config):
  (path): path/to/base.yaml
  (variables):
    (carryover) <var>:
---
# Base YAML
# path/to/base.yaml
base_key: <var>
```

Renders to:

```yaml
base_key: sub_value
```

---

## Inheritance: `(base_config)`

`(base_config)` loads another YAML file and **inherits** it into the current node while preserving depth hierarchy.

You can inherit another YAML file.

```yaml
(base_config): path/to/base.yaml
```

And/or inherit another YAML file while applying variables.

```yaml
(base_config):
  (path): path/to/base.yaml
  (variables):
    <name>: service_a
    <port>: 8080
```

And/or inherit multiple YAML files.

```yaml
(base_config):
  - path/to/base1.yaml
  - path/to/base2.yaml
  - (path): path/to/base3.yaml
    (variables):
      <name>: base_name3
```

For multiple inheritance, YAML files are applied in order (top to bottom) of sequence.

Variables are typically scoped locally to each YAML file, unless declaring certain keywords like `(global)` and `(carryover)`.
That means `(variables)` in the sub YAML do not pass to base YAMLs or nested base YAMLs. 
Likewise, `(variables)` declared within `(base_config)` are scoped locally to only the base YAML. 

Example:

```yaml
# Sub YAML
(variables):
  <name>: sub_name
sub_name: <name>
sub_key: sub_value
services:
  base_service1:
    (base_config):
      (path): path/to/base.yaml
      (variables):
        <name>: service_a
        <port>: 8080
  base_service2:
    (base_config):
      (path): path/to/base.yaml
      (variables):
        <name>: service_b
        <port>: 8080
---
# Base YAML
# path/to/base.yaml
base_name: <name>
base_port: <port>
```

Renders to:

```yaml
sub_name: sub_name
sub_key: sub_value
services:
  base_service1:
    base_name: service_a
    base_port: 8080
  base_service2:
    base_name: service_b
    base_port: 8080
```

---

### Inheritance Keywords: `(override)`, `(abstract)`, `(sealed)`, `(private)`

These keywords modify the behavior of `(base_config)` inheritance: `(override)`, `(abstract)`, `(sealed)`, and `(private)`. 

`(override)`: When the same key at the same depth exists in both sub and base YAMLs, declaring `(override)` on the sub key will ensure that the sub value is used for the rendered YAML.
`(abstract)`: When an `(abstract)` key exists in the base YAML, the sub YAML must contain the same key at the same depth. Otherwise, `oopify()` will throw `AbstractKeyNotImplementedException`.
`(sealed)`: When a `(sealed)` key exists in the base YAML, a sub key at the same depth may not `(override)` it. Otherwise, `oopify()` will throw `KeySealedException`.
`(private)`: When a `(private)` key exists in the base YAML, it will be deleted before the `(base_config)` inheritance process.

These keywords persist to all child elements, including '(base_config)' YAMLs within those child elements.

Example:

```yaml
# Base YAML
(abstract) required_key:
(sealed) locked_key: base_locked
(private) internal_only: should_not_render
shared_key: base_shared
```

```yaml
# Sub YAML
(base_config): base.yaml
required_key: provided_by_sub
# (override) locked_key: # Would throw KeySealedException
(override) shared_key: sub_shared
# shared_key: sub_shared # Would throw AbstractKeyNotImplementedException
```

Renders to:

```yaml
# rendered output
required_key: provided_by_sub
locked_key: base_locked
shared_key: sub_shared
```

---

### List Inheritance Keywords: `(append)`, `(prepend)`, `(merge)`

These keywords modify the behavior of `(base_config)` inheritance when the value of the sub and base keys are sequences.

`(append)`: Appends the sub sequence behind the base sequence in the final rendered sequence.
`(prepend)`: Inserts the sub sequence before the base sequence in the final rendered sequence.
`(merge)`: Each index of the sub sequence applies `(base_config)` inheritance rules for each matching index of the base sequence.

These keywords **do not persist** to child elements and only apply to matching keys of the same depth.
If either base value or sub value is not a sequence, then `oopify()` will throw `MismatchedValueTypeException`.

Example:

```yaml
# Base YAML
append_key:
  - base_key3: data3
  - base_key4: data4
prepend_key:
  - base_key3: data3
  - base_key4: data4
merge_key:
  - base_key3: data3
  - base_key4: data4 
override_key:
  - base_key3: data3
  - base_key4: data4 
---
# Sub YAML
(base_config): path/to/base.yaml
(append) append_key:
  - sub_key1: data1
  - sub_key2: data2
(prepend) prepend_key:
  - sub_key1: data1
  - sub_key2: data2
(merge) merge_key:
  - sub_key1: data1
    (override) base_key3: overriden_data
  - sub_key2: data2 
(override) override_key:
  - sub_key1: data1
  - sub_key2: data2
```

Renders to:

```yaml
append_key:
  - base_key3: data3
  - base_key4: data4
  - sub_key1: data1
  - sub_key2: data2
prepend_key: 
  - sub_key1: data1
  - sub_key2: data2
  - base_key3: data3
  - base_key4: data4
merge_key:
  - sub_key1: data1
    base_key3: overriden_data
  - sub_key2: data2
    base_key4: data4
override_key:
  - sub_key1: data1
  - sub_key2: data2
```

---

## Other Features:

### File-wide Keywords: `(abstract_file)`, `(sealed_file)`, `(override_file)`

`File` keywords apply other keywords to the entire file.

Example:

```yaml
(abstract_file)
abstract_key1: must_be_implemented
  abstract_subkey: must_be_implemented
abstract_key2: must_be_implemented
```

Is the equivalent to:

```yaml
(abstract) abstract_key1: must_be_implemented
  abstract_subkey: must_be_implemented  # Also abstract, as abstract persists to child elements
(abstract) abstract_key2: must_be_implemented
```

---

## Exceptions

All Exceptions raised use a base exception: `YamlOOPException`.

- `NoOverrideException` tried to override without `(override)`
- `AbstractKeyNotImplementedException` sub YAML did not implement `(abstract)` key
- `KeySealedException` attempted to override `(sealed)` key/variable
- `ConflictingDeclarationException` incompatible declarations applied together
- `InvalidInstantiationException` bad `(base_config)` structure
- `CircularInheritanceException` cyclic `(base_config)` references
- `InvalidDeclarationException` declaration with incorrect key/value content or format
- `MismatchedValueTypeException` declaration with incorrect value type (i.e. using dict when a `(append)` expects a list)

---
