# Template and Placeholder System

> **Relevant source files**
> * [ArWikiCats/translations_formats/DataModel/__init__.py](../ArWikiCats/translations_formats/DataModel/__init__.py)
> * [ArWikiCats/translations_formats/DataModel/model_data.py](../ArWikiCats/translations_formats/DataModel/model_data.py)
> * [ArWikiCats/translations_formats/DataModel/model_data_base.py](../ArWikiCats/translations_formats/DataModel/model_data_base.py)
> * [ArWikiCats/translations_formats/DataModel/model_data_time.py](../ArWikiCats/translations_formats/DataModel/model_data_time.py)
> * [ArWikiCats/translations_formats/DataModel/model_data_v2.py](../ArWikiCats/translations_formats/DataModel/model_data_v2.py)
> * [ArWikiCats/translations_formats/DataModelDouble/model_multi_data_double.py](../ArWikiCats/translations_formats/DataModelDouble/model_multi_data_double.py)
> * [ArWikiCats/translations_formats/__init__.py](../ArWikiCats/translations_formats/__init__.py)
> * [ArWikiCats/translations_formats/data_new_model.py](../ArWikiCats/translations_formats/data_new_model.py)
> * [ArWikiCats/translations_formats/data_with_time.py](../ArWikiCats/translations_formats/data_with_time.py)
> * [ArWikiCats/translations_formats/multi_data.py](../ArWikiCats/translations_formats/multi_data.py)

This page documents the placeholder syntax, pattern compilation mechanisms, and replacement strategies used throughout the ArWikiCats formatting system. For information about the base formatter classes, see [FormatDataBase Architecture](23.FormatDataBase-Architecture.md). For usage examples with factory functions, see [Factory Functions and Usage](27.Factory-Functions-and-Usage.md)

The template and placeholder system is the core mechanism that enables pattern-based translation. It defines how English category patterns are matched and how Arabic translations are generated through systematic placeholder replacement.

## Placeholder Syntax Overview

The system uses curly-brace delimited placeholders in both template keys and template values. These placeholders serve two distinct purposes: pattern matching (in keys) and value substitution (in values)

### Placeholder Categories

The system employs several categories of placeholders:

| Category | Example Placeholders | Purpose | Usage Context |
| --- | --- | --- | --- |
| Element placeholders | `{sport}`, `{nat}`, `{country}` | Match keys in English patterns | Template keys |
| Value placeholders | `{sport_label}`, `{nat_ar}`, `{demonym}` | Mark substitution points | Template values |
| Temporal placeholders | `{year1}`, `{year2}` | Match year/decade/century patterns | Time-based templates |
| Genre placeholders | `{film_key}`, `{film_ar}` | Match film/TV genres | Film category templates |
| Default placeholders | `xoxo`, `natar` | Generic pattern matching | Internal processing |

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L75-L96](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L75-L96), [ArWikiCats/translations_formats/multi_data.py L47-L48](../ArWikiCats/translations_formats/multi_data.py#L47-L48)

### Template Structure

Templates consist of two parts:

```yaml
{key_pattern} → {value_pattern}

Example:
"{sport} players" → "لاعبو {sport_label}"
```

The key pattern contains element placeholders that match English category components. The value pattern contains value placeholders that are replaced with Arabic translations.

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data.py L14-L26](../ArWikiCats/translations_formats/DataModel/model_data.py#L14-L26)

## Pattern Compilation Process

The pattern compilation process transforms data dictionaries into compiled regular expressions for efficient matching.

```mermaid
flowchart TD

DataList["data_list<br/>{'football': 'كرة القدم',<br/>'basketball': 'كرة السلة'}"]
KeyPlaceholder["key_placeholder<br/>'{sport}'"]
CreateAlt["create_alternation()<br/>Sort keys by:<br/>1. Space count (descending)<br/>2. Length (descending)"]
BuildRegex["keys_to_pattern()<br/>Build regex pattern"]
Alternation["alternation<br/>'basketball|football'"]
Pattern["pattern<br/>(?<!\w)(basketball|football)(?!\w)"]
InputCat["Category:<br/>'football players'"]
MatchKey["match_key()<br/>Search for pattern"]
Result["Matched key:<br/>'football'"]

DataList --> CreateAlt
KeyPlaceholder --> BuildRegex
Pattern --> MatchKey

subgraph Matching ["Category Matching"]
    InputCat
    MatchKey
    Result
    InputCat --> MatchKey
    MatchKey --> Result
end

subgraph Compilation ["Pattern Compilation"]
    CreateAlt
    BuildRegex
    Alternation
    Pattern
    CreateAlt --> Alternation
    Alternation --> BuildRegex
    BuildRegex --> Pattern
end

subgraph Input ["Input Data"]
    DataList
    KeyPlaceholder
end
```

**Diagram: Pattern Compilation Flow from Data to Regex**

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L106-L133](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L106-L133)

### Key Sorting Strategy

The `create_alternation()` method sorts keys using a specific strategy to prevent matching errors:

```
keys_sorted = sorted(self.data_list_ci.keys(), key=lambda k: (-k.count(" "), -len(k)))
```

This ensures that "black-and-white" is matched before "black", preventing incorrect partial matches.

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L115](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L115)

### Regex Pattern Construction

The `keys_to_pattern()` method constructs a word-boundary-aware regex pattern:

```
Pattern format: (?<!{regex_filter})({alternation})(?!{regex_filter})

Default regex_filter: \w
Default pattern: (?<!\w)(basketball|football)(?!\w)
```

The negative lookbehind `(?<!...)` and negative lookahead `(?!...)` ensure that keys are matched as complete tokens, not as substrings.

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L119-L133](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L119-L133)

## Placeholder Normalization

Before template lookup, categories are normalized by replacing matched keys with their placeholders.

```mermaid
flowchart TD

Input["Input:<br/>'football players'"]
Replace["Regex Substitution"]
MatchedKey["Matched Key:<br/>'football'"]
Normalized["Normalized:<br/>'{sport} players'"]
HandleTexts["handle_texts_before_after()"]
Final["Final:<br/>'{sport} players'"]

subgraph Normalization ["normalize_category() Process"]
    Input
    Replace
    MatchedKey
    Normalized
    HandleTexts
    Final
    Input --> Replace
    MatchedKey --> Replace
    Replace --> Normalized
    Normalized --> HandleTexts
    HandleTexts --> Final
end
```

**Diagram: Category Normalization Process**

The normalization process uses regex substitution to replace the first occurrence of the matched key:

```
normalized = re.sub(
    rf"(?<!{self.regex_filter}){re.escape(sport_key)}(?!{self.regex_filter})",
    f"{self.key_placeholder}",
    f" {normalized_category.strip()} ",
    flags=re.IGNORECASE,
    count=1,
)
```

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L193-L216](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L193-L216)

### Text Before/After Handling

The `handle_texts_before_after()` method removes configured prefix/suffix text surrounding placeholders:

```
if self.text_before:
    if f"{self.text_before}{self.key_placeholder}" in normalized:
        normalized = normalized.replace(f"{self.text_before}{self.key_placeholder}", self.key_placeholder)

if self.text_after:
    if f"{self.key_placeholder}{self.text_after}" in normalized:
        normalized = normalized.replace(f"{self.key_placeholder}{self.text_after}", self.key_placeholder)
```

This handles cases where the nationality data includes "the " prefix (e.g., "the British actors" → "{nat_en} actors")

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L158-L191](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L158-L191)

## Replacement Strategies

The system implements three distinct replacement strategies depending on the formatter type.

```mermaid
flowchart TD

Simple["FormatData<br/>Simple String Replacement"]
Dict["FormatDataV2<br/>Dictionary-based Replacement"]
Temporal["YearFormatData<br/>Callback-based Replacement"]
SimpleTmpl["Template:<br/>'لاعبو {sport_label}'"]
SimpleVal["Value:<br/>'كرة القدم'"]
SimpleReplace["template.replace(placeholder, value)"]
SimpleResult["Result:<br/>'لاعبو كرة القدم'"]
DictTmpl["Template:<br/>'{demonym} كتاب'"]
DictVal["Value:<br/>{'demonym': 'يمنيون'}"]
DictLoop["For each key, val in dict:<br/>template.replace('{key}', val)"]
DictResult["Result:<br/>'يمنيون كتاب'"]
CallbackTmpl["Template:<br/>'كتاب في {year1}'"]
CallbackVal["Callback:<br/>convert_time_to_arabic('14th-century')"]
CallbackResult["Result:<br/>'كتاب في القرن 14'"]

Simple --> SimpleTmpl
Dict --> DictTmpl
Temporal --> CallbackTmpl

subgraph CallbackImpl ["Callback Replacement"]
    CallbackTmpl
    CallbackVal
    CallbackResult
    CallbackTmpl --> CallbackVal
    CallbackVal --> CallbackResult
end

subgraph DictImpl ["Dictionary Replacement"]
    DictTmpl
    DictVal
    DictLoop
    DictResult
    DictTmpl --> DictVal
    DictVal --> DictLoop
    DictLoop --> DictResult
end

subgraph SimpleImpl ["Simple Replacement"]
    SimpleTmpl
    SimpleVal
    SimpleReplace
    SimpleResult
    SimpleTmpl --> SimpleVal
    SimpleVal --> SimpleReplace
    SimpleReplace --> SimpleResult
end

subgraph Strategies ["Replacement Strategies"]
    Simple
    Dict
    Temporal
end
```

**Diagram: Three Replacement Strategy Implementations**

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data.py L100-L116](../ArWikiCats/translations_formats/DataModel/model_data.py#L100-L116), [ArWikiCats/translations_formats/DataModel/model_data_v2.py L81-L121](../ArWikiCats/translations_formats/DataModel/model_data_v2.py#L81-L121)

### FormatData: Simple String Replacement

The `FormatData` class implements the simplest replacement strategy:

```python
def apply_pattern_replacement(self, template_label: str, sport_label: str) -> str:
    final_label = template_label.replace(self.value_placeholder, sport_label)

    if self.value_placeholder not in final_label:
        return final_label.strip()

    return ""
```

This replaces all occurrences of `value_placeholder` with the Arabic label string.

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data.py L100-L116](../ArWikiCats/translations_formats/DataModel/model_data.py#L100-L116)

### FormatDataV2: Dictionary-based Replacement

The `FormatDataV2` class supports dictionary values with multiple placeholders:

```python
def apply_pattern_replacement(self, template_label: str, sport_label: Union[str, Dict[str, str]]) -> str:
    if not isinstance(sport_label, dict):
        return template_label

    final_label = template_label

    if isinstance(sport_label, dict):
        for key, val in sport_label.items():
            if isinstance(val, str) and val:
                final_label = final_label.replace(f"{{{key}}}", val)

    return final_label.strip()
```

Each key-value pair in the dictionary replaces a corresponding placeholder in the template.

Example:

```css
data_list = {"yemen": {"demonym": "يمنيون", "country_ar": "اليمن"}}
template = "{demonym} كتاب من {country_ar}"
# Result: "يمنيون كتاب من اليمن"
```

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_v2.py L81-L100](../ArWikiCats/translations_formats/DataModel/model_data_v2.py#L81-L100)

### FormatDataFrom: Callback-based Replacement

The `FormatDataFrom` class (used by `YearFormatData`) uses callback functions for dynamic replacement:

```markdown
# YearFormatData creates a FormatDataFrom with callbacks
return FormatDataFrom(
    formatted_data={},
    key_placeholder=key_placeholder,
    value_placeholder=value_placeholder,
    search_callback=convert_time_to_arabic,
    match_key_callback=match_time_en_first,
    fixing_callback=standardize_time_phrases,
)
```

The callbacks handle temporal pattern conversion (e.g., "14th-century" → "القرن 14")

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_time.py L34-L66](../ArWikiCats/translations_formats/DataModel/model_data_time.py#L34-L66)

## Placeholder Naming Conventions

The codebase follows specific naming conventions for placeholders to maintain consistency across different formatter types.

### Naming Patterns

| Pattern Type | Key Placeholder | Value Placeholder | Example |
| --- | --- | --- | --- |
| Sports | `{sport}` | `{sport_label}`, `{sport_ar}` | Football → كرة القدم |
| Nationalities | `{nat}`, `{nat_en}` | `{nat_ar}`, `{demonym}` | British → بريطانيون |
| Countries | `{country}`, `{country1}` | `{country_ar}`, `{demonym}` | Yemen → اليمن |
| Years | `{year1}`, `{year2}` | `{year1}`, `{year2}` | 14th-century → القرن 14 |
| Films | `{film_key}` | `{film_ar}` | Action → أكشن |

**Sources:** [ArWikiCats/translations_formats/multi_data.py L47-L48](../ArWikiCats/translations_formats/multi_data.py#L47-L48), [ArWikiCats/translations_formats/data_with_time.py L41-L42](../ArWikiCats/translations_formats/data_with_time.py#L41-L42)

### Default Placeholders

Two special default placeholders are used internally:

```
YEAR_PARAM = "xoxo"
COUNTRY_PARAM = "natar"
```

These serve as fallback values when placeholders are not explicitly specified. The choice of unusual strings ("xoxo", "natar") minimizes the risk of collision with actual category text.

**Sources:** [ArWikiCats/translations_formats/multi_data.py L47-L48](../ArWikiCats/translations_formats/multi_data.py#L47-L48)

## Complete Translation Flow

The following diagram shows how placeholders flow through the entire translation process:

```mermaid
flowchart TD

Cat["Category:<br/>'british football players'"]
Templates["formatted_data:<br/>{'{nat} {sport} players':<br/>'لاعبو {sport_ar} {nat_ar}'}"]
Data1["data_list (nationalities):<br/>{'british': 'بريطانيون'}"]
Data2["data_list2 (sports):<br/>{'football': 'كرة القدم'}"]
Match1["match_key()<br/>Pattern: (?"]
Key1["Matched: 'british'"]
Norm1["normalize_category()<br/>'british football players'<br/>→ '{nat} football players'"]
Match2["match_key()<br/>Pattern: (?"]
Key2["Matched: 'football'"]
Norm2["normalize_category()<br/>'{nat} football players'<br/>→ '{nat} {sport} players'"]
Lookup["get_template_ar()<br/>Key: '{nat} {sport} players'"]
Template["Template:<br/>'لاعبو {sport_ar} {nat_ar}'"]
Replace1["Replace {sport_ar}<br/>with 'كرة القدم'"]
Replace2["Replace {nat_ar}<br/>with 'بريطانيون'"]
Final["Result:<br/>'لاعبو كرة القدم بريطانيون'"]

Cat --> Match1
Data1 --> Match1
Norm1 --> Match2
Data2 --> Match2
Norm2 --> Lookup
Templates --> Lookup
Template --> Replace1
Data2 --> Replace1
Data1 --> Replace2

subgraph Step4 ["Step 4: Placeholder Replacement"]
    Replace1
    Replace2
    Final
    Replace1 --> Replace2
    Replace2 --> Final
end

subgraph Step3 ["Step 3: Template Lookup"]
    Lookup
    Template
    Lookup --> Template
end

subgraph Step2 ["Step 2: Match Second Element"]
    Match2
    Key2
    Norm2
    Match2 --> Key2
    Key2 --> Norm2
end

subgraph Step1 ["Step 1: Match First Element"]
    Match1
    Key1
    Norm1
    Match1 --> Key1
    Key1 --> Norm1
end

subgraph Input ["Input"]
    Cat
    Templates
    Data1
    Data2
end
```

**Diagram: Complete Placeholder Flow for Dual-Element Translation**

**Sources:** [ArWikiCats/translations_formats/multi_data.py L96-L197](../ArWikiCats/translations_formats/multi_data.py#L96-L197)

## Placeholder Validation

The system includes validation to ensure all placeholders are properly replaced before returning results.

```python
def check_placeholders(self, category: str, result: str) -> str:
    if "{" in result:
        logger.warning(f">>> Found unprocessed placeholders in {category=}: {result=}")
        return ""
    return result
```

Any result containing a literal `{` character indicates an unprocessed placeholder, which triggers a warning and returns an empty string to prevent invalid translations.

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L371-L387](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L371-L387)

## Edge Cases and Special Handling

### Case Sensitivity

All placeholder matching is case-insensitive:

```
self.formatted_data_ci: Dict[str, str] = {k.lower(): v for k, v in formatted_data.items()}
self.data_list_ci: Dict[str, Any] = {k.lower(): v for k, v in data_list.items()}
```

This allows "British", "british", and "BRITISH" to match the same template.

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L93-L94](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L93-L94)

### Multi-word Keys

Multi-word keys like "black-and-white" are prioritized over single-word keys through the sorting strategy:

```
keys_sorted = sorted(self.data_list_ci.keys(), key=lambda k: (-k.count(" "), -len(k)))
```

This prevents "black" from matching when "black-and-white" is the intended key.

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L115](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L115)

### Arabic Category Prefix

The system handles the "category:" prefix specially:

```python
def prepend_arabic_category_prefix(self, category, result) -> str:
    if result and category.lower().startswith("category:") and not result.startswith("تصنيف:"):
        result = "تصنيف:" + result
    return result
```

When the English input starts with "category:", the Arabic output is prefixed with "تصنيف:" unless it already has that prefix.

**Sources:** [ArWikiCats/translations_formats/DataModel/model_data_base.py L339-L352](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L339-L352)

## Code Entity Reference

### Key Classes and Methods

| Class | Method | Purpose | Location |
| --- | --- | --- | --- |
| `FormatDataBase` | `create_alternation()` | Build regex alternation from keys | [model_data_base.py L106-L117](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L106-L117) |
| `FormatDataBase` | `keys_to_pattern()` | Compile regex pattern | [model_data_base.py L119-L133](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L119-L133) |
| `FormatDataBase` | `match_key()` | Find matching key in category | [model_data_base.py L135-L156](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L135-L156) |
| `FormatDataBase` | `normalize_category()` | Replace key with placeholder | [model_data_base.py L193-L216](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L193-L216) |
| `FormatDataBase` | `check_placeholders()` | Validate final result | [model_data_base.py L371-L387](../ArWikiCats/translations_formats/DataModel/model_data_base.py#L371-L387) |
| `FormatData` | `apply_pattern_replacement()` | Simple string replacement | [model_data.py L100-L116](../ArWikiCats/translations_formats/DataModel/model_data.py#L100-L116) |
| `FormatDataV2` | `apply_pattern_replacement()` | Dictionary-based replacement | [model_data_v2.py L81-L100](../ArWikiCats/translations_formats/DataModel/model_data_v2.py#L81-L100) |
| `YearFormatData` | Factory function | Create temporal formatter | [model_data_time.py L34-L66](../ArWikiCats/translations_formats/DataModel/model_data_time.py#L34-L66) |

**Sources:** Multiple files as referenced in the table.
