---
name: integrate-dot-metrics
description: Integrates the dot-metrics package into an existing project that already has metrics implemented. Use when the user wants to migrate ad-hoc metric functions, quality scores, constraint checks, or evaluation logic to the dot-metrics framework. Covers identifying existing metrics, defining input types, registering metrics and constraints, wiring batch evaluation, and replacing call sites.
---

# Skill: Integrate dot-metrics into an existing project

## Description

Step-by-step guide to migrate an existing metrics implementation to the `dot-metrics` framework. Covers identifying existing metrics, defining an input type, registering metrics and constraints, wiring batch evaluation, and replacing call sites.

## When to use

- A project already computes metrics (quality scores, constraint checks, KPIs, evaluation functions) with ad-hoc code
- You want structured, typed, batch-capable metric evaluation via `dot-metrics`

## Prerequisites

- Python 3.12+
- The project must already have some form of metrics or evaluation logic

---

## Step-by-step integration

### 1. Install dot-metrics

```bash
uv pip install dot-metrics
# or in pyproject.toml dependencies:
# "dot-metrics>=0.1.0"
```

### 2. Identify the existing metrics landscape

Scan the codebase for:
- Functions that compute numeric scores, rates, counts, or pass/fail checks
- A "compute all metrics" orchestrator function (often returns a list or dict of results)
- The data types those functions receive as input (the "context" or "input" objects)

Typical patterns you'll find:

```python
# Pattern A: Functions returning custom Metric objects
def coverage_score(context, response) -> Metric:
    value = preserved / total * 100
    return Metric(name="coverage_score", value=value, type=MetricType.PERCENTAGE, unit="%", ...)

# Pattern B: Functions returning raw floats
def compute_fulfillment_rate(request, schedule) -> float:
    return fulfilled / total

# Pattern C: Dict-based definitions dispatched by key
METRICS = {"coverage": {"fn": compute_coverage, "higher_is_better": True, ...}}
```

**Inventory what you find.** For each metric, note:
- Its **key/name** (e.g. `"coverage_score"`)
- Whether it's a **metric** (continuous value) or a **constraint** (pass/fail against a threshold)
- Its **unit**, **direction** (`higher_is_better`), and any **threshold** for constraints
- Whether the function returns **debug info** alongside the value

### 3. Define the metric input type

dot-metrics metric functions take **exactly one argument**. If your existing functions take multiple arguments (e.g. `context` and `response`), bundle them into a single object:

```python
from dataclasses import dataclass

@dataclass
class EvalInput:
    context: Schedule
    response: RecommendationResponse
```

Or use a Pydantic model, a TypedDict, a plain dict, a NamedTuple -- anything works. The key is: **one argument in, one number out**.

> **Favour a type that can represent different parametric scenarios.** If you foresee running the same metric set on multiple batches (e.g. different schedules, different parameter configs), a single bundled input type makes `compute_batch()` natural.

### 4. Create the MetricSet

Create a dedicated module (e.g. `metrics/metric_set.py` or `evaluation/metrics.py`):

```python
from dot_metrics import MetricSet, ComputedValue

metric_set: MetricSet[EvalInput] = MetricSet()
```

The generic annotation `MetricSet[EvalInput]` is optional but gives you static type checking on all registered functions.

### 5. Register metrics

Translate each existing metric function. Two styles are available:

#### Decorator style (preferred for standalone metric modules)

```python
@metric_set.metric("coverage_score", unit="%", description="Percentage of initial items preserved")
def coverage_score(data: EvalInput):
    """Percentage of initial items preserved in the final output.

    Range: 0-100
    Interpretation:
        - 90-100: Excellent retention
        - <70: Significant loss
    """
    initial = len(data.context.items)
    preserved = sum(1 for u in data.response.updates if u.action != Action.DELETE)
    return (preserved / initial) * 100 if initial else 100.0
```

#### Imperative style (preferred when migrating many metrics in bulk)

```python
metric_set.add(
    "coverage_score",
    lambda data: (
        sum(1 for u in data.response.updates if u.action != Action.DELETE)
        / len(data.context.items) * 100
        if data.context.items else 100.0
    ),
    unit="%",
    description="Percentage of initial items preserved",
)
```

#### Attaching debug info

If a metric currently returns debug data (e.g. list of violations, IDs of affected items), return a `ComputedValue`:

```python
@metric_set.metric("scheduling_rate")
def scheduling_rate(data: EvalInput):
    scheduled = [e for e in data.response.entries if e.scheduled]
    unscheduled_ids = [e.id for e in data.response.entries if not e.scheduled]
    return ComputedValue(
        value=len(scheduled) / len(data.context.appointments),
        debug={"unscheduled": unscheduled_ids},
    )
```

### 6. Register constraints

Constraints are metrics with a **threshold**. A constraint passes when `value <= threshold`.

```python
@metric_set.constraint("constraint_violations", threshold=0, unit="violations")
def constraint_violations(data: EvalInput):
    """Total count of all constraint violations.

    Range: 0+
    Interpretation:
        - 0: All constraints satisfied
        - >0: Schedule has violations that need attention
    """
    return (
        count_double_bookings(data)
        + count_availability_violations(data)
        + count_schedule_violations(data)
    )
```

The sub-counts (`count_double_bookings`, etc.) can be helper functions, or they can themselves be registered as separate constraints if you want individual pass/fail tracking:

```python
@metric_set.constraint("double_bookings", threshold=0, unit="violations")
def double_bookings(data: EvalInput):
    overlaps = find_overlapping_slots(data)
    return ComputedValue(value=len(overlaps), debug={"overlaps": overlaps})
```

### 7. Replace the orchestrator

**Before** (typical pattern):

```python
def compute_metrics(context, response) -> list[Metric]:
    metrics = []
    metrics.append(coverage_score(context, response))
    metrics.append(constraint_violations(context, response))
    metrics.extend(recommendation_counts(context, response))
    return metrics
```

**After** (single compute call):

```python
from myproject.metrics.metric_set import metric_set, EvalInput

def compute_metrics(context, response):
    return metric_set.compute(EvalInput(context=context, response=response))
```

The returned `EvalResult` gives you:
- `result.score("coverage_score")` -- float value
- `result.metrics["coverage_score"].debug` -- debug dict
- `result.constraints_ok` -- all constraints passed?
- `result.violations` -- list of failed constraints
- `result.assert_constraints()` -- raise if any failed

### 8. Wire up batch evaluation

If your code runs metrics on multiple scenarios (benchmark runs, parameter sweeps, A/B comparisons), use `compute_batch()`:

```python
# Dict-keyed batch (recommended -- keys become result labels)
inputs = {
    "baseline": EvalInput(context=ctx, response=baseline_response),
    "optimized_v1": EvalInput(context=ctx, response=v1_response),
    "optimized_v2": EvalInput(context=ctx, response=v2_response),
}
batch = metric_set.compute_batch(inputs)

batch["baseline"].score("coverage_score")       # single value
batch.scores("coverage_score")                  # {"baseline": 95.0, "optimized_v1": 97.2, ...}

# List batch (indexed by position)
batch = metric_set.compute_batch([input1, input2, input3])
batch[0].score("coverage_score")
```

**Even if you only have single-input evaluation today, prefer structuring your code to support batch.** Future visualization features (3D parametric graphs, comparative charts) will consume `BatchResult` natively.

### 9. Update call sites

Find all places that currently call the old orchestrator or access the old metric objects. Common patterns to search for:

```
# Search for old call sites
grep -r "compute_metrics" --include="*.py"
grep -r "\.value" --include="*.py" | grep -i metric
```

**Before:**
```python
metrics = compute_metrics(context, response)
coverage = next(m for m in metrics if m.name == "coverage_score")
print(f"Coverage: {coverage.value}%")
violations = [m for m in metrics if m.name == "constraint_violations"]
```

**After:**
```python
result = compute_metrics(context, response)
print(f"Coverage: {result.score('coverage_score')}%")
print(f"Constraints OK: {result.constraints_ok}")
for v in result.violations:
    print(f"  FAILED: {v.key} = {v.value} (threshold: {v.threshold})")
```

### 10. Terminal visualization

```python
from dot_metrics import draw_terminal_chart

result = metric_set.compute(data)
print(draw_terminal_chart(result))
```

For metrics where lower is better (e.g. latency, violation counts), set `higher_is_better=False` during registration. For absolute metrics (unbounded values like time-in-ms), add `metadata={"absolute": True}` -- the chart will normalize them differently.

### 11. Clean up

Once all call sites are migrated:

- Delete the old `Metric` / `MetricType` model classes (if they were project-specific)
- Delete the old orchestrator function
- Delete any manual normalization / chart code that dot-metrics now handles
- Remove metric-related utility functions that are now inlined in metric registrations

---

## File organization recommendations

```
your_project/
  metrics/
    __init__.py          # exports metric_set and EvalInput
    metric_set.py        # MetricSet definition + EvalInput dataclass
    quality.py           # @metric_set.metric / .constraint registrations
    performance.py       # more registrations
```

Keep all registrations as module-level decorators. They execute at import time, so importing the module registers the metrics. Your `__init__.py` just needs:

```python
from your_project.metrics.metric_set import metric_set, EvalInput

# Import modules to trigger registration
import your_project.metrics.quality
import your_project.metrics.performance
```

---

## Migration checklist

- [ ] `dot-metrics` added to dependencies
- [ ] Input type defined (single object bundling all metric inputs)
- [ ] `MetricSet` created with generic annotation
- [ ] All metrics registered (decorator or imperative)
- [ ] All constraints registered with thresholds
- [ ] Debug info attached via `ComputedValue` where needed
- [ ] Docstrings added for metric documentation (Range, Interpretation, Notes)
- [ ] Orchestrator replaced with `metric_set.compute()`
- [ ] Batch evaluation wired where applicable
- [ ] Call sites updated to use `EvalResult` API
- [ ] Old metric models and utilities removed
- [ ] `higher_is_better` and `metadata={"absolute": True}` set correctly for visualization

---

## Common gotchas

**Metric functions must take exactly one argument.** If your existing functions take `(context, response)`, you must bundle them. Don't try to use `functools.partial` or closures to work around this -- define a proper input type.

**Constraints pass when `value <= threshold`.** This is a "lower is better" model for violations. If you have a constraint like "availability must be >= 95%", invert it: register the shortfall (`100 - availability`) with `threshold=5`.

**Don't duplicate computation.** If multiple metrics share expensive computation (e.g. building a conflict graph), extract it into a helper that takes the input type and cache or precompute as needed. The metric functions can then call the shared helper.

**Batch inputs must all be the same type.** `compute_batch()` runs the same metric set on each input. If you need different metric sets for different scenarios, use separate `MetricSet` instances.
