# Verification Notes — abi-evals

## Traceability Markers

### Full node ID: determinism

```
tests/test_runner.py::TestRunner::test_run_suite_deterministic
```

Runs `run_suite` twice with identical `run_id` and pinned timestamp, then
asserts `r1 == r2` — proving the eval runner is fully deterministic.

### Full node ID: timestamp exclusion

N/A — abi-evals does not produce hashed payloads. All runner tests pin
timestamps to `2025-01-01T00:00:00Z` to eliminate variance. The stability
node below serves as the second traceability anchor.

### Full node ID: stability

```
tests/test_runner.py::TestRunner::test_run_suite_stable_json
```

Asserts that JSON serialization of eval results is stable (double-serialize,
compare), confirming no non-deterministic ordering in output.

---

## Tranche History

- **Control Plane Tranche** (base HEAD `425b82b`) — determinism hardening, evidence deepening, contract enforcement

### Control Plane Tranche — Capability Advancement

**Determinism hardening** — 8 tests in `tests/test_determinism_hardening.py`

Full node ID:
```
tests/test_determinism_hardening.py::TestDeterminismHardening::test_hash_stability
```

**Evidence deepening** — 8 tests in `tests/test_evidence_deepening.py`

Full node ID:
```
tests/test_evidence_deepening.py::TestEvidenceDeepening::test_eval_result_has_required_fields
```

**Contract enforcement** — 5 tests in `tests/test_contract_enforcement.py`

Full node ID:
```
tests/test_contract_enforcement.py::TestContractEnforcement::test_top_level_keys_present
```

21 new tests total. All pass. SHA-256 hash comparison for eval result
determinism. Schema validation for tier-2 eval structures.

### Model Governance & Routing Spec v1.0 — Routing Eval Tranche

**Base HEAD**: `425b82b`

New module: `src/abi_evals/routing_eval.py` — `validate_decision_record()`, `check_deterministic_replay()`, `check_fallback_correctness()`, `evaluate_routing_quality()`.

6 new tests in `tests/test_routing_eval.py`.

Full node ID:
```
tests/test_routing_eval.py::TestRoutingEval::test_validate_decision_record
```

verify.log: EXIT_CODE=0, Results: 4 passed, 0 failed
