# Source Validation - Evolution Log

Reverse chronological (newest first).

## 2026-02-25: fromId wired into opendeviationbar-providers (full stack)

- **Rust**: Added `HistoricalError::RateLimited`, `From<RestAggTrade>` (uses `normalize_timestamp`), `send_with_rate_limit_backoff()` (1s→16s, 5 retries), `fetch_aggtrades_by_id(from_id, limit)`, `fetch_latest_aggtrade()`
- **Retrofit**: Existing `fetch_aggtrades_rest()` now uses 429 backoff + `From` trait (was inline conversion with raw `* 1000`)
- **PyO3**: Two new bindings in `binance_bindings.rs`, registered in `lib.rs`
- **Python**: `_fetch_aggtrades_by_id()` and `_fetch_latest_aggtrade()` wrappers in `backfill.py`
- **Tests**: 3 new unit tests (From impl, CSV consistency, backoff constants), 41/41 pass
- **Integration tests**: `fromid_integration.rs` (8 tests) + `rest_ws_junction_integration.rs` (3 tests) with NDJSON telemetry
- **Skill**: Added "REST→WS Reliability (3 Failure Modes)" section, "Production Resilience Summary" table, updated Template D (15-step), new Template F (10-step junction verification)
- **Status**: `startTime` callers still exist (backward compatible); `fromId` is the new correct path

## 2026-02-25: Field cross-validation added

- Added Phase 4 field-level cross-validation: overlapping trades between REST bridge and WebSocket compared field-by-field (price, volume, timestamp, is_buyer_maker)
- Result: 42/42 identical, 0 mismatches — proves REST and WS return bit-identical data for the same agg_trade_id
- Telemetry: `telemetry_20260225_235803.ndjson`

## 2026-02-25: Rewrite to fromId exact targeting

- Eliminated multi-minute pagination loop (startTime pagination)
- Replaced with 3 REST calls using fromId exact targeting
- Total validation time: 10.5s (down from >5min)
- Telemetry: `telemetry_20260225_235020.ndjson`

## 2026-02-25: Initial creation

- Created validation harness in /tmp, empirically proved:
  - Vision day-to-day contiguity (0 gaps across 3.5M trades)
  - Vision→REST junction seamless (id_delta=1)
  - REST→WS junction bridgeable (overlap dedup)
  - startTime pagination DROPS TRADES (3 gaps in 5 pages)
  - fromId pagination has ZERO gaps
- Python probes: `test_rest_pagination.py`, `test_rest_ws_junction.py`
- Migrated to canonical skill directory `.claude/skills/source-validation/`
