# opendeviationbar-py — Project

## What This Is

Rust workspace with Python bindings for open deviation bar construction. Performance-optimized production system processing 42M+ bars across 17 symbols on bigblack, with zero-gap zero-overlap guarantees validated by continuous telemetry.

## Core Value

Zero-gap, zero-overlap Ouroboros day-mode bars across every restart, every midnight boundary, every deploy — verified by continuous telemetry.

## Requirements

### Validated

- ✓ Rust-side bar dedup prevents overlaps at source (v13.33.0)
- ✓ Sidecar overlap self-heal detects+repairs today's overlaps (v13.32.0)
- ✓ Deploy hardening: monit unmonitor, mutation cleanup, OPTIMIZE FINAL (v13.32.1)
- ✓ P0-P8: v1.2 Performance milestone (all 12 requirements)
- ✓ Atomic replace in sync_flush using store_bars_replacing() (v1.3)
- ✓ Legacy pruning: dead restart-gap code removed (v1.3)
- ✓ Clean slate DELETE today before fill_from_rest (v1.3)
- ✓ drain_bars() synchronous ring buffer drain (v1.3)
- ✓ 1000 dbps removed, 100 dbps added for BTC/ETH/SOL (v13.38.0)
- ✓ committed_floors from last completed bar, not forming bar tail — v1.4 (RUST-01, RUST-02)
- ✓ Midnight crossover trade ID verification via REST API preflight — v1.4 (PREFLIGHT-01-03)
- ✓ Heartbeat unified per-symbol Stathera continuity + LOUD alerts — v1.4 (TELEM-01-04)
- ✓ WS resilience: ping/pong keepalive, ban handling, stream signals — v1.4 (Phase 4)
- ✓ Seeder REST fallback + cross-process rate limit coordination — v1.4 (SEEDER-01-04)
- ✓ Memory efficiency: Copy AggTrade, Arrow streaming, GIL release, Python optimizations — v1.4 (MEM-01-08)
- ✓ Yesterday integrity check + signal file — v1.5 (STARTUP-01-02)
- ✓ Kintsugi yesterday priority queue — v1.5 (STARTUP-03)
- ✓ Sub-minute sidecar startup (27s measured) — v1.5 (STARTUP-04)

### Active

(None — next milestone requirements to be defined via `/gsd:new-milestone`)

## Current State

Shipped v1.5 (2026-03-23). 3 milestones complete (v1.0 → v1.4 → v1.5).
Next milestone to be defined via `/gsd:new-milestone`.

### Out of Scope

- OpenDeviationBar struct split (P9) — major refactor, separate milestone
- Full 18-symbol streaming — planned for later milestone
- Kintsugi two-day processor spans — preflight verification is simpler and sufficient
- Plugin Polars->Pandas — no plugins in production

## Key Decisions

| Decision                                   | Rationale                                       | Outcome                            |
| ------------------------------------------ | ----------------------------------------------- | ---------------------------------- |
| Clean slate DELETE today on restart        | Proven zero-gap pattern (v1.3)                  | ✓ Good — 0/0 Stathera on restart   |
| store_bars_replacing for sync_flush        | Proven kintsugi mechanism                       | ✓ Good — atomic per-chunk          |
| REST API crossover verification            | User insight — pinpoint exact midnight trade ID | ✓ Good — verified in production    |
| committed_floors from completed bar        | 6-agent spike root cause (Bug 1)                | ✓ Good — junction gaps eliminated  |
| Unified Stathera in heartbeat              | No more intra-day vs midnight classification    | ✓ Good — single per-symbol check   |
| File-based rate limit coordination         | Cross-process budget sharing without Redis      | ✓ Good — atomic JSON, stale expiry |
| Oracle golden snapshots before refactoring | Bit-exact regression baseline                   | ✓ Good — caught 0 regressions      |
| AggTrade Copy + Arrow zero-copy            | Eliminate clone overhead + heap allocation      | ✓ Good — verified bit-exact        |

## Context

Shipped v1.4 with 6 phases, 16 plans, 32 tasks across 2 days (2026-03-21 → 2026-03-23).

**Known tech debt:**

- TELEM-03: `ws_first_tid` and `committed_floors` junction telemetry sub-fields not yet populated (informational only)
- Phases 1-3 lack formal VERIFICATION.md artifacts (pre-verifier; integration-verified)
- REQUIREMENTS.md MEM traceability table text was stale (now archived)

## Constraints

- Python 3.13 only
- No Rust API changes that break PyO3 bindings
- Spot-only data sources (Binance)
- 10 hard constraints from git history (see v1.3 debug findings)

## Evolution

This document evolves at phase transitions and milestone boundaries.

**After each phase transition** (via `/gsd:transition`):

1. Requirements invalidated? → Move to Out of Scope with reason
2. Requirements validated? → Move to Validated with phase reference
3. New requirements emerged? → Add to Active
4. Decisions to log? → Add to Key Decisions
5. "What This Is" still accurate? → Update if drifted

**After each milestone** (via `/gsd:complete-milestone`):

1. Full review of all sections
2. Core Value check — still the right priority?
3. Audit Out of Scope — reasons still valid?
4. Update Context with current state

---

_Last updated: 2026-03-23 after v1.4 milestone completion_
