# Phase 1: Atomic Replace + Heal Scoping - Context

**Gathered:** 2026-03-20
**Status:** Ready for planning

<domain>
## Phase Boundary

Replace `_write_batch_with_retry()` in `_sync_flush_rest_bars()` with `store_bars_replacing()` (kintsugi's proven per-chunk atomic DELETE+INSERT). Scope `_heal_day_boundary_gaps()` to today only. Fix `_check_and_heal_overlaps()` to use `valid_pairs`.

</domain>

<decisions>
## Implementation Decisions

### Atomic Replace Mechanism

- Use `store_bars_replacing()` directly from `_sync_flush_rest_bars()` — proven kintsugi mechanism
- The lightweight DELETE uses trade-ID range overlap matching: `first_agg_trade_id <= new_last AND last_agg_trade_id >= new_first`
- Per-chunk crash-safe: no bars left deleted-without-replacement
- Redis write lock via `_skip_lock=False` (sync_flush doesn't hold the lock externally)
- Convert bar dicts to Polars DataFrame before calling (store_bars_replacing requires Polars)
- Fallback to `_write_batch_with_retry()` on exception (non-fatal, kintsugi repairs)

### Heal Scoping

- Add `AND toDate(open_time_ms/1000) = today()` to `_heal_day_boundary_gaps()` query
- Historical gaps are kintsugi's domain — heal should only patch today's sidecar gaps
- `_check_and_heal_overlaps()` call site should use `valid_pairs` not raw cartesian product

### Claude's Discretion

- Error handling details in the atomic replace path
- Log message formatting

</decisions>

<code_context>

## Existing Code Insights

### Reusable Assets

- `python/opendeviationbar/clickhouse/maintenance.py:398` — `_do_replace()` with the exact lightweight DELETE SQL
- `python/opendeviationbar/clickhouse/maintenance.py` — `store_bars_replacing()` function (full signature known from kintsugi-pattern agent)
- `python/opendeviationbar/clickhouse/cache.py` — `store_bars_batch(overlap_strategy="replace")` routes to `store_bars_replacing()`
- `python/opendeviationbar/sidecar.py:762-824` — current `_sync_flush_rest_bars()` (the function to modify)
- `python/opendeviationbar/sidecar.py:618-705` — current `_heal_day_boundary_gaps()` (needs today-only scoping)

### Established Patterns

- Kintsugi calls `store_bars_batch(overlap_strategy="replace", _skip_lock=True)` — lock already held
- `_sync_flush_rest_bars()` should call with `_skip_lock=False` — let store_bars_replacing acquire its own lock
- Bar dicts from `engine.next_bar()` include `_symbol` and `_threshold` metadata (popped before processing)
- Polars DataFrame creation: `pl.DataFrame(bar_dicts)` — same pattern as `_write_bars_batch_to_clickhouse()`

### Integration Points

- `_sync_flush_rest_bars()` → `OpenDeviationBarCache.store_bars_replacing()` (new call)
- `_heal_day_boundary_gaps()` → ClickHouse query (add date filter)
- `_check_and_heal_overlaps()` → call site in main loop (pass `valid_pairs`)

</code_context>

<specifics>
## Specific Ideas

### 10 Hard Constraints from Git History (MUST satisfy all)

1. No DELETE of today's bars on startup (data vacuum) — store_bars_replacing is per-range atomic, no vacuum
2. No ALTER TABLE DELETE in hot write path — sync_flush is one-time at startup
3. No backfill_gap() on live engine for old gaps — not used here
4. No full-day REST reconstruction — not used here (fill_from_rest is chunked, from CH seed)
5. No individual bar deletion cascade — store_bars_replacing uses range matching
6. Stale checkpoints guarded by TID comparison — already in place
7. fill_from_rest uses same processor as WS — already in place
8. REST fetch chunked ≤50K — already in place
9. REST bars flushed to CH before heal queries — sync_flush ensures this
10. Kintsugi today-guard absolute — not violated (sidecar writes today)

</specifics>

<deferred>
## Deferred Ideas

None — discussion stayed within phase scope.

</deferred>
