# Day-Mode Invariant Validation

**Parent**: [/python/opendeviationbar/CLAUDE.md](/python/opendeviationbar/CLAUDE.md)

## Purpose

Enforces the **day-mode invariant**: every open deviation bar belongs to exactly one UTC day.
`toDate(open_time_ms / 1000, 'UTC') == toDate(close_time_ms / 1000, 'UTC')` for all bars.

Violation of this invariant causes cascading Sisyphean repair cycles in kintsugi (#264, #263).

## Architecture: 3 Defense Layers

| Layer                | File             | Where It Runs                             | Fail Mode                             |
| -------------------- | ---------------- | ----------------------------------------- | ------------------------------------- |
| **Pre-write gate**   | `guards.py`      | `store_bars_batch()`, `store_bars_bulk()` | Reject bar before ClickHouse INSERT   |
| **Heartbeat oracle** | `oracle.py`      | `health_heartbeat.py` every 5 min         | Persistent Telegram alert             |
| **CLI observation**  | `observation.py` | Claude Code `/loop` or manual             | 4 atomic queries with expected values |

## Invariant Definition

- **Day-mode** (`OPENDEVIATIONBAR_OUROBOROS_MODE=day`): processor resets at UTC midnight
- **Cross-midnight bar**: `open_time_ms` and `close_time_ms` fall on different UTC days
- **All `tid_delta > 0` are real gaps**: REST and Vision produce identical deterministic day boundaries (`tid_gap=0` at every UTC midnight), so there are no "structural" day-boundary gaps to exclude

## Key Functions

| Function                                                 | Module           | Purpose                                            |
| -------------------------------------------------------- | ---------------- | -------------------------------------------------- |
| `validate_bar_day_mode(bar)`                             | `guards.py`      | Returns bool; raises on violation if `strict=True` |
| `validate_bars_day_mode(bars)`                           | `guards.py`      | Batch validation for list of bar dicts             |
| `split_trades_by_day(trades)`                            | `guards.py`      | Split trade list at UTC midnight boundaries        |
| `cross_midnight_count(client, symbol, threshold)`        | `oracle.py`      | ClickHouse query returning violation count         |
| `stathera_day_classification(client, symbol, threshold)` | `oracle.py`      | Returns (gaps, overlaps)                           |
| `run_all_observations(client)`                           | `observation.py` | Run all 4 atomic observations, return pass/fail    |

## Integration Points

- `backfill.py:_fetch_and_process_gap()` — calls `split_trades_by_day()` for day-mode reset
- `backfill.py:_process_gap_fallback_starttime()` — calls `split_trades_by_day()`
- `kintsugi.py:verify_repair()` — calls `cross_midnight_count_range()` as oracle check
- `clickhouse/bulk_operations.py:store_bars_batch()` — calls `validate_bars_day_mode()` pre-write gate
- `health_heartbeat.py` — calls `cross_midnight_count_global()` in Stathera audit section

---

## Bit-Exact Day Boundary Oracle

**Purpose**: Verify the Stathera invariant holds at UTC midnight — that `day_x.last_agg_trade_id + 1 == day_x+1.first_agg_trade_id` exactly, and that no bar crosses midnight. Run this after any kintsugi repair, code change to `backfill.py`/`kintsugi.py`, or as part of a production health audit.

### Three Invariants Checked Simultaneously

| Check                  | Column       | Pass Condition | Failure Meaning                                                    |
| ---------------------- | ------------ | -------------- | ------------------------------------------------------------------ |
| Trade ID adjacency     | `tid_delta`  | Exactly `0`    | `> 0` = gap (kintsugi repairs); `< 0` = overlap (data corruption)  |
| Close timestamp sanity | `close_sane` | `OK`           | `CROSS-MIDNIGHT` = pre-write gate failed to catch a violating bar  |
| Open timestamp sanity  | `open_sane`  | `OK`           | `CROSS-MIDNIGHT` = first bar of new day has a stale open timestamp |

### Query (run via `ssh bigblack`)

```sql
WITH
last_per_day AS (
    SELECT
        toDate(open_time_ms / 1000, 'UTC')             AS bar_day,
        argMax(last_agg_trade_id, first_agg_trade_id)  AS last_tid,
        argMax(close_time_ms,     first_agg_trade_id)  AS close_ms,
        count()                                         AS bar_count
    FROM opendeviationbar_cache.open_deviation_bars FINAL
    WHERE symbol = 'BTCUSDT'
      AND threshold_decimal_bps = 250
      AND ouroboros_mode = 'day'
    GROUP BY bar_day
    SETTINGS do_not_merge_across_partitions_select_final = 1
),
first_per_day AS (
    SELECT
        toDate(open_time_ms / 1000, 'UTC')              AS bar_day,
        argMin(first_agg_trade_id, first_agg_trade_id)  AS first_tid,
        argMin(open_time_ms,       first_agg_trade_id)  AS open_ms,
        count()                                          AS bar_count
    FROM opendeviationbar_cache.open_deviation_bars FINAL
    WHERE symbol = 'BTCUSDT'
      AND threshold_decimal_bps = 250
      AND ouroboros_mode = 'day'
    GROUP BY bar_day
    SETTINGS do_not_merge_across_partitions_select_final = 1
)
SELECT
    l.bar_day                                              AS day_x,
    l.bar_count                                            AS bars_x,
    l.last_tid                                             AS last_tid_day_x,
    toDateTime64(l.close_ms / 1000, 3, 'UTC')             AS close_utc,
    if(toDate(l.close_ms / 1000, 'UTC') = l.bar_day,
       'OK', 'CROSS-MIDNIGHT')                            AS close_sane,
    f.bar_day                                              AS day_x1,
    f.bar_count                                            AS bars_x1,
    f.first_tid                                            AS first_tid_day_x1,
    toDateTime64(f.open_ms / 1000, 3, 'UTC')              AS open_utc,
    if(toDate(f.open_ms / 1000, 'UTC') = f.bar_day,
       'OK', 'CROSS-MIDNIGHT')                            AS open_sane,
    (f.first_tid - l.last_tid - 1)                        AS tid_delta,
    multiIf(
        (f.first_tid - l.last_tid - 1) = 0
            AND toDate(l.close_ms / 1000, 'UTC') = l.bar_day
            AND toDate(f.open_ms  / 1000, 'UTC') = f.bar_day, 'PERFECT ✓',
        (f.first_tid - l.last_tid - 1) > 0,
            concat('GAP +', toString(f.first_tid - l.last_tid - 1)),
        (f.first_tid - l.last_tid - 1) < 0,
            concat('OVERLAP ', toString(f.first_tid - l.last_tid - 1)),
        'BAD-TIMESTAMP'
    ) AS verdict
FROM last_per_day l
JOIN first_per_day f ON toDate(addDays(l.bar_day, 1)) = f.bar_day
ORDER BY l.bar_day DESC
LIMIT 5
FORMAT PrettyCompact
```

### Real Production Output (bigblack, 2026-03-12, BTCUSDT@250dbps)

```
┌──────day_x─┬─bars_x─┬─last_tid_day_x─┬───────────────close_utc─┬─close_sane─┬─────day_x1─┬─bars_x1─┬─first_tid_day_x1─┬────────────────open_utc─┬─open_sane─┬─tid_delta─┬─verdict───┐
│ 2026-03-11 │    125 │     3901480937 │ 2026-03-11 23:59:59.739 │ OK         │ 2026-03-12 │      11 │       3901480938 │ 2026-03-12 00:00:00.081 │ OK        │         0 │ PERFECT ✓ │
│ 2026-03-10 │    168 │     3900165464 │ 2026-03-10 23:59:59.009 │ OK         │ 2026-03-11 │     125 │       3900165465 │ 2026-03-11 00:00:00.109 │ OK        │         0 │ PERFECT ✓ │
│ 2026-03-09 │    144 │     3898606471 │ 2026-03-09 23:59:59.698 │ OK         │ 2026-03-10 │     168 │       3898606472 │ 2026-03-10 00:00:00.116 │ OK        │         0 │ PERFECT ✓ │
│ 2026-03-08 │     95 │     3897105526 │ 2026-03-08 23:59:59.396 │ OK         │ 2026-03-09 │     144 │       3897105527 │ 2026-03-09 00:00:00.116 │ OK        │         0 │ PERFECT ✓ │
│ 2026-03-07 │     34 │     3896167900 │ 2026-03-07 23:59:59.989 │ OK         │ 2026-03-08 │     95  │       3896167901 │ 2026-03-08 00:00:00.179 │ OK        │         0 │ PERFECT ✓ │
└────────────┴────────┴────────────────┴─────────────────────────┴────────────┴────────────┴─────────┴──────────────────┴─────────────────────────┴───────────┴───────────┴───────────┘
```

**Reading the output**: On 2026-03-11 → 2026-03-12, the last aggTrade consumed by day Mar 11 was `3901480937` (at 23:59:59.739 UTC). The first aggTrade of day Mar 12 is `3901480938` — exactly +1. UTC midnight (00:00:00.000) fell between these two consecutive trades at 81ms past midnight. `tid_delta = 0` means zero missing trades, zero overlap.

### Interpretation Guide

| `tid_delta` | `close_sane`     | `open_sane`      | Verdict         | Action                                                                           |
| ----------- | ---------------- | ---------------- | --------------- | -------------------------------------------------------------------------------- |
| `0`         | `OK`             | `OK`             | `PERFECT ✓`     | Day boundary is clean                                                            |
| `> 0`       | `OK`             | `OK`             | `GAP +N`        | N aggTrades missing; kintsugi will repair via Vision ZIP                         |
| `< 0`       | any              | any              | `OVERLAP N`     | Bars from two writers share aggTrades; investigate concurrent writers            |
| `0`         | `CROSS-MIDNIGHT` | `OK`             | `BAD-TIMESTAMP` | Last bar of day X has close_time_ms past midnight; pre-write gate regression     |
| `0`         | `OK`             | `CROSS-MIDNIGHT` | `BAD-TIMESTAMP` | First bar of day X+1 has open_time_ms before midnight; cross-midnight bar leaked |
| any         | `CROSS-MIDNIGHT` | `CROSS-MIDNIGHT` | `BAD-TIMESTAMP` | Both ends violated; major regression in `split_trades_by_day()`                  |

### Why `argMax(last_agg_trade_id, first_agg_trade_id)` — Not `max(close_time_ms)`

The last bar of a day is the one with the **largest `first_agg_trade_id`** — not the latest `close_time_ms`. In low-volume periods (e.g., WIFUSDT@1000 at 03:00 UTC), multiple bars can share the same millisecond timestamp. Using `argMax(..., first_agg_trade_id)` gives the true last bar by trade position, which is what Stathera cares about.

### Why `addDays(l.bar_day, 1)` — Not `l.bar_day + 1`

ClickHouse `Date + 1` adds one day correctly, but `addDays()` is explicit and handles month/year crossings without ambiguity. `2024-12-31 → 2025-01-01` works correctly. Always use `addDays()` in date arithmetic for clarity.

### Why `FINAL` + `do_not_merge_across_partitions_select_final = 1`

The `open_deviation_bars` table is a `ReplacingMergeTree`. Without `FINAL`, unmerged duplicate parts from concurrent writers appear in the result. Without `do_not_merge_across_partitions_select_final = 1`, ClickHouse skips cross-partition dedup in `FINAL` reads (a performance trade-off that breaks correctness here since bars are partitioned by `toYYYYMM(close_time_ms)`). Both settings together guarantee deduplicated reads.

### Adapting the Query for Other Symbols/Thresholds

Change `symbol = 'BTCUSDT'` and `threshold_decimal_bps = 250` at both CTE filter sites. For all-pairs audit, remove the WHERE clause and add `symbol, threshold_decimal_bps` to the SELECT and GROUP BY:

```sql
-- All-pairs version: add to SELECT list and group by in both CTEs:
--   symbol, threshold_decimal_bps
-- Join on:
--   f.bar_day = toDate(addDays(l.bar_day, 1)) AND f.symbol = l.symbol AND f.threshold_decimal_bps = l.threshold_decimal_bps
-- Order by:
--   l.bar_day DESC, l.symbol, l.threshold_decimal_bps
```
