# Versioned Storage and Deribit Archiver

## Storage model
The storage layer uses immutable per-run writes.

Implemented behavior:
- every write carries `run_id`, `run_ts`, `logical_ts`, and `write_mode`
- one Parquet file is written per run
- one manifest is written per run
- dataset specs define primary keys
- default reads use latest-by-primary-key dedupe
- audit/debug reads can request all runs

Main modules:
- `src/uforia/storage/datasets.py`
- `src/uforia/storage/store.py`

## Operational datasets
Two operational datasets were added:

`raw_deribit_option_snapshot`
- stores raw payload archives for collected hours

`ingestion_runs`
- stores one row per archiver attempt
- statuses: `success`, `gap`, `failed`, `skipped_existing`

## Deribit archiver
Implemented in `src/uforia/ingest/deribit/archiver.py`.

Behavior:
- collects one logical UTC hour at a time
- writes raw payloads and normalized snapshots together
- skips existing successful hours unless forced
- emits gap markers for missed hours
- never pretends to backfill unavailable historical live snapshots
- supports both BTC and ETH collectors through the same immutable dataset layout

Key commands:
- `uv run uforia-admin ingest deribit-options-once --underlying BTC|ETH --as-of 2026-03-09T12:00:00Z`
- `uv run uforia-admin ingest deribit-options-daemon --underlying BTC|ETH --poll-seconds 5 --grace-seconds 90`
- `uv run uforia-admin ingest ingestion-runs --start ... --end ...`
