# Mac Mini Setup

## Goal
Run `uforia` continuously on a local Mac Mini for:
- Deribit BTC/ETH option snapshot archiving
- periodic BTC/ETH DVOL, VVIX, Razor, and options-dashboard refresh
- periodic BTC/ETH funding, vol-cone, gamma, regime-context, and alpha-canvas refresh
- periodic BTC/ETH perps carry, liquidation, toxicity, positioning, and fragility refresh
- periodic BTC/ETH and market composite sentiment / direction / fragility refresh
- microstructure historical import / retraining / paper-signal generation
- microstructure live inference daemon
- always-on API
- always-on internal UI

The current deployment target is:
- host: `192.168.1.69`
- repo path: `/Users/mubaris/dev/uforia`
- DB runtime: native local Postgres on the current Mac Mini, with Docker Compose support remaining available if Docker is installed later

## Repo-managed runner scripts
Tracked service entrypoints live in `ops/`:
- `ops/run-api.sh`
- `ops/run-web.sh`
- `ops/run-archiver.sh`
- `ops/run-refresh.sh`
- `ops/run-microstructure-live.sh`
- `ops/run-microstructure-backfill.sh`
- `ops/run-microstructure-train.sh`
- `ops/run-microstructure-inference.sh`
- `ops/run-db.sh`
- `ops/run-db-backfill.sh`
- `ops/run-db-backfill-ui.sh`
- `ops/run-db-backfill-raw.sh`
- `ops/run-db-sync-hot.sh`
- `ops/run-db-sync-raw-recent.sh`
- `ops/run-db-verify.sh`
- `ops/run-db-maintenance.sh`
- `ops/install-launchd.sh`

These scripts are the stable command surface for `launchd`.

Repo-managed `launchd` templates live in `launchd/`:
- `launchd/com.uforia.api.plist`
- `launchd/com.uforia.web.plist`
- `launchd/com.uforia.archiver.plist`
- `launchd/com.uforia.refresh.plist`
- `launchd/com.uforia.microstructure-live.plist`
- `launchd/com.uforia.microstructure-inference.plist`
- `launchd/com.uforia.microstructure-train.plist`
- `launchd/com.uforia.db-sync-hot.plist`
- `launchd/com.uforia.db-sync-raw-recent.plist`
- `launchd/com.uforia.db-backfill-maintenance.plist`

## Required one-time install
On the Mac Mini:

```bash
cd /Users/mubaris/dev/uforia

uv sync --extra dev
npm install
npm --workspace apps/web run build
chmod +x ops/*.sh
mkdir -p /Users/mubaris/Library/Logs/uforia
./ops/run-db.sh
export UFORIA_DB_ENABLED=true
export UFORIA_DB_DSN=postgresql:///uforia?user=mubaris
export UFORIA_DB_READ_MODE=auto
export UFORIA_DB_WRITE_MODE=dual
export UFORIA_DB_STRICT_DATASETS="$(uv run python - <<'PY'
from uforia.contracts import UI_EXPOSED_DATASETS
print(",".join(UI_EXPOSED_DATASETS))
PY
)"
uv run uforia-admin db init
./ops/install-launchd.sh
```

Historical Timescale backfill and parity checks:

```bash
./ops/run-db-backfill-ui.sh
./ops/run-db-verify.sh
```

`ops/run-db-backfill-ui.sh` migrates every dataset exposed by the API/UI so the API runner can stay DB-only for those reads.

For a lower-impact staged migration, backfill the serving datasets first:

```bash
export UFORIA_DB_BACKFILL_DATASETS=ingestion_runs,dvol_snapshot,funding_rate_snapshot,vol_index_series,vvix_series,razor_series,options_signal_series,perps_composite_series,microstructure_model_output,microstructure_signal_series
./ops/run-db-backfill.sh
```

`ops/run-db-backfill.sh` runs each dataset at lower scheduler priority via `nice`. Use the staged list above during normal hours and leave the full raw microstructure backfill for a maintenance window.

For the full raw microstructure history, use:

```bash
./ops/run-db-backfill-raw.sh
```

An optional maintenance `launchd` template is checked in at:
- `launchd/com.uforia.db-backfill-maintenance.plist`

It is configured for Sunday 02:00 local time and runs the raw-only backfill script.

## Frequent DB sync jobs
New Python writes already run in dual-write mode, so fresh rows should reach Parquet and DB at the same time for:
- API/refresh builders
- Deribit archiver
- microstructure training/inference jobs

Two additional `launchd` sync timers are now available to repair drift quickly if any DB write is missed:

- `com.uforia.db-sync-hot`
  - runs every `10` minutes
  - syncs the hot serving datasets for the last `72` hours
- `com.uforia.db-sync-raw-recent`
  - runs every `30` minutes
  - syncs recent raw microstructure datasets for the last `6` hours

Default hot datasets:
- `ingestion_runs`
- `dvol_snapshot`
- `funding_rate_snapshot`
- `vol_index_series`
- `vvix_series`
- `razor_series`
- `options_signal_series`
- `perps_composite_series`
- `microstructure_model_output`
- `microstructure_signal_series`
- `microstructure_training_run`
- `microstructure_backtest_run`

Default recent raw datasets:
- `microstructure_raw_event`
- `microstructure_book_state`
- `microstructure_bar`

You can override the windows/datasets via:
- `UFORIA_DB_SYNC_HOT_HOURS`
- `UFORIA_DB_SYNC_HOT_DATASETS`
- `UFORIA_DB_SYNC_RAW_HOURS`
- `UFORIA_DB_SYNC_RAW_DATASETS`

For the current native-Postgres setup, `ops/run-db.sh`:
- starts Homebrew `postgresql@17` if Docker is unavailable
- creates the `uforia` database if missing
- applies ingest-friendly WAL/checkpoint tuning for large backfills
- leaves the API/web/refresh stack responsible for dual-write and DB-backed reads

If needed for web startup, ensure a Node binary exists under:
- `$HOME/.nvm/versions/node/*/bin`

`ops/run-web.sh` adds detected NVM Node bins to `PATH` automatically.

## Launchd services
The machine uses four core `LaunchAgents`:
- `com.uforia.api`
- `com.uforia.web`
- `com.uforia.archiver`
- `com.uforia.refresh`

Optional additional agents once the microstructure path is enabled:
- `com.uforia.microstructure-live`
- `com.uforia.microstructure-inference`
- `com.uforia.microstructure-train`

Execution agents:
- `com.uforia.execution-engine`
- `com.uforia.execution-state-sync`

Expected working directories and paths:
- working directory: `/Users/mubaris/dev/uforia`
- logs: `/Users/mubaris/Library/Logs/uforia`

Expected runtime behavior:
- API bound to `0.0.0.0:8000`
- web bound to `0.0.0.0:3000`
- refresh runs on schedule and exits `0`
- archiver stays alive and spawns both BTC and ETH collectors
- refresh also builds the composite family for BTC, ETH, and the market aggregate
- refresh also builds the alpha-canvas options family: surface-factor RV, ML carry toxicity, expiry flow, adaptive hedging, sessionality clocks, BTC-ETH dispersion, and funding-basis skew
- microstructure live runner is available as a separate `launchd` unit when enabled
- the live engine now writes raw JSONL and health files under `data/microstructure_live`

## Launchd lifecycle
Bootstrap:

```bash
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.api.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.web.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.archiver.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.refresh.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.microstructure-live.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.microstructure-inference.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.microstructure-train.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.execution-engine.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.uforia.execution-state-sync.plist
```

Restart:

```bash
launchctl kickstart -k gui/$(id -u)/com.uforia.api
launchctl kickstart -k gui/$(id -u)/com.uforia.web
launchctl kickstart -k gui/$(id -u)/com.uforia.archiver
launchctl kickstart -k gui/$(id -u)/com.uforia.refresh
launchctl kickstart -k gui/$(id -u)/com.uforia.microstructure-live
launchctl kickstart -k gui/$(id -u)/com.uforia.microstructure-inference
launchctl kickstart -k gui/$(id -u)/com.uforia.microstructure-train
launchctl kickstart -k gui/$(id -u)/com.uforia.execution-engine
launchctl kickstart -k gui/$(id -u)/com.uforia.execution-state-sync
```

Inspect:

```bash
launchctl print gui/$(id -u)/com.uforia.api | sed -n '1,60p'
launchctl print gui/$(id -u)/com.uforia.web | sed -n '1,60p'
launchctl print gui/$(id -u)/com.uforia.archiver | sed -n '1,60p'
launchctl print gui/$(id -u)/com.uforia.refresh | sed -n '1,80p'
launchctl print gui/$(id -u)/com.uforia.microstructure-live | sed -n '1,80p'
launchctl print gui/$(id -u)/com.uforia.microstructure-inference | sed -n '1,80p'
launchctl print gui/$(id -u)/com.uforia.microstructure-train | sed -n '1,80p'
launchctl print gui/$(id -u)/com.uforia.execution-engine | sed -n '1,80p'
launchctl print gui/$(id -u)/com.uforia.execution-state-sync | sed -n '1,80p'
```

Logs:

```bash
tail -f /Users/mubaris/Library/Logs/uforia/api.err.log
tail -f /Users/mubaris/Library/Logs/uforia/web.err.log
tail -f /Users/mubaris/Library/Logs/uforia/archiver.err.log
tail -f /Users/mubaris/Library/Logs/uforia/refresh.err.log
tail -f /Users/mubaris/Library/Logs/uforia/microstructure-live.err.log
tail -f /Users/mubaris/Library/Logs/uforia/microstructure-inference.err.log
tail -f /Users/mubaris/Library/Logs/uforia/microstructure-train.err.log
tail -f /Users/mubaris/Library/Logs/uforia/execution-engine.err.log
tail -f /Users/mubaris/Library/Logs/uforia/execution-state-sync.err.log
```

## Deploy workflow
For changes that affect the Mac Mini runtime, use this order:

1. change code locally
2. run local checks
3. commit locally
4. push to Git
5. SSH into the Mac Mini
6. pull the new Git revision
7. rebuild Python and web artifacts if needed
8. restart the affected services
9. verify API, UI, and data behavior on the machine

This repo now treats that as the default production workflow.

## Current network behavior
Use plain HTTP on the LAN:
- UI: `http://192.168.1.69:3000`
- API: `http://192.168.1.69:8000/api/v1/system/health`

There is currently no HTTPS/TLS termination in front of the services.

Browser-facing fixes already applied:
- browser-side API calls derive the host from the page URL
- browser-side WebSocket calls derive the host from the page URL
- API CORS allows private-network origins such as `http://192.168.1.69:3000`

## Troubleshooting notes
Common failure modes already encountered:

`EX_CONFIG` from `launchd`
- usually means bad paths, missing log directories, or non-executable runner scripts

`STREAM DOWN`
- can mean missing WebSocket support in the API environment
- can also mean browser-side host mismatch if the UI points at `127.0.0.1`

UI loads once then blanks out
- usually means SSR worked but client-side refresh failed
- the main cause encountered so far was CORS rejecting the LAN origin

DVOL around `6000`
- caused by a units bug where Deribit DVOL values were incorrectly multiplied by `100`
- the fix is in Git and should be present on any deployed revision after `76c70ce`

## Operational verification
Minimal checks after deployment:

```bash
curl -s http://127.0.0.1:8000/api/v1/system/health
curl -I --max-time 5 http://127.0.0.1:3000
curl -i -X OPTIONS http://127.0.0.1:8000/api/v1/system/health \
  -H "Origin: http://192.168.1.69:3000" \
  -H "Access-Control-Request-Method: GET"
```

WebSocket smoke test:

```bash
uv run python - <<'PY'
import asyncio
import websockets

async def main():
    async with websockets.connect("ws://127.0.0.1:8000/api/v1/stream") as ws:
        print(await asyncio.wait_for(ws.recv(), timeout=6))

asyncio.run(main())
PY
```

## Data refresh behavior
`ops/run-refresh.sh` now rebuilds, for both `BTC` and `ETH`:
- `dvol_snapshot`
- `funding_rate_snapshot`
- `vol_index_series`
- `vvix_series`
- `surface_features` when enough option history exists
- `razor_series`
- `options_surface_snapshot`
- `options_carry_series`
- `options_skew_series`
- `options_dislocation_series`
- `options_positioning_series`
- `options_signal_series`
- `vol_cone_series`
- `term_structure_momentum_series`
- `gamma_exposure_series`
- `funding_context_series`
- `straddle_breakeven_series`

## Execution credentials and smoke checks

Execution services load credentials and account policy from env plus:
- `/Users/mubaris/dev/uforia/config/execution_accounts.json`

Use the example in the repo as the template:
- `config/execution_accounts.example.json`

Recommended execution env vars on the Mac Mini:
- `UFORIA_EXECUTION_ENABLED=true`
- `UFORIA_EXECUTION_ACCOUNTS_PATH=/Users/mubaris/dev/uforia/config/execution_accounts.json`
- `UFORIA_EXECUTION_DEFAULT_MODE=dry_run`
- `UFORIA_EXECUTION_KILL_SWITCH=true`
- `UFORIA_HYPERLIQUID_PRIVATE_KEY=...`
- `UFORIA_LIGHTER_API_PRIVATE_KEY=...`
- `UFORIA_DERIVE_SESSION_KEY=...`
- `UFORIA_DERIVE_PRIVATE_KEY=...`
- `UFORIA_BEBOP_PRIVATE_KEY=...`
- `UFORIA_BEBOP_RPC_URL=...`

Execution smoke checks:

```bash
cd /Users/mubaris/dev/uforia
cargo run -p uforia-execution-app -- dry-run-order --venue hyperliquid --symbol BTC-PERP
cargo run -p uforia-execution-app -- dry-run-order --venue bebop --symbol BTC-USD
uv run uforia-admin execution seed-markets --asset BTC
uv run uforia-admin execution sync-state --health-path /Users/mubaris/dev/uforia/data/execution_live/health/latest.json
uv run uforia-admin execution sync-live-state --venue hyperliquid --account hyperliquid-main --asset BTC
curl http://127.0.0.1:8000/api/v1/execution/venues
curl http://127.0.0.1:8000/api/v1/execution/health
```

For Bebop live spot execution on the Mac Mini:
- request quotes first
- ensure the Bebop account config includes `execution_mode=self_execute` or a gasless order path
- use the stored quote payload plus `UFORIA_BEBOP_RPC_URL` for self-execution broadcast
- `vol_regime_transition_series`
- `perps_signal_series`
- `perps_composite_series`
- `perps_transition_series`
- cross-asset `options_relative_value_series`
- per-asset validation outputs under `data/validation-btc-latest.json` and `data/validation-eth-latest.json`
- per-asset options validation outputs under `data/options-validation-btc-latest.json` and `data/options-validation-eth-latest.json`
- and may be followed by microstructure retraining / signal generation jobs, depending on whether you enable the optional microstructure agents

## One-time archive rebuilds
When a new normalized option-chain field is added, such as `gamma`, the raw Deribit archive can be re-normalized without waiting for all-new market hours.

Example:

```bash
cd /Users/mubaris/dev/uforia

uv run uforia-admin maintenance rebuild-options-from-archive \
  --underlying BTC \
  --start 2026-03-09T00:00:00Z \
  --end 2026-03-09T23:00:00Z

uv run uforia-admin maintenance rebuild-options-from-archive \
  --underlying ETH \
  --start 2026-03-09T00:00:00Z \
  --end 2026-03-09T23:00:00Z
```

After a rebuild, rerun the refresh or at least:
- `build gamma-exposure`
- `build options-positioning`
- `build options-signals`
- `build vol-regime-transitions`

## Microstructure startup on the Mac Mini

Initial import and training from the legacy history source:

```bash
cd /Users/mubaris/dev/uforia

uv run uforia-admin microstructure import-history --root /Users/mubaris/dev/crypto-microstructure

for ASSET in BTC ETH; do
  uv run uforia-admin microstructure build-features --asset "$ASSET" --start 2026-01-01T00:00:00Z --end 2026-03-09T00:00:00Z
  uv run uforia-admin microstructure build-labels --asset "$ASSET" --start 2026-01-01T00:00:00Z --end 2026-03-09T00:00:00Z --horizons 30s,1m,2m,5m
  uv run uforia-admin microstructure build-regime --asset "$ASSET" --start 2026-01-01T00:00:00Z --end 2026-03-09T00:00:00Z
  for HORIZON in 1m 2m 5m 30s; do
    uv run uforia-admin microstructure train --asset "$ASSET" --horizon "$HORIZON"
    uv run uforia-admin microstructure evaluate --asset "$ASSET" --horizon "$HORIZON"
    uv run uforia-admin microstructure generate-signals --asset "$ASSET" --horizon "$HORIZON"
  done
done
```

Live engine smoke checks:

```bash
cd /Users/mubaris/dev/uforia

cargo run -p uforia-ms-app -- --config engines/microstructure/config/default.toml health
cat data/microstructure_live/health/latest.json | jq .
tail -f data/microstructure_live/raw/market/date=$(date -u +%F)/events.jsonl
tail -f data/microstructure_live/book_state/date=$(date -u +%F)/state.jsonl
./ops/run-microstructure-inference.sh
```

Operational note:
- Binance books should only be trusted once `is_seeded=true` and `is_sequence_valid=true`.
- repaired microstructure inference only accepts `tcn_boost_fusion_v2` artifacts
- old `v1` training artifacts and backtest numbers should be treated as invalid

Minimal live verification after training:

```bash
curl -s http://127.0.0.1:8000/api/v1/microstructure/signals
curl -s http://127.0.0.1:8000/api/v1/microstructure/health
```

The checked-in `com.uforia.microstructure-live.plist` keeps the Rust feed engine alive. The checked-in `com.uforia.microstructure-inference.plist` turns that live state into predictions and paper signals. The checked-in `com.uforia.microstructure-train.plist` runs the full import/build/train/evaluate/generate cycle nightly at `02:15`.
