# QuickCall Trace — Deployment Guide

How to release a new version without breaking developer machines.

---

## The Release Pipeline

```
You: bump version + tag + push
  │
  ├─► release.yml ──► test ──► GitHub Release ──► Build ──► PyPI publish
  │
  └─► deploy.yml  ──► Docker build ──► Push to ACR ──► Azure Container Apps
```

**Tag triggers both workflows.** A single `git push --tags` kicks off PyPI publish AND server deploy.

---

## Step-by-Step Release

### 1. Bump version in BOTH places

```bash
# These MUST match. If they don't, the daemon auto-update will loop.
vim qc_trace/__init__.py    # __version__ = "X.Y.Z"
vim pyproject.toml           # version = "X.Y.Z"
```

**Why both?** `pyproject.toml` is what PyPI reads. `__init__.py` is what the running daemon reports. If they disagree, the daemon thinks it's running a different version than what's published and keeps restarting.

### 2. Commit, tag, push

```bash
git add qc_trace/__init__.py pyproject.toml
git commit -m "chore: bump version to X.Y.Z"
git tag vX.Y.Z
git push && git push --tags
```

### 3. Wait for CI

- **release.yml**: runs tests → creates GitHub Release → builds wheel → publishes to PyPI
- **deploy.yml**: builds Docker image → pushes to ACR → deploys to Azure Container Apps

Check both at: `https://github.com/quickcall-dev/trace/actions`

### 4. Update version_config in the database

**This is what tells developer daemons to update.** The daemon checks your server, not PyPI.

```sql
-- Connect to your database
INSERT INTO version_config (key, value)
VALUES ('latest_version', 'X.Y.Z')
ON CONFLICT (key) DO UPDATE SET value = 'X.Y.Z';
```

If you skip this step, daemons won't auto-update even though PyPI has the new version.

### 5. Verify

```bash
# Check PyPI has the new version
curl -s https://pypi.org/pypi/qc-trace/json | python3 -c "import json,sys; print(json.load(sys.stdin)['info']['version'])"

# Check your server reports the new version
curl -s https://<your-server>/api/latest-version

# Check the server is healthy after deploy
curl -s https://<your-server>/health
```

---

## How Auto-Update Works

```
┌─────────────────────────────────┐
│  Developer's Mac                │
│                                 │
│  launchd                        │
│    └─► quickcall-daemon         │  ← wrapper script (restarts on exit)
│          └─► uvx quickcall run  │  ← foreground daemon process
│                │                │
│                │ every 5 min:   │
│                │ GET /api/latest-version
│                │                │
│                ▼                │
│  version != current?            │
│    YES → exit(0)                │
│    └─► wrapper restarts         │
│       └─► uvx fetches new ver   │
│          └─► quickcall run      │  ← now on new version
│                                 │
│  version == current?            │
│    NO-OP → keep running         │
└─────────────────────────────────┘
```

Key points:
- The daemon checks **your server** (`/api/latest-version`), not PyPI
- The wrapper runs `uvx --no-cache --from "qc-trace@${VERSION}" quickcall run`
- `quickcall run` = **foreground** mode (blocks, launchd manages lifecycle)
- `quickcall start` = **background** mode (spawns subprocess, returns immediately — DO NOT use with launchd)
- On version mismatch the daemon exits cleanly, wrapper sleeps 10s, fetches new version from PyPI via uvx, restarts

---

## Rollback

Because the daemon checks your server (not PyPI), rollback is just a DB update:

```sql
-- Roll back all daemons to 0.4.3
UPDATE version_config SET value = '0.4.3' WHERE key = 'latest_version';
```

Within 5 minutes, all connected daemons will restart and `uvx` will install 0.4.3 from PyPI.

**Prerequisite**: The old version must still exist on PyPI. If you yanked it, rollback won't work.

---

## Common Mistakes (Learned the Hard Way)

### 1. Version mismatch between pyproject.toml and __init__.py

**Symptom**: Daemon starts as v0.4.4 but PyPI shows v0.4.1. Auto-update loops forever.

**Cause**: Only bumped one file.

**Fix**: Always bump both:
```
pyproject.toml:     version = "X.Y.Z"
qc_trace/__init__.py:  __version__ = "X.Y.Z"
```

### 2. Wrapper uses `quickcall start` instead of `quickcall run`

**Symptom**: Daemon keeps dying. `quickcall start` works manually but launchd can't keep it alive.

**Cause**: `quickcall start` spawns a background process and returns immediately. launchd sees the wrapper exit, restarts it, spawns another daemon, repeat.

**Fix**: Wrapper must use `quickcall run` (foreground mode). launchd expects the managed process to stay alive.

### 3. Forgot to update version_config in the database

**Symptom**: New version is on PyPI but no developers auto-update.

**Cause**: The daemon checks your server's `/api/latest-version`, which reads from `version_config` table.

**Fix**: After every release, update the DB:
```sql
INSERT INTO version_config (key, value)
VALUES ('latest_version', 'X.Y.Z')
ON CONFLICT (key) DO UPDATE SET value = 'X.Y.Z';
```

### 4. Auto-update used `!=` on PyPI (downgrade loop)

**Symptom**: Daemon picks up new version, then "updates" back to old version, loops.

**Cause**: Old code checked PyPI with `!=`. If PyPI returned a cached/stale version, daemon would downgrade.

**Fix**: (Already fixed) Daemon now checks your server, not PyPI. You control the target version.

### 5. `uvx qc-trace` instead of `uvx --from qc-trace quickcall`

**Symptom**: `An executable named 'qc-trace' is not provided by package 'qc-trace'`

**Cause**: The package name (`qc-trace`) differs from the executable name (`quickcall`).

**Fix**: Always use `uvx --from qc-trace quickcall <command>`

---

## File Reference

| File | What it does |
|------|-------------|
| `pyproject.toml` | Package version for PyPI |
| `qc_trace/__init__.py` | Runtime version (daemon reports this) |
| `qc_trace/daemon/main.py` | Auto-update check logic (5 min interval, hits your server) |
| `qc_trace/cli/traced.py` | CLI: `run` (foreground), `start` (background), `status`, `logs` |
| `scripts/install.sh` | Generates `~/.quickcall-trace/quickcall-daemon` wrapper |
| `.github/workflows/release.yml` | Tag → test → GitHub Release → PyPI publish |
| `.github/workflows/deploy.yml` | Tag → Docker build → ACR → Azure Container Apps |
| DB: `version_config` | Server-controlled version gating for daemon updates |

---

## Linux/systemd Verification Runbook

After installing on a Linux VM, run through these checks to verify v0.4.12+ behavior.

### Setup

```bash
curl -fsSL https://quickcall.dev/trace/install.sh -o /tmp/install-trace.sh
bash /tmp/install-trace.sh <ORG> <API_KEY>
```

### A. Monitor catch-up via /api/monitor

```bash
curl -s -H "X-API-Key: <ADMIN_KEY>" https://trace.quickcall.dev/api/monitor | python3 -m json.tool
```

During catch-up: `status: "receiving"`, `messages_last_1m` high, `idle_seconds` low.
After catch-up: `status: "idle"`, `messages_last_1m: 0`, `idle_seconds` growing.

Verify `total_dropped: 0` and `flush_failures: 0` throughout.

### B. Daemon-side via quickcall status

```bash
export PATH="$HOME/.local/bin:$PATH"
uvx --quiet --no-cache --from qc-trace quickcall status
```

During catch-up: `Rate` should be >100 msg/min (typically >500).

### C. Kill daemon mid-push, verify no data loss

```bash
# Record pre-kill message count
curl -s -H "X-API-Key: <ADMIN_KEY>" https://trace.quickcall.dev/api/monitor \
  | python3 -c "import json,sys; print(json.load(sys.stdin)['database']['total_messages'])"

# Kill daemon
kill -9 $(pgrep -f "quickcall run" | head -1)

# Wait for auto-restart (wrapper restarts within 5-10s)
sleep 10

# Verify service restarted
systemctl --user is-active quickcall

# Check post-restart count (should be >= pre-kill, no duplicates)
curl -s -H "X-API-Key: <ADMIN_KEY>" https://trace.quickcall.dev/api/monitor \
  | python3 -c "import json,sys; print(json.load(sys.stdin)['database']['total_messages'])"
```

### D. Check daemon logs for batched pushes

```bash
journalctl --user-unit quickcall --no-pager -n 30
```

v0.4.12+ should show ~4 log lines per cycle with "Accumulation cap reached" messages, NOT hundreds of per-file POST lines.

### E. Debug reference

| File / Command | What to check |
|------|----|
| `~/.quickcall-trace/quickcall.log` | Daemon errors, push failures |
| `~/.quickcall-trace/push_status.json` | queue_size, backoff, daemon_version |
| `~/.quickcall-trace/state.json` | Per-file last_line_processed |
| `journalctl --user-unit quickcall -f` | systemd service logs |
| `curl /api/monitor` | Server ingestion health |
| `curl /health` | Basic server health |

---

## Quick Reference

```bash
# Full release
vim qc_trace/__init__.py pyproject.toml   # bump both to X.Y.Z
git add -A && git commit -m "chore: bump version to X.Y.Z"
git tag vX.Y.Z && git push && git push --tags
# Then update DB: UPDATE version_config SET value = 'X.Y.Z' WHERE key = 'latest_version';

# Rollback
# UPDATE version_config SET value = '<old_version>' WHERE key = 'latest_version';

# Check what version a developer is running
quickcall status --json | python3 -c "import json,sys; print(json.load(sys.stdin)['version'])"

# Force restart daemon locally (for testing)
quickcall stop && sleep 2 && launchctl list | grep quickcall
```
