# Code Review: `qc_trace/daemon/`

**Reviewer:** Claude Opus 4.6
**Date:** 2026-02-07
**Scope:** All files in `qc_trace/daemon/` -- the file-watching daemon that polls IDE session files, transforms them, and pushes to an ingest server.

---

## Critical

### C1. Gemini/Cursor collectors load entire files into memory with no size guard

**Files:** `collector.py:184`, `collector.py:241`

The watcher enforces `max_file_size` (100 MB) at the discovery stage, but `_collect_gemini` and `_collect_cursor` then call `_read_file()` which reads the entire file content into a string. A 100 MB string is then hashed (`hashlib.sha256(content.encode())`), which means at peak you hold both the original string and its encoded bytes -- roughly 200 MB of transient memory for a single file. If multiple large files are discovered in one poll cycle, the daemon could OOM.

The JSONL collectors (`_collect_claude`, `_collect_codex`) stream line-by-line and do not have this problem.

**Recommendation:** Stream the hash computation (feed chunks to `hashlib`), and for the content itself, consider a streaming parser or at minimum lower the `max_file_size` for JSON/text sources.

---

### C2. Push failure does not prevent state advancement

**File:** `main.py:137-149`

When `pusher.push(batch)` returns `False` (server unreachable), the code continues to the next batch and then unconditionally calls `self.state_mgr.set_state(result.new_state)` on line 147. This means the daemon records "I have processed up to line N" even though the data was never successfully delivered to the server. If the retry queue is later dropped (it has a cap of 10,000 items and silently discards the oldest), those messages are permanently lost.

The reconciliation mechanism (`_reconcile`) partially mitigates this on restart, but between restarts the data gap is silent.

**Recommendation:** Only advance state when push succeeds, or at minimum mark the state as "pending push" so it can be retried.

---

## High

### H1. `progress.py` is dead code within the daemon

**File:** `progress.py` (all 129 lines)

The module docstring says it is for "inter-agent communication during parallel worktree builds." It is never imported from anywhere inside `qc_trace/daemon/` or anywhere in the `qc_trace` package. The only import is from `tests/test_cli.py`, which tests the utility functions directly.

This module is scaffolding for an external orchestration feature, not part of the daemon's runtime. It should be relocated (e.g., to `qc_trace/utils/progress.py` or a separate orchestration package) or removed from the daemon package.

---

### H2. `remove_state` and `reset_state` are exact duplicates

**File:** `state.py:95-105`

```python
def remove_state(self, file_path: str) -> None:
    if file_path in self._states:
        del self._states[file_path]
        self._dirty = True

def reset_state(self, file_path: str) -> None:
    if file_path in self._states:
        del self._states[file_path]
        self._dirty = True
```

The bodies are identical. `remove_state` is never called from production code (only from tests). `reset_state` is used by `main.py` reconciliation. One should be an alias of the other, or one should be removed.

---

### H3. Codex collector replays entire file from line 0 every poll cycle

**File:** `collector.py:131-151`

The comment on line 131 explains that Codex requires replaying from the beginning to rebuild context, but only emitting messages for new lines. This means for a 50,000-line Codex JSONL file, every poll cycle parses all 50,000 lines even if only 1 new line was appended. As files grow, this becomes O(n) per poll for each Codex file. For very active sessions this could make the daemon CPU-bound.

**Recommendation:** Consider persisting the `CodexTransformContext` in state so replay can be avoided, or checkpoint the context periodically.

---

### H4. `_backoff` is computed but never enforced before push attempts

**File:** `pusher.py:108-110`, `main.py:143-145`

The `Pusher._on_failure()` method computes a backoff value, but the pusher itself never sleeps or delays before the next `_post_batch` call. The backoff is only consumed in `main.py:144-145` where the daemon sleeps between batches *within a single file's messages*. However:

1. The very next file in the same `_poll_cycle` calls `pusher.push()` immediately with no backoff check.
2. The `_drain_queue()` method (line 78-91) retries immediately after a success with no backoff.
3. If the server is down, the daemon will hammer it on every poll cycle (every 5s) with no exponential delay between cycles.

**Recommendation:** Apply the backoff delay before any push attempt, not just between batches of one file.

---

## Medium

### M1. Unused import `sys` in `__main__.py`

**File:** `__main__.py:4`

```python
import sys
```

`sys` is imported but never referenced in the file.

---

### M2. No file descriptor safety in JSONL collectors

**Files:** `collector.py:77`, `collector.py:136`

Both `_collect_claude` and `_collect_codex` open files with `open(changed.path, "r")` inside a `with` block, which is fine for normal cases. However, if the file is being actively written to by the IDE, the collector could read a partial final line. The code handles `json.JSONDecodeError` gracefully (logs and skips), but the line counter (`start_line = line_num`) still advances past the bad line. On the next poll cycle, that partial line is lost permanently because the daemon thinks it was already processed.

**Recommendation:** Only advance `start_line`/`last_line` after a successful parse, not for every line encountered. In `_collect_claude`, the assignment `start_line = line_num` on line 97 is inside the `try` block after `transform_claude_v1`, so it is actually correct. But in `_collect_codex`, `last_line = line_num` on line 151 is set for every line including failures (the `continue` on line 150 skips the `messages.extend` but not the `last_line` update). This means Codex silently drops failed lines.

---

### M3. Exception handler is overly broad in collectors

**Files:** `collector.py:92`, `collector.py:146`, `collector.py:207`, `collector.py:262`

```python
except (json.JSONDecodeError, Exception) as e:
```

`json.JSONDecodeError` is a subclass of `Exception`, so listing both is redundant -- this catches all exceptions. This swallows unexpected errors (e.g., `TypeError`, `KeyError` from transforms) that should arguably propagate. The outer handler in `main.py:126` would catch them and properly increment the retry counter.

**Recommendation:** Catch `json.JSONDecodeError` specifically for parse failures, and let transform errors propagate.

---

### M4. Reconciliation URL construction is fragile

**File:** `main.py:101`

```python
base_url = self.config.ingest_url.rsplit("/", 1)[0]  # strip /ingest
url = f"{base_url}/api/sync"
```

This assumes the ingest URL always ends with exactly `/ingest`. If the user configures `QC_TRACE_INGEST_URL` to something like `http://host:8080/v1/ingest/` (trailing slash) or `http://host:8080/api/v2/ingest`, the `rsplit` produces the wrong base URL. This would silently hit the wrong endpoint or 404.

**Recommendation:** Use `urllib.parse.urljoin` or derive the sync URL from a separate config field.

---

### M5. Retry queue silently drops oldest messages with no logging

**File:** `pusher.py:71-76`

```python
def _enqueue(self, batch: list[dict]) -> None:
    for item in batch:
        if len(self._queue) >= self.config.retry_queue_max:
            self._queue.popleft()  # drop oldest
        self._queue.append(item)
```

When the queue is full and messages are dropped, there is no log line. Data loss happens silently. This is especially dangerous combined with C2 (state is advanced even on push failure).

**Recommendation:** Log a warning when messages are dropped, including a count.

---

### M6. State file growth is unbounded

**File:** `state.py`

The state file tracks every file ever discovered. If files are deleted or rotated by the IDE, their entries remain in state forever. Over months this could grow to tens of thousands of entries. There is no pruning or garbage collection of stale entries (e.g., files that no longer exist on disk).

**Recommendation:** Add periodic pruning -- if the file no longer exists on disk, remove its state entry.

---

### M7. PID file has no stale-PID detection

**File:** `main.py:166-169`

```python
def _write_pid(self) -> None:
    self.config.base_dir.mkdir(parents=True, exist_ok=True)
    self.config.pid_file.write_text(str(os.getpid()))
```

If a previous daemon crashed without cleaning up its PID file, the new daemon overwrites it silently. There is no check for whether the old PID is still running. The CLI layer (`tests/test_cli.py` suggests there is one) may handle this, but the daemon itself does not prevent duplicate instances.

---

## Low

### L1. `__init__.py` exports only `DaemonConfig`, not the `Daemon` class

**File:** `__init__.py:3-5`

```python
from qc_trace.daemon.config import DaemonConfig
__all__ = ["DaemonConfig"]
```

The main entry point (`__main__.py`) imports `Daemon` directly from `qc_trace.daemon.main`. If the package's public API is `DaemonConfig`, it would be natural to also export `Daemon`.

---

### L2. `_collect_gemini` constructs `SessionContext` directly while others use `_build_session_context`

**File:** `collector.py:214-219`

The Gemini collector manually constructs a `SessionContext` instead of using the shared `_build_session_context()` helper. This means Gemini sessions do not get the `resolve_repo()` enrichment (repo URL, branch, commit) that other sources get. If a `cwd` equivalent is ever available for Gemini (e.g., from `projectHash` resolution), this inconsistency would cause it to be missed.

---

### L3. `_now_iso()` in `progress.py` uses manual strftime instead of `.isoformat()`

**File:** `progress.py:18-19`

```python
def _now_iso() -> str:
    return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
```

This could be `datetime.now(timezone.utc).isoformat(timespec='seconds')` which is more idiomatic, though the current approach works fine. Cosmetic only.

---

### L4. Watcher uses `glob.glob` (stdlib) instead of `pathlib.Path.glob`

**File:** `watcher.py:45`

The watcher imports and uses `glob.glob` with string paths while the rest of the codebase uses `pathlib.Path` extensively. This is a style inconsistency but functionally equivalent. Note that `glob.glob` with `recursive=True` and `**` can be slow on deep directory trees.

---

### L5. `_sleep` can compute negative sleep duration

**File:** `main.py:155`

```python
time.sleep(min(0.5, end - time.monotonic()))
```

If the loop body takes longer than expected, `end - time.monotonic()` can be negative. `time.sleep` with a negative value raises `ValueError` on some platforms. Should be `max(0, min(0.5, end - time.monotonic()))`.

---

## Architecture Assessment

**Overall structure:** The separation into watcher/collector/pusher/state is clean and follows a sensible pipeline: discover changed files, collect new messages, push to server, persist state. Each component has a single responsibility.

**Strengths:**
- Atomic state writes via tempfile+rename is correctly implemented.
- Line-resume for JSONL sources is efficient and avoids re-processing.
- Graceful shutdown via signal handlers with cooperative sleep checks.
- The reconciliation mechanism to sync with server state on startup is a good reliability feature.

**Weaknesses:**
- `progress.py` does not belong in this package; it is unrelated to the daemon's function.
- The push-failure-but-advance-state gap (C2) is the most architecturally significant issue -- it breaks the at-least-once delivery guarantee the retry queue is trying to provide.
- The backoff mechanism is split awkwardly between `Pusher` (computes it) and `Daemon` (partially applies it), making the actual retry behavior hard to reason about.
- No metrics or health endpoint -- the daemon is a black box at runtime. The only observability is log lines.

---

## Summary

| Severity | Count | Key themes |
|----------|-------|------------|
| Critical | 2 | Memory safety on large files; data loss on push failure |
| High | 4 | Dead code (progress.py); duplicate methods; O(n) replay; broken backoff |
| Medium | 7 | Unused import; silent data drops; unbounded state growth; fragile URL parsing |
| Low | 5 | Style inconsistencies; minor edge cases |
