# Reviewer Mode

## Goal

Add a `--reviewer <executable>` CLI option. After the agent produces a final answer, the reviewer executable is spawned with the base directory as its sole argument and the agent's answer piped to its stdin. The reviewer's exit code determines what happens next:

- **0** — "fine": accept the answer, print it, exit normally.
- **1** — "try again": feed the reviewer's stdout back as a new user message, reset the turn counter, and re-enter the agent loop (preserving full conversation context).
- **2** — "error on my end": treat the reviewer itself as broken, print a warning, and exit with the answer as-is.

Any other exit code is treated like 2.

Incompatible with `--repl` (error out if both are specified).

## Design decisions

**Reviewer speaks through stdout.** The reviewer process writes its review/feedback to stdout. Swival captures that and, on exit code 1, appends it as a `{"role": "user", "content": review}` message. This is identical to how `repl_loop`'s `/continue` command works — the model sees a new user message and resumes tool-calling.

**Turn counter resets, context doesn't.** The loop restarts with `turns = 0` but the `messages` list is untouched (it already contains all prior assistant/tool messages). This gives the model a fresh budget to work with the review feedback while retaining full history.

**Hard cap of 5 review rounds.** The reviewer loop runs at most `MAX_REVIEW_ROUNDS = 5` times. If the reviewer keeps returning 1 after 5 rounds, the last answer is accepted with a warning. This prevents infinite loops from a misconfigured or adversarial reviewer. The constant lives at module level in `agent.py`.

**Reviewer timeout.** The subprocess gets a 120-second timeout (matching `run_command`'s cap). If it times out, treat it as exit code 2 with a warning.

**All reviewer runtime failures are non-fatal.** The executable is validated at startup (exists + executable bit). If it later fails to spawn at runtime (permissions changed, binary deleted, timeout, crash), the failure is downgraded to exit code 2 semantics: warn on stderr, accept the answer as-is. The agent's work is never lost due to a broken reviewer.

**Quiet mode: reviewer diagnostics follow the `verbose` gate.** All reviewer-related stderr output goes through `fmt` and is gated on `verbose`, same as every other diagnostic in the codebase. `--quiet` suppresses everything. Intermediate answers (rejected by the reviewer) are never printed to stdout; only the final accepted answer is.

**Report interaction.** When `--report` is also active, the report captures the full timeline across all review rounds with cumulative turn numbering. `turn_offset` is set to `report.max_turn_seen` before each retry so that `report.record_llm_call(turn + turn_offset, ...)` produces monotonically increasing turn numbers. An additional `review_rounds` field in the stats tracks how many times the reviewer was invoked.

## Implementation plan

### 1. CLI argument in `build_parser()`

Add to the argument parser (near `--report`):

```python
parser.add_argument(
    "--reviewer",
    metavar="EXECUTABLE",
    default=None,
    help="Path to reviewer executable. Called after each answer with base_dir as argument "
         "and answer on stdin. Exit 0=accept, 1=retry with stdout as feedback, 2=reviewer error.",
)
```

Validation in `main()`, alongside the existing `--report` + `--repl` check:
- `--reviewer` + `--repl` → `parser.error("--reviewer is incompatible with --repl")`

Executable validation in `_run_main()`, before the loop:
- Verify the executable exists and is executable: `shutil.which(args.reviewer)` or `Path(args.reviewer).is_file()` and `os.access(path, os.X_OK)`. Raise `AgentError` if not found at startup.

### 2. `run_reviewer()` function in `agent.py`

A standalone function that spawns the reviewer process and returns a structured result:

```python
MAX_REVIEW_ROUNDS = 5

def run_reviewer(
    reviewer_cmd: str,
    base_dir: str,
    answer: str,
    verbose: bool,
    timeout: int = 120,
) -> tuple[int, str]:
    """Run the reviewer executable.

    Returns (exit_code, stdout_text).
    Never raises — all failures return (2, "") with a warning on stderr.
    """
```

Implementation:
- Use `subprocess.run()` with `input=answer.encode()`, `capture_output=True`, `timeout=timeout`.
- Pass `[reviewer_cmd, base_dir]` as the command.
- On `subprocess.TimeoutExpired`: log warning via `fmt.warning()`, return `(2, "")`.
- On `FileNotFoundError` / `PermissionError` / `OSError`: log warning via `fmt.warning()`, return `(2, "")`.
- Return `(proc.returncode, proc.stdout.decode("utf-8", errors="replace"))`.

### 3. Review loop in `_run_main()` single-shot path

Replace the current single-shot block (lines 974–997) with a loop:

```python
# Single-shot path
messages.append({"role": "user", "content": args.question})
review_round = 0
turn_offset = 0

while True:
    answer, exhausted = run_agent_loop(
        messages, tools, **loop_kwargs, report=report,
        turn_offset=turn_offset,
    )

    if not args.reviewer or answer is None or exhausted:
        # No reviewer, no answer, or exhausted — proceed as before
        break

    review_round += 1
    if args.verbose:
        fmt.info(f"Review round {review_round}: sending answer to reviewer")

    exit_code, review_text = run_reviewer(
        args.reviewer, base_dir, answer, args.verbose
    )

    if exit_code == 0:
        if args.verbose:
            fmt.info("Reviewer accepted the answer")
        break
    elif exit_code == 1:
        if review_round >= MAX_REVIEW_ROUNDS:
            if args.verbose:
                fmt.warning(f"Max review rounds ({MAX_REVIEW_ROUNDS}) reached, accepting answer")
            break
        if args.verbose:
            fmt.info(f"Reviewer requested changes:\n{review_text[:500]}")
        # Feed review back as a user message, reset turns
        messages.append({"role": "user", "content": review_text})
        # Snapshot turn_offset: max_turn_seen is already the high-water mark
        # across all prior rounds, so assign (not accumulate) to avoid overcounting.
        if report:
            turn_offset = report.max_turn_seen
        loop_kwargs["max_turns"] = args.max_turns  # reset turn budget
        continue
    else:
        # exit code 2 or anything else — reviewer error
        if args.verbose:
            fmt.warning(f"Reviewer exited with code {exit_code}, accepting answer as-is")
        break

# Existing post-loop logic (history, print, exit code) unchanged.
# _write_report call gains review_rounds=review_round (0 when no reviewer).
```

The key behaviors:
- `loop_kwargs` is mutated to reset `max_turns` on each retry round.
- `messages` accumulates across rounds (context preserved).
- `turn_offset` ensures report timeline has monotonically increasing turn numbers.
- The `MAX_REVIEW_ROUNDS` cap prevents infinite loops.
- All `fmt` calls are gated on `verbose` — quiet mode sees nothing on stderr.

### 4. Stderr formatting in `fmt.py`

No new function needed. The review loop uses existing `fmt.info()` and `fmt.warning()`, both gated by `if verbose:` at the call site. This matches the existing pattern throughout the codebase and avoids adding surface area for a few log lines.

### 5. Report integration

When `--report` and `--reviewer` are both active:

- Thread `review_rounds` end-to-end: add a `review_rounds: int = 0` parameter to `_write_report()`, which passes it to `report.finalize()`, which passes it to `build_report()`, which includes it in the `stats` dict. Default is 0 so non-reviewer runs need no changes at any call site.
- Add a `turn_offset: int = 0` parameter to `run_agent_loop()`. Inside the loop, all `report.record_*()` calls use `turns + turn_offset` instead of bare `turns`. The caller in the review loop sets `turn_offset = report.max_turn_seen` before each retry (see section 3). This gives monotonically increasing turn numbers across rounds with no changes to `ReportCollector` itself.
- Each `run_agent_loop` call records its own timeline events normally — no special handling needed since the same `report` instance accumulates across rounds.

### 6. Tests in `tests/test_reviewer.py`

**Core behavior:**
- **Accepted on first try (exit 0):** Mock reviewer returns 0. Verify answer is printed once, `run_agent_loop` called once.
- **Retry then accept (exit 1 → 0):** Mock reviewer returns 1 with feedback text, then 0. Verify feedback appended as user message, `run_agent_loop` called twice, turn counter reset.
- **Reviewer error (exit 2):** Mock reviewer returns 2. Verify warning on stderr, answer printed as-is.
- **Unknown exit code (exit 42):** Treated as exit code 2 semantics. Verify warning, answer accepted.
- **No answer (exhausted):** Agent exhausts turns. Verify reviewer is NOT called.
- **Incompatible with `--repl`:** Verify argparse error.
- **Multiple retry rounds:** Reviewer returns 1 three times, then 0. Verify messages accumulate correctly, four `run_agent_loop` calls.
- **Context preservation:** After retry, verify messages list contains original + tool + answer + review feedback.

**Failure modes:**
- **Reviewer timeout:** Mock `subprocess.run` to raise `TimeoutExpired`. Verify warning, answer accepted, no raise.
- **Reviewer not found at startup:** Invalid `--reviewer` path at startup validation. Verify `AgentError`.
- **Reviewer spawn failure at runtime:** Mock `subprocess.run` to raise `FileNotFoundError`. Verify warning, answer accepted (not `AgentError`).
- **Reviewer permission error at runtime:** Mock `subprocess.run` to raise `PermissionError`. Same as above.
- **Max review rounds hit:** Reviewer returns 1 six times. Verify only 5 rounds run, last answer accepted with warning.
- **Empty stdout on exit 1:** Reviewer returns 1 with empty stdout. Verify empty string is appended as user message (the model can handle it; the reviewer is the one misbehaving).

**Interactions:**
- **Quiet mode:** `--quiet` + `--reviewer`. Verify no stderr output during review rounds.
- **Report mode:** `--report` + `--reviewer`. Verify `review_rounds` in stats, cumulative turn numbers in timeline.

### 7. Namespace updates in existing tests

Tests that build `argparse.Namespace` objects by hand need `reviewer=None` added. Grep for `argparse.Namespace(` and `SimpleNamespace(` across `tests/` and patch them.

## File change summary

| File | Change |
|------|--------|
| `swival/agent.py` | `--reviewer` arg, `MAX_REVIEW_ROUNDS` constant, `run_reviewer()` function, review loop in `_run_main()`, `turn_offset` plumbing for report |
| `swival/report.py` | `review_rounds` field in stats |
| `tests/test_reviewer.py` | New test file (18 test cases) |
| `tests/test_*.py` | Add `reviewer=None` to hand-built namespace objects |
