# Plan: `--reviewer-mode` — Use Swival As A Reviewer Without A Wrapper Script

## Problem

Using swival as an LLM-as-judge reviewer currently requires writing a ~30-line bash wrapper script (`judge.sh`) that:
- Reads stdin, creates temp files, traps cleanup
- Spawns an inner swival instance with a hand-crafted prompt
- Parses "VERDICT: ACCEPT/RETRY" from output
- Maps verdicts to exit codes (0/1/2)
- Handles spawn failures and empty output

This is boilerplate. The wrapper is always the same shape — only the review criteria change. We should bake it into swival itself.

## Solution

Two changes that work together:

### 1. `--reviewer-mode` flag

Makes swival act as a reviewer executable, speaking the `--reviewer` protocol natively:

```sh
# Swival as the reviewer — no bash wrapper needed
swival "Fix the tests" --reviewer "swival --reviewer-mode"
```

When `--reviewer-mode` is active, swival:

1. Reads `base_dir` from the first positional argument (as per reviewer protocol)
2. Reads the agent's answer from stdin
3. Reads context from env vars (`SWIVAL_TASK`, `SWIVAL_REVIEW_ROUND`)
4. Builds a review prompt (built-in, or custom via `--review-prompt`)
5. Calls the LLM (using whatever `--model`/`--provider`/`--base-url` were passed)
6. Parses the verdict from the LLM response
7. Exits with 0 (accept), 1 (retry + feedback on stdout), or 2 (reviewer error)

**Positional arg semantics change in reviewer mode:** In normal mode, the positional arg is `question` (optional, `nargs="?"`) and `--base-dir` defaults to `.`. In reviewer mode, exactly one positional arg is required and it is interpreted as `base_dir` (the project root passed by the outer swival), not as a user question. The parser enforces this: if `--reviewer-mode` is set and the positional arg is missing, it's an error; if both `--reviewer-mode` and a `question`-shaped positional are present, the positional is always treated as `base_dir`.

**Config loading in reviewer mode:** Current `main()` calls `load_config(base_dir)` at `agent.py:1502-1504` using `args.base_dir` (the `--base-dir` flag, defaulting to `.`). In reviewer mode, `base_dir` comes from the positional arg instead. The reviewer-mode early path in `main()` must:
1. Extract `base_dir` from the positional arg (not `args.base_dir`)
2. Call `load_config(base_dir)` with that value so project-level `swival.toml` resolves correctly (the outer swival passes its project root, so the inner reviewer picks up the same project config)
3. Apply config to args as normal, then branch into `run_as_reviewer()`

This means reviewer mode inherits the outer project's config (model, provider, review_prompt, verify, etc.) unless explicitly overridden on the inner command line.

**Config inheritance hazard:** The project config may set `reviewer = "swival --reviewer-mode"`. When the inner reviewer-mode swival loads that same config, it would inherit the `reviewer` key. Fix: before config merge, snapshot whether `--reviewer` was explicitly passed on the CLI (`reviewer_from_cli = args.reviewer is not _UNSET`). After config merge, if `reviewer_from_cli` is true, error out (explicit `--reviewer` alongside `--reviewer-mode` is a user mistake). Otherwise, force `args.reviewer = None` (silently clear the config-inherited value). This is not a general "ignore config" — only the `reviewer` key is cleared, because a reviewer process cannot itself have a reviewer. All other config keys (model, provider, review_prompt, verify, etc.) apply normally.

Incompatible with `--repl`.

### 2. Shell-split `--reviewer` commands

Change `run_reviewer()` to accept full command strings, not just bare paths:

```python
# Before: only a path works
[reviewer_cmd, base_dir]

# After: full commands work too
shlex.split(reviewer_cmd) + [base_dir]
```

This lets users pass options to the reviewer without a wrapper:

```sh
# Different model for the reviewer
swival "Fix the tests" --reviewer "swival --reviewer-mode --model qwen3-coder-next"

# Custom review instructions
swival "Fix the tests" --reviewer "swival --reviewer-mode --review-prompt 'Focus on error handling'"

# Verification file with acceptance criteria
swival "Fix the tests" --reviewer "swival --reviewer-mode --verify verification/working.md"
```

**Startup validation** wraps `shlex.split()` in a try/except `ValueError` and raises `ConfigError` on malformed quoting (e.g. unbalanced quotes). Also checks for empty result (whitespace-only or empty string → `ConfigError`). After splitting, checks the first token for existence/executability using the existing `shutil.which()` / `os.access()` logic.

## Reviewer Mode Options

These options only apply when `--reviewer-mode` is active:

| Option | Description |
|--------|-------------|
| `--review-prompt TEXT` | Custom instructions appended to the built-in review prompt |
| `--objective FILE` | Read the task description from a file instead of `SWIVAL_TASK` env var |
| `--verify FILE` | Read verification/acceptance criteria from a file and include them in the review prompt |

All existing model/provider options (`--model`, `--provider`, `--base-url`, `--api-key`, `--temperature`) work normally in reviewer mode since they control the LLM call.

**Config file support:** `review_prompt`, `objective`, and `verify` keys in `swival.toml`. **Not** `reviewer_mode` — that stays CLI-only to prevent a config file from accidentally turning normal runs into reviewer mode (config values auto-apply to CLI args when unset via `apply_config_to_args`, so a project config with `reviewer_mode = true` would silently break standard usage).

**Path resolution for `--objective` and `--verify`:** Relative paths resolve against `base_dir` (the first positional arg in reviewer mode, which is always the project root passed by the outer swival). This matches how the agent's own file tools resolve paths and keeps behavior reproducible regardless of the reviewer process's CWD. In config files, these paths resolve against the config directory, consistent with how `reviewer`, `allowed_dirs`, and `skills_dir` paths already resolve (see `config.py:160-174`).

## Config Path Resolution for Reviewer Commands

Currently `config.py:170-174` treats the `reviewer` value as a file path:

```python
if "reviewer" in config:
    r = Path(config["reviewer"]).expanduser()
    if not r.is_absolute():
        r = config_dir / config["reviewer"]
    config["reviewer"] = str(r)
```

This mangles command strings like `"swival --reviewer-mode"` — `Path("swival --reviewer-mode").expanduser()` treats the whole thing as a filename.

**Fix:** Split with `shlex.split()`, resolve only the first token (the executable) if it looks like a path, then rejoin. A token is "path-like" if it starts with `/`, `./`, `../`, or `~` — these get expanded/resolved. Bare command names like `swival` are left untouched (they resolve via `PATH` at runtime, not the filesystem). This matches how shells distinguish `./foo` (filesystem lookup) from `foo` (PATH lookup).

```python
import re

_PATH_LIKE = re.compile(r"^(?:[/~]|\.\.?/)")

if "reviewer" in config:
    try:
        parts = shlex.split(config["reviewer"])
    except ValueError as e:
        raise ConfigError(f"{source}: malformed reviewer command: {e}")
    if not parts:
        raise ConfigError(f"{source}: reviewer command is empty")
    exe = parts[0]
    if _PATH_LIKE.match(exe):
        expanded = Path(exe).expanduser()
        if expanded.is_absolute():
            parts[0] = str(expanded)
        else:
            parts[0] = str(config_dir / exe)
    config["reviewer"] = shlex.join(parts)
```

The regex `_PATH_LIKE` only matches tokens that explicitly reference the filesystem. A bare name like `swival` does **not** match, so it passes through unchanged. This prevents `config_dir / "swival"` from producing a bogus path like `/home/user/project/swival`.

Existing tests (`test_reviewer_relative_resolves`, `test_reviewer_home_expansion`) must be updated to use `shlex.join`-compatible expectations, and new tests added for multi-token commands like `reviewer = "swival --reviewer-mode --verify checks.md"`.

## Built-In Review Prompt

The default prompt (used when no `--review-prompt` is given):

```
You are reviewing a coding agent's work.

<task>
{task}
</task>

{verification_section}

<answer>
{answer}
</answer>

Evaluate whether the answer correctly and completely addresses the task.
{custom_instructions}

You MUST end your response with exactly one of these lines:
  VERDICT: ACCEPT
  VERDICT: RETRY

If RETRY, explain what needs to be fixed above the verdict line. Be specific and actionable.
```

Where:
- `{task}` comes from `--objective FILE` or `SWIVAL_TASK` env var
- `{verification_section}` is included only when `--verify FILE` is provided:
  ```
  <verification>
  {file contents}
  </verification>

  The answer must satisfy these verification criteria.
  ```
- `{custom_instructions}` comes from `--review-prompt`, if given
- `{answer}` is the agent's answer read from stdin

## Verdict Parsing

The LLM response is scanned for the last line matching `VERDICT: ACCEPT` or `VERDICT: RETRY` (case-insensitive). If found:
- ACCEPT → exit 0, print LLM response to stdout
- RETRY → exit 1, print LLM response to stdout (the outer swival uses it as feedback)

If no verdict is found → exit 2, print diagnostic to stderr, print LLM response to stdout.

**Stderr visibility caveat:** The outer `run_reviewer()` currently uses `capture_output=True` (`agent.py:1228`), which captures and discards reviewer stderr (`agent.py:1240` only returns stdout). This means inner-swival stderr diagnostics are silently lost today. This plan adds two changes to fix that:

1. `run_reviewer()` returns a third value: stderr text. Signature becomes `-> tuple[int, str, str]` (exit_code, stdout, stderr).
2. When verbose, `run_reviewer()` prints captured stderr to the outer process's stderr via `fmt.warning()`. When `--report` is active, stderr is recorded in the review event alongside stdout.

This keeps protocol output (stdout) clean while making reviewer diagnostics actually reachable.

## Usage Examples

### Minimal (no wrapper script at all)

```sh
swival "Refactor error handling in src/api.py" \
    --reviewer "swival --reviewer-mode"
```

### Different model for review

```sh
swival "Fix the failing tests" \
    --model local/qwen-coder \
    --reviewer "swival --reviewer-mode --model qwen3-coder-next"
```

### Project with verification file

```sh
swival "Write a Python HTTP server on port 5000" \
    --reviewer "swival --reviewer-mode --verify verification/working.md"
```

### With custom review focus

```sh
swival "Add pagination to the API" \
    --reviewer "swival --reviewer-mode --review-prompt 'Verify that all endpoints return proper Link headers and that page_size defaults to 20'"
```

### In CI with report

```sh
swival "Fix the security vulnerability in auth.py" \
    --allowed-commands python3,pytest \
    --reviewer "swival --reviewer-mode --verify ci/security-criteria.md" \
    --report results.json \
    --quiet
```

### Project config (`swival.toml`)

```toml
reviewer = "swival --reviewer-mode"
verify = "verification/working.md"
review_prompt = "Focus on correctness and test coverage"
```

The `reviewer` value is shell-split; the first token (`swival`) is resolved via PATH, remaining tokens are preserved. The `verify` path resolves relative to the config directory (consistent with `allowed_dirs` etc.).

## Implementation Steps

### Step 1: Shell-split reviewer commands

**Files:** `swival/agent.py`, `swival/config.py`, `swival/report.py`, `tests/test_reviewer.py`, `tests/test_config.py`, `tests/test_report.py`

- Change `run_reviewer()` to use `shlex.split(reviewer_cmd) + [base_dir]`
- Change `run_reviewer()` return type to `tuple[int, str, str]` (exit_code, stdout, stderr); update all call sites
- When verbose, print captured stderr via `fmt.warning()`
- Update `ReportCollector.record_review()` to accept and store a `stderr` field; update `tests/test_report.py` for the new field
- Wrap `shlex.split()` at startup in try/except `ValueError`, raise `ConfigError` on malformed quoting
- Update startup validation to split first, then check `parts[0]` for existence/executability
- Update `config.py:170-174` to split the reviewer value, resolve only path-like first tokens (matching `_PATH_LIKE` regex: starts with `/`, `./`, `../`, or `~`), leave bare command names untouched, rejoin with `shlex.join()`
- Update existing config tests to expect `shlex.join`-compatible output
- Add tests for: multi-word commands, bare command names preserved, path-like tokens resolved, malformed quoting error, empty/whitespace-only command error

### Step 2: Reviewer mode core

**Files:** `swival/reviewer.py` (new), `swival/agent.py`, `swival/config.py`

- Add `--reviewer-mode` flag to argument parser (CLI-only, **not** added to `CONFIG_KEYS` or `_ARGPARSE_DEFAULTS`)
- Add `--review-prompt`, `--objective`, `--verify` to argument parser with `default=_UNSET` (not `None` — the `_UNSET` sentinel is required for config merge in `apply_config_to_args()` to work; if the default were `None`, the config value would never apply because `_is_unset()` checks for the sentinel)
- Add `review_prompt`, `objective`, `verify` to `CONFIG_KEYS` (type `str`) and `_ARGPARSE_DEFAULTS` (default `None` — this is the *fallback* default applied during the sweep phase after config merge, not the parser default)
- Add path resolution for `objective` and `verify` in config loading (same pattern as other path keys)
- Create `swival/reviewer.py` with `run_as_reviewer()`:
  - Read base_dir from positional arg
  - Read answer from stdin
  - Read task from `--objective` file (resolved relative to base_dir) or `SWIVAL_TASK` env var; exit 2 if neither available
  - Read verification from `--verify` file (resolved relative to base_dir) if given
  - Build review prompt
  - Call LLM (reuse existing `call_llm` / provider resolution machinery)
  - Parse verdict (last `VERDICT:` line, case-insensitive)
  - Exit with appropriate code; diagnostics to stderr, LLM response to stdout
- Wire into `main()` with this exact sequence when `--reviewer-mode` is set:
  1. Reinterpret `args.question` as `base_dir` (exactly one positional arg required, error if missing)
  2. **Before config merge:** snapshot `reviewer_from_cli = args.reviewer is not _UNSET` (True means the user explicitly passed `--reviewer` on the command line)
  3. Run config loading with that base_dir, apply config to args
  4. **After config merge:** if `reviewer_from_cli`, error out (`--reviewer-mode` + explicit CLI `--reviewer` is a user mistake). Otherwise, force `args.reviewer = None` (silently clear the value inherited from project config — see "Config inheritance hazard")
  5. Call `run_as_reviewer()` and `sys.exit()` before normal flow
- Validate incompatibility: `--reviewer-mode` + `--repl`
- Validate: exactly one positional arg required in reviewer mode (it's the base_dir, not a question)

### Step 3: Tests

**Files:** `tests/test_reviewer_mode.py` (new)

- Unit tests for `run_as_reviewer()`:
  - Reads stdin and env vars correctly
  - `--objective` file overrides `SWIVAL_TASK`
  - `--verify` file content included in prompt
  - Custom `--review-prompt` appended
  - ACCEPT verdict → exit 0, LLM response on stdout
  - RETRY verdict → exit 1, LLM response on stdout
  - No verdict → exit 2, diagnostic on stderr, LLM response on stdout
  - Missing `SWIVAL_TASK` and no `--objective` → exit 2 with diagnostic
  - `--objective` relative path resolves against base_dir
  - `--verify` relative path resolves against base_dir
- Integration tests:
  - Outer swival with `--reviewer "swival --reviewer-mode"` end-to-end (mocked LLM for both instances)
  - Shell-split command with extra options preserved
- Update existing reviewer tests for `shlex.split` changes and new `run_reviewer()` 3-tuple return
- Test stderr forwarding: reviewer stderr captured and printed when outer is verbose
- Test stderr in report: reviewer stderr recorded in review event when `--report` active
- Test config inheritance: project config with `reviewer = "swival --reviewer-mode"` does not cause inner reviewer to fail (reviewer key silently cleared)
- Test empty reviewer command: empty string and whitespace-only both raise `ConfigError`
- Test `_UNSET` defaults: config values for `review_prompt`/`objective`/`verify` apply when CLI doesn't set them

### Step 4: Documentation

**Files:** `docs.md/reviews.md`

- Rewrite "Using Swival As The Reviewer" section to lead with `--reviewer-mode`
- Keep the old bash wrapper as a "fully custom reviewer" example below it
- Document `--review-prompt`, `--objective`, `--verify` options
- Document config file usage with command strings
- Update examples throughout
- Run `make website` to regenerate `docs/pages/` HTML

## Design Decisions

**Why `reviewer_mode` is CLI-only:** Config values auto-apply to CLI args via `apply_config_to_args()` when the CLI value is `_UNSET`. A project config with `reviewer_mode = true` would silently force every `swival` invocation into reviewer mode, breaking normal usage. The other reviewer-mode options (`review_prompt`, `objective`, `verify`) are safe in config because they're no-ops without `--reviewer-mode`.

**Why not `--self-review`:** A `--self-review` flag on the outer swival that skips the subprocess entirely would be simpler but breaks the current architecture — the reviewer is a separate process with its own timeout and failure isolation. Reviewer crashes never kill the outer agent. We could add `--self-review` later as sugar that internally sets `--reviewer "swival --reviewer-mode"`.

**Why exit-2 diagnostics go to stderr:** The outer swival captures reviewer stdout for the report's `feedback` field. Mixing diagnostics into stdout pollutes that field. The inner reviewer writes diagnostics to stderr; the outer `run_reviewer()` is updated to forward captured stderr (see "Stderr visibility caveat" in Verdict Parsing section) so these diagnostics are actually visible rather than silently dropped.

**Outer `--objective` is out of scope:** Reading the agent task from a file (`--objective`) on the *outer* swival is a useful feature but separate from this plan. This plan only covers `--objective` in the context of reviewer mode, where it provides the task description to the judge.

**Token cost:** Reviewer mode makes one LLM call per review round (answer passed in the prompt, not via tools). Up to 5 rounds max. Predictable and bounded.
