<!-- STATUS: COMPLETE -->

# Phase 15: Pipeline Output Quality

## Goal

Fix three critical bugs that prevent the mission pipeline from producing usable output: (1) validated data is discarded before persistence, (2) chain-mission links break because chains reference Film Noir titles instead of `title_plain`, (3) JSON truncation recovery drops all chain data. Secondary: reduce validation warning volume via prompt improvements.

## Dependencies

- Phase 14 (provider-agnostic factory) — complete
- Phase 13 (campaign orchestration) — `persist_pipeline_data()` is the primary consumer
- Phase 10 (pipeline stabilization) — truncation recovery in `structured.py`

## Research Notes

### Bug 1: Validated data not wired into persistence

**Current flow (pipeline.py:146-173):**
1. `generate_category_missions_multi()` returns `CategoryMissionsResponse` with `.missions` and `.chains`
2. `validate_category_missions()` returns `ValidationResult` with cleaned `.missions` and `.chains`
3. Pipeline stores raw response in `result.category_missions[name]` (line 167)
4. Pipeline stores validation in `result.category_validations[name]` (line 168)
5. `persist_pipeline_data()` reads from `result.category_missions` — the raw, unvalidated data

**Chosen approach: Option C — Replace missions/chains in the response object with validated versions.**

This is the simplest fix. After validation, overwrite `response.missions` and `response.chains` with the validated versions before storing in `result.category_missions`. The persister's interface stays unchanged. No signature changes needed.

Why not Option A (replace in result dict): Requires creating a new `CategoryMissionsResponse` or mutating it, which is the same as Option C.
Why not Option B (consume validations in persister): Would require changing persister to understand `ValidationResult`, coupling it to the validation layer.

### Bug 2: Chain `mission_titles` uses Film Noir titles

**Root cause:** The LLM generates chain `mission_titles` using the Film Noir `title` field instead of `title_plain`. The schema field description says "Ordered list of title_plain values" but LLMs (especially via JSON text fallback) don't follow this consistently.

**Three-layer fix:**
1. **Prompt reinforcement** (prompts.py) — Add explicit instruction that chain `mission_titles` MUST use `title_plain`, not the Film Noir `title`.
2. **Validation-layer normalization** (validation.py `_validate_chains`) — Build a mapping from `title` → `title_plain` and normalize chain titles before validation. This catches the LLM mistake at the earliest point.
3. **Defensive matching in persister** (persister.py `_persist_category_chains`) — Build a secondary `title` → mission_id lookup as fallback. Belt-and-suspenders defense.

### Bug 3: Truncation recovery drops chains

**Current behavior (structured.py `_recover_truncated_json` lines 593-694):**
After finding the last complete mission object, the recovery always appends `', "chains": []}'` regardless of whether chain data exists. Even the "preserve chains" code path (lines 663-677) always writes `"chains": []`.

**Fix approach:**
1. After recovering missions, scan remaining text for complete chain objects using the same brace-depth parser
2. If complete chain objects found, include them in the recovered JSON
3. If chains were lost (partial/no chain data), log a warning and set `chains: []` (current behavior, but now intentional)

Similarly fix `_recover_truncated_json_with_info` which has the same issue.

### Prompt Improvements (Secondary)

After examining the current `MISSION_SYSTEM_PROMPT` and `MISSION_HUMAN_PROMPT`:
- Add explicit constraint that `mission_titles` in chains MUST match `title_plain` exactly
- Add note that `title_plain` should be descriptive enough to be unambiguous
- Reinforce that file paths must come from the provided context (reduce hallucinated paths)

### Phase 14 Review Fold-In Items

Per `phase_shift_requirements_phase15.md`:
- Verify Phase 14 test assertions for new defaults (glm-5, api.z.ai URLs) → checked during research, tests use correct values
- Verify `supports_tool_calling` flag routes correctly → confirmed: GLM has `supports_tool_calling=False`, `_should_use_structured_output` checks it
- Check for dead `ChatZhipuAI` code → no remaining references found

---

## Tasks

### Task 1: Wire validated data into persistence (Bug 1) — DONE

**File:** `src/unfamiliar/llm/missions/pipeline.py`

**Action:** After validation, replace the response's missions and chains with the validated versions before storing in `result.category_missions`.

**Changes:**
- After `validate_category_missions()` (line 164), add:
  ```python
  response.missions = validation.missions
  response.chains = validation.chains
  ```
- This means `result.category_missions[category_name]` now contains validated data
- The persister reads validated missions and chains without any interface change

**Verify:** Unit test — mock `generate_category_missions_multi` to return missions with bad prerequisites and out-of-range line numbers. Run through pipeline. Assert that the `PipelineResult.category_missions` contains the validated (fixed) data, not the raw LLM data.

**Done when:** `result.category_missions[name].missions` contains the post-validation missions (cleaned prerequisites, clamped lines), not the raw LLM output.

---

### Task 2: Add chain title normalization to validation (Bug 2, Layer 1) — DONE

**File:** `src/unfamiliar/llm/missions/validation.py`

**Action:** In `_validate_chains()`, build a mapping from Film Noir `title` → `title_plain` and normalize chain `mission_titles` before checking membership.

**Changes:**
- Build `title_to_plain = {m.title: m.title_plain for m in missions}` alongside existing `mission_titles = {m.title_plain for m in missions}`
- For each title in `chain.mission_titles`, first check if it's already in `mission_titles` (already a `title_plain`). If not, check `title_to_plain` and normalize it. Log a warning when normalization is applied.
- Add a `fixes_applied` list return (currently `_validate_chains` doesn't return fixes) — or append to warnings

**Verify:** Unit test — create missions with `title="The Factory Floor"` and `title_plain="Understand Factory Pattern"`. Create a chain with `mission_titles=["The Factory Floor"]`. Run `_validate_chains()`. Assert the chain's `mission_titles` is normalized to `["Understand Factory Pattern"]`.

**Done when:** `_validate_chains` normalizes Film Noir titles to `title_plain` values automatically.

---

### Task 3: Add defensive title matching to persister (Bug 2, Layer 2) — DONE

**File:** `src/unfamiliar/campaign/persister.py`

**Action:** In `_persist_category_chains()`, build a secondary lookup from `title` (Film Noir) → mission_id as fallback.

**Changes:**
- `_persist_category_missions` already returns `mission_id_map` keyed on `title_plain`
- In `_persist_category_chains`, accept an additional parameter `missions: list[MissionItemResponse]` to build the fallback map
- Build `title_fallback = {m.title: mission_id_map.get(m.title_plain) for m in missions}` (Film Noir title → mission_id)
- When `mission_id_map.get(title)` returns None, try `title_fallback.get(title)` before logging "unknown mission"
- Log a warning when fallback matching is used

**Verify:** Unit test — call `_persist_category_chains` with a chain that uses Film Noir titles. Assert chain-mission links are created via fallback matching.

**Done when:** Chain linking falls back to Film Noir title matching when `title_plain` lookup fails.

---

### Task 4: Reinforce chain title constraint in prompts (Bug 2, Layer 3) — DONE

**File:** `src/unfamiliar/llm/missions/prompts.py`

**Action:** Add explicit instruction that chain `mission_titles` MUST use exact `title_plain` values.

**Changes to `MISSION_SYSTEM_PROMPT`:**
- In the "## Output Requirements" section, add a bullet: `- **Chain mission_titles**: Must use the exact `title_plain` string from the corresponding mission, NOT the Film Noir `title`
- In the chains description area, reinforce: "IMPORTANT: The mission_titles array in each chain must contain the exact title_plain values, not the Film Noir themed titles."

**Changes to `MISSION_HUMAN_PROMPT`:**
- In the "Remember:" section, add: `- Chain mission_titles must use exact title_plain values, not Film Noir titles`

**Verify:** Read the updated prompt text and confirm the instruction is clear and placed where the LLM will see it during chain generation.

**Done when:** Both system and human prompts explicitly instruct the LLM to use `title_plain` in chain `mission_titles`.

---

### Task 5: Improve truncation recovery to preserve chains (Bug 3) — DONE

**File:** `src/unfamiliar/llm/structured.py`

**Action:** Update both `_recover_truncated_json` and `_recover_truncated_json_with_info` to attempt chain recovery instead of always writing `"chains": []`.

**Changes to `_recover_truncated_json` (line 593):**
- After finding the last complete mission (`last_complete_end`), check for chain data
- If `"chains"` key exists in the JSON text after missions, attempt to find complete chain objects using the same brace-depth parser
- If complete chains found, include them: `', "chains": [<recovered chains>]}'`
- If no complete chains found, fall back to `', "chains": []}'` with a logged warning
- Extract chain recovery into a helper `_recover_chains_from_text(json_text, search_start)` shared by both recovery functions

**Changes to `_recover_truncated_json_with_info` (line 115):**
- Same chain recovery logic via the shared helper

**Verify:** Unit test — create truncated JSON with 3 complete missions, 2 complete chains, and a partial 3rd chain. Run `_recover_truncated_json()`. Assert recovered JSON includes the 2 complete chains. Also test with truncated JSON where chains section is completely missing.

**Done when:** Truncation recovery preserves complete chain objects instead of always dropping them.

---

### Task 6: Wire fallback missions list through persister chain linking — DONE

**Files:** `src/unfamiliar/campaign/persister.py`

**Action:** Pass the missions list to `_persist_category_chains` so it can build the Film Noir title fallback map. Update `persist_pipeline_data` to pass `category_missions.missions` through.

**Changes:**
- Add `missions: list[MissionItemResponse]` parameter to `_persist_category_chains`
- In `persist_pipeline_data`, pass `category_missions.missions` when calling `_persist_category_chains`
- Build the fallback map inside `_persist_category_chains` as designed in Task 3

**Verify:** Full flow test — the combination of Tasks 2, 3, and 6 should handle the Film Noir title problem at both the validation and persistence layers.

**Done when:** `_persist_category_chains` receives mission data and can build fallback lookups.

---

### Task 7: Update pipeline prerequisite resolution to use validated data — DONE

**File:** `src/unfamiliar/campaign/persister.py`

**Action:** Verify that `_resolve_all_prerequisites` also benefits from Task 1's fix (validated data in `category_missions`). Since it reads from `pipeline_result.category_missions`, and Task 1 ensures those contain validated missions (with cleaned prerequisites), this should already work. Verify and add a test.

**Verify:** Unit test — mock a pipeline result where validation removed a bad prerequisite. Assert that `_resolve_all_prerequisites` does NOT try to create a link for the removed prerequisite.

**Done when:** Prerequisite resolution only processes validated (cleaned) prerequisite lists.

---

### Task 8: Verify all existing tests pass and run linting — DONE

**Action:** Run the full test suite and linting checks to ensure no regressions.

**Verify:**
- `pytest tests/ -v --tb=short` — all 903 tests pass
- `ruff check src/unfamiliar/` — no lint errors

**Done when:** All tests pass, no lint errors.

---

## Interface Contracts

### Modified: `_validate_chains` (validation.py)

```python
def _validate_chains(
    missions: list[MissionItemResponse],
    chains: list[ChainResponse],
) -> tuple[list[ChainResponse], list[str]]:
```

**New behavior:**
- Builds `title_to_plain = {m.title: m.title_plain for m in missions}`
- For each title in `chain.mission_titles`: if title not in `mission_titles` set but IS in `title_to_plain`, normalize to the `title_plain` value
- Warnings include "Normalized Film Noir title '...' to title_plain '...'" for each normalization
- All other existing behavior unchanged

**Invariants:**
- Output chain `mission_titles` always contain `title_plain` values (never Film Noir titles)
- Chains with < 2 valid missions after normalization are still dropped
- Original validation logic (missing mission check, minimum 2 missions) still runs after normalization

**Edge cases:**
- Chain title matches both a `title_plain` AND a Film Noir `title` of a different mission → `title_plain` match takes priority (already in `mission_titles` set)
- Chain title matches neither → logged as missing, excluded from chain
- Empty chains after normalization → dropped with warning

### Modified: `_persist_category_chains` (persister.py)

```python
def _persist_category_chains(
    db: CampaignDB,
    campaign_id: int,
    category: InvestigationCategory,
    chains: list[ChainResponse],
    mission_id_map: dict[str, int],
    warnings: list[str],
    missions: list[MissionItemResponse] | None = None,  # NEW
) -> int:
```

**New behavior:**
- When `missions` is provided, builds `title_fallback = {m.title: mission_id_map.get(m.title_plain) for m in missions if m.title_plain in mission_id_map}`
- For each chain title: first tries `mission_id_map.get(title)`, then `title_fallback.get(title)` if missions provided
- Logs warning when fallback matching succeeds: "Chain linked via Film Noir title fallback: '...'"
- All other existing behavior unchanged

**Invariants:**
- `mission_id_map` primary lookup always tried first
- Fallback only used when primary returns None and `missions` is provided
- If both primary and fallback miss, existing "unknown mission" warning behavior unchanged
- Backward compatible: `missions=None` means no fallback (existing behavior)

### Modified: `_recover_truncated_json` (structured.py)

**New behavior:**
- After finding `last_complete_end` for missions, scans text after missions for a `"chains"` key
- If found, applies same brace-depth parser to extract complete chain objects
- Returns recovered JSON with actual chain data when available
- Falls back to `"chains": []` only when no complete chains found
- Logs "Recovered N chains from truncated response" or "No complete chains recoverable"

**Invariants:**
- Mission recovery behavior unchanged
- Chain recovery is additive — never degrades mission recovery
- Recovered JSON always valid (json.loads check still runs)
- Partial/incomplete chain objects are dropped (only complete objects preserved)

### Modified: `_recover_truncated_json_with_info` (structured.py)

Same chain recovery behavior as `_recover_truncated_json`, applied via shared helper function `_recover_chains_from_text`.
