<!-- STATUS: COMPLETE -->

## At a Glance

**Fixes three critical bugs preventing the mission pipeline from producing usable output: validated data now flows to persistence, chain-mission links work correctly, and truncation recovery preserves chain data.**

### Files Changed
- `src/unfamiliar/llm/missions/pipeline.py` — Wires validated missions/chains into the result object before persistence
- `src/unfamiliar/llm/missions/validation.py` — Normalizes Film Noir titles to `title_plain` in chain validation
- `src/unfamiliar/campaign/persister.py` — Adds Film Noir title fallback lookup for chain linking
- `src/unfamiliar/llm/missions/prompts.py` — Reinforces `title_plain` constraint for chain `mission_titles`
- `src/unfamiliar/llm/structured.py` — Preserves complete chains during JSON truncation recovery

### Key Functions Added/Modified
- `run_mission_pipeline()` — Replaces response.missions/chains with validated versions before storing
- `_validate_chains()` — Builds `title_to_plain` map and normalizes Film Noir titles automatically
- `_persist_category_chains()` — New `missions` parameter enables Film Noir title fallback lookup
- `_find_last_complete_object()` — Shared helper for brace-depth JSON object extraction
- `_recover_chains_from_text()` — Extracts complete chain objects from truncated JSON
- `_recover_truncated_json()` — Now preserves chains instead of always writing `"chains": []`
- `_recover_truncated_json_with_info()` — Same chain recovery via shared helper

### Behavior Changes
- Pipeline result now contains validated (cleaned) data, not raw LLM output
- Chains with Film Noir titles in `mission_titles` are automatically normalized to `title_plain`
- Chain-mission links are created even when LLM uses Film Noir titles
- Truncated JSON responses preserve complete chain objects instead of dropping all chains
- Prompt explicitly instructs LLM to use `title_plain` in chain references

---

## What Was Built

This phase addressed three bugs discovered during the first successful pipeline run (Phase 14):

### Bug 1: Validated Data Discarded

The validation layer ran correctly but its results were stored in a separate dict (`category_validations`). The persister read from `category_missions`, which still contained raw LLM output with potentially bad prerequisites and out-of-range line numbers.

**Fix:** After validation, overwrite `response.missions` and `response.chains` with the validated versions before storing in `result.category_missions`. The persister's interface stays unchanged.

### Bug 2: Chain Links Broken

Chains referenced missions using Film Noir titles (e.g., "The Factory Floor") instead of `title_plain` (e.g., "Understand Factory Pattern"). The persister's lookup by `title_plain` failed.

**Three-layer fix:**
1. **Prompt reinforcement** — Explicit instruction that `mission_titles` MUST use `title_plain`
2. **Validation normalization** — Build `title` → `title_plain` map and auto-normalize
3. **Persistence fallback** — Secondary lookup by Film Noir title when `title_plain` misses

### Bug 3: Truncation Drops Chains

The truncation recovery code always appended `', "chains": []}'` regardless of whether chain data existed in the truncated text.

**Fix:** After recovering missions, scan remaining text for a `"chains"` key. If found, use the same brace-depth parser to extract complete chain objects. Only fall back to empty chains if no complete chains exist.

## Key Decisions

1. **Option C for Bug 1** — Replace missions/chains in the response object rather than changing persister interface. Simplest fix with no signature changes.

2. **Three-layer defense for Bug 2** — Prompt + validation + persistence. No single layer is sufficient because LLMs don't always follow instructions and JSON text fallback bypasses schema validation.

3. **Shared helper for chain recovery** — Extract `_find_last_complete_object()` and `_recover_chains_from_text()` to avoid code duplication between `_recover_truncated_json` and `_recover_truncated_json_with_info`.

## Major Logic Flows In This Phase

### Validated Data Flow

```
generate_category_missions_multi() → response (raw)
    ↓
validate_category_missions() → validation (cleaned)
    ↓
response.missions = validation.missions  ← NEW
response.chains = validation.chains      ← NEW
    ↓
result.category_missions[cat] = response
    ↓
persist_pipeline_data() reads validated data
```

### Chain Title Normalization (Validation Layer)

```
_validate_chains(missions, chains)
    ↓
Build title_to_plain = {m.title: m.title_plain for m in missions}
    ↓
For each chain.mission_titles:
    if title in mission_titles: keep as-is (already title_plain)
    elif title in title_to_plain: normalize, log warning
    else: mark as missing
    ↓
Return chains with normalized titles
```

### Chain Title Fallback (Persistence Layer)

```
_persist_category_chains(..., missions=missions)
    ↓
Build title_fallback = {m.title: mission_id for m in missions}
    ↓
For each chain title:
    mission_id = mission_id_map.get(title)  # Primary: title_plain
    if None: mission_id = title_fallback.get(title)  # Fallback: Film Noir
    if found via fallback: log warning
```

### Truncation Chain Recovery

```
_recover_truncated_json(text, start_pos)
    ↓
Find last complete mission object
    ↓
Call _recover_chains_from_text(json_text, search_start)
    ↓
    Find "chains" key and array start
    Use _find_last_complete_object() to find complete chains
    Return chain JSON text or "" if none found
    ↓
Build recovered JSON with chains if available, else empty chains
```

## Connection to Previous Phases

- **Phase 10** — Introduced truncation recovery; this phase fixes its chain handling
- **Phase 13** — Campaign persistence layer; this phase extends `_persist_category_chains`
- **Phase 14** — Provider-agnostic factory; first successful pipeline run revealed these bugs

---

## Review Fix

During review, a UX issue was discovered in the CLI output:
- Original: "0/46 missions available" showed `completed/total` but worded confusingly
- Fixed: Now shows clear breakdown of available/locked/completed missions

**Additional review fixes:**
1. Line clamping warnings moved to DEBUG level (reduced noise)
2. Added "Conservative line numbers" instruction to prompts
3. Added "VALID FILE PATHS" section to mission context — explicit list of all valid paths

**Files changed:** `src/unfamiliar/cli.py`, `src/unfamiliar/llm/missions/pipeline.py`, `src/unfamiliar/llm/missions/prompts.py`, `src/unfamiliar/llm/missions/context.py`
