# Phase 7: LLM Integration — Mission Generation, Validation, and Checkpointing

## Goal

Build the mission generation pipeline: per-category LLM calls that produce structured missions with file references, prerequisites, and chain groupings — wrapped in a LangGraph StateGraph with SQLite checkpointing for fault-tolerant, resumable execution. Include programmatic validation and auto-fix for LLM hallucinations (nonexistent file paths, out-of-bounds line numbers, circular prerequisites).

## Dependencies

- **Phase 6**: `BriefingResult`, `generate_briefing()`, `build_briefing_context()`, `with_llm_retry()`, exception hierarchy, provider factory
- **Phase 3–4**: `AnalysisResult` (static analysis output — files, modules, entry points, security results)
- **Phase 2**: Pydantic models (`Mission`, `MissionFileRef`, `MissionChain`, `ModuleMissionLink`, `ChainMissionLink`, `MissionPrerequisite`), DB repositories
- **Phase 1**: Config system, constants (`InvestigationCategory`, `MissionDifficulty`, `MAX_PREREQUISITE_DEPTH`, `MIN_ROOT_MISSIONS_PER_CATEGORY`)

## Research Notes

### LangGraph StateGraph + Checkpointing

- `StateGraph(TypedDict)` defines the state schema for the pipeline
- `InMemorySaver()` for testing, `SqliteSaver(conn)` for persistence
- `graph.compile(checkpointer=checkpointer)` enables checkpointing
- Resume by re-invoking with same `thread_id` config: `{"configurable": {"thread_id": "..."}}`
- State updates are additive — each node returns a partial dict that merges into the state
- `Annotated[list, add]` operator allows list accumulation across nodes
- Imports: `from langgraph.graph import StateGraph, START, END`; `from langgraph.checkpoint.memory import InMemorySaver`
- For SQLite persistence: `from langgraph.checkpoint.sqlite import SqliteSaver` (may require `langgraph-checkpoint-sqlite` package — verify at build time)

### Mission Generation Strategy (per requirements)

- **7 category-level LLM calls** (not one monolithic call) — Architecture, Domain & Business Logic, Data Flow, Dependencies, Conventions, Testing, Technical Debt
- **Category ordering** matters for cross-category prerequisites:
  1. Architecture (first — maps the territory)
  2. Data Flow (builds on architecture understanding)
  3. Domain & Business Logic (builds on data flow)
  4. Dependencies (parallel with conventions in concept, but sequential for simplicity)
  5. Conventions (after dependencies)
  6. Testing (depends on understanding conventions and architecture)
  7. Technical Debt (last — requires full context)
- Each category node receives previously generated mission titles for cross-category prerequisite awareness
- Within-category prerequisites are LLM-generated; cross-category prerequisites follow the algorithmic baseline (category ordering)

### Structured Output for Missions

- Use `.with_structured_output(MissionSetResponse)` like briefing (tool-calling based, provider-portable)
- Each LLM call returns: missions list, chain groupings, within-category prerequisite edges
- LLM references files by path (from analysis) and modules by index (from briefing modules)
- File references include line ranges — these are validated and clamped post-generation

### Validation Rules (from requirements)

- **File path validation**: Verify referenced files exist at generation time
- **Line range validation**: Clamp line numbers to actual file bounds
- **Prerequisite graph validation**: No cycles, max depth 3, every mission reachable from a root
- **Root mission validation**: Each category must have 1-2 missions with zero prerequisites
- **Schema validation**: All required fields present, valid enum values
- **Auto-fix where safe**: Clamp line ranges, break circular prerequisites (remove one edge), remove missing prerequisites
- **No LLM calls for validation** — all checks are programmatic

### Token Efficiency

- Mission context is smaller than briefing context — only category-relevant modules and files
- Reuse the briefing summary (already generated) rather than re-analyzing
- Each category call produces 3-8 missions typically

### Security Considerations

- API keys are already handled by Phase 6's provider factory — no new key handling needed
- File contents read during validation use the same UTF-8 + error handling patterns from Phase 6
- No user input flows into LLM prompts (all context is from static analysis)
- Mission descriptions may reference file paths — these are validated, not user-controlled

---

## Tasks

### Task 1: Add MissionGenerationError to exception hierarchy

**Files:** `src/unfamiliar/llm/exceptions.py`
**Action:** Add `MissionGenerationError(LLMError)` with `attempts` and `category` attributes. Follows the same pattern as `BriefingGenerationError`.
**Verify:** `ruff check src/unfamiliar/llm/exceptions.py` passes.
**Done when:** Exception class is defined, importable, and stores category + attempt count.

### Task 2: Create mission generation Pydantic schemas

**Files:** `src/unfamiliar/llm/missions/schemas.py`
**Action:** Define Pydantic models for LLM structured output:
- `MissionFileRefResponse(BaseModel)`: `file_path: str`, `line_start: int | None`, `line_end: int | None`, `description: str`
- `MissionItemResponse(BaseModel)`: `title: str`, `title_plain: str`, `description: str`, `difficulty: str` (quick/medium/deep_dive), `completion_condition: str`, `security_tag: bool`, `standard_tag: str | None`, `narrative_intro: str`, `narrative_outro: str`, `file_refs: list[MissionFileRefResponse]`, `module_index: int` (index into briefing modules list), `prerequisite_titles: list[str]` (title_plain of prerequisite missions within this category)
- `ChainResponse(BaseModel)`: `name: str`, `name_plain: str`, `description: str`, `mission_titles: list[str]` (title_plain values in order)
- `CategoryMissionsResponse(BaseModel)`: `missions: list[MissionItemResponse]`, `chains: list[ChainResponse]`

Field descriptions guide LLM output (same pattern as briefing schemas).
**Verify:** `ruff check` passes. Models can be instantiated with sample data.
**Done when:** Schema models are defined with clear field descriptions for LLM guidance.

### Task 3: Create mission generation prompt template

**Files:** `src/unfamiliar/llm/missions/prompts.py`
**Action:** Define `MISSION_PROMPT` as a `ChatPromptTemplate.from_messages()` with:
- System message: role definition (expert mission designer for codebase investigation), output expectations (structured missions for a specific category), Film Noir theming instructions for titles/narrative, difficulty guidelines (quick=10-15min, medium=20-30min, deep_dive=30-45min)
- Human message: `{category}` name, `{briefing_summary}` from Phase 6, `{category_context}` with relevant modules/files, `{previous_missions}` titles from completed categories (for cross-category prerequisite awareness), `{module_descriptions}` for module linking
- Prompt emphasizes: specific file paths and line numbers, clear completion conditions, prerequisite logic (entry points first, hub files before leaf files), 3-8 missions per category, at least 1-2 root missions (no prerequisites)
**Verify:** Template renders without error with sample inputs.
**Done when:** Prompt template produces correct message structure.

### Task 4: Build category-specific mission context

**Files:** `src/unfamiliar/llm/missions/context.py`
**Action:** Implement `build_mission_context(analysis, briefing_result, category, previous_missions, token_budget) -> str` that:
- Includes briefing summary (from BriefingResult)
- Filters modules to those relevant to this category (using `category_hints` from DetectedModule)
- Includes key file contents for relevant modules (within token budget share — budget / 7 per category)
- For TESTING category: includes test coverage info
- For all categories: includes security findings if relevant
- Lists previous mission titles (from completed categories) for cross-category prerequisite awareness
- Lists module descriptions with indices so LLM can reference them by `module_index`
**Verify:** `ruff check` passes. Function produces readable output for each category.
**Done when:** Context builder produces category-focused LLM context.

### Task 5: Implement per-category mission generator

**Files:** `src/unfamiliar/llm/missions/generator.py`
**Action:** Implement `generate_category_missions(analysis, briefing_result, category, model, previous_missions, token_budget) -> CategoryMissionsResponse` that:
- Calls `build_mission_context()` to prepare category context
- Creates chain: `MISSION_PROMPT | model.with_structured_output(CategoryMissionsResponse)`
- Invokes with retry handling via `with_llm_retry()`
- Returns raw `CategoryMissionsResponse` from LLM
- Raises `MissionGenerationError` on permanent failure (includes category name and attempt count)
**Verify:** `ruff check` passes.
**Done when:** Generator produces structured mission data for a single category via LLM.

### Task 6: Implement mission validation and auto-fix

**Files:** `src/unfamiliar/llm/missions/validation.py`
**Action:** Implement validation functions:
- `validate_file_refs(file_refs, repo_path) -> list[FileRefValidation]`: Check each file path exists, clamp line ranges to actual file bounds, compute file hash. Returns validated refs with warnings.
- `validate_prerequisites(missions, category) -> list[PrerequisiteValidation]`: Build DAG from prerequisite_titles, detect cycles (break by removing one edge), verify max depth <= `MAX_PREREQUISITE_DEPTH` (trim if exceeded), ensure all missions reachable from a root, verify at least `MIN_ROOT_MISSIONS_PER_CATEGORY` root missions.
- `validate_category_missions(missions_response, repo_path) -> ValidationResult`: Orchestrates all validations. Returns `ValidationResult` with validated missions, warnings list, and auto-fix actions taken.
- `ValidationResult` dataclass: `missions: list[MissionItemResponse]`, `warnings: list[str]`, `fixes_applied: list[str]`, `valid: bool`
**Verify:** `ruff check` passes.
**Done when:** Validation catches bad file paths, out-of-range lines, circular prereqs, and missing roots — auto-fixing where safe.

### Task 7: Build LangGraph pipeline with checkpointing

**Files:** `src/unfamiliar/llm/missions/pipeline.py`
**Action:** Implement the LangGraph StateGraph pipeline:
- Define `MissionPipelineState(TypedDict)` with: `completed_categories: list[str]`, `category_missions: dict[str, CategoryMissionsResponse]`, `category_validations: dict[str, ValidationResult]`, `errors: list[str]`, `current_category: str`
- Define `CATEGORY_ORDER` list: Architecture, Data Flow, Domain Business Logic, Dependencies, Conventions, Testing, Technical Debt
- Create one node function `process_category(state) -> dict` that:
  - Reads `current_category` from state
  - Calls `generate_category_missions()` for the current category
  - Calls `validate_category_missions()` on the result
  - Returns state update with results added to `category_missions` and `category_validations`
- Create routing function `next_category(state) -> str` that returns the next uncompleted category or END
- Build StateGraph: START → process_category → (conditional: next_category or END)
- Implement `run_mission_pipeline(analysis, briefing_result, model, campaign_dir, token_budget) -> PipelineResult`:
  - Creates `SqliteSaver` using campaign directory for checkpoint persistence (fall back to `InMemorySaver` if sqlite checkpoint unavailable)
  - Compiles graph with checkpointer
  - Uses `thread_id` derived from campaign directory hash for resumability
  - Invokes graph, returning accumulated results
  - On resume: graph automatically skips completed categories via checkpoint state
- `PipelineResult` dataclass: `category_missions: dict`, `category_validations: dict`, `completed_categories: list[str]`, `errors: list[str]`
**Verify:** `ruff check` passes.
**Done when:** Pipeline processes all 7 categories with checkpointing, and can resume from last completed category on failure.

### Task 8: Wire up public API and exports

**Files:** `src/unfamiliar/llm/missions/__init__.py`, `src/unfamiliar/llm/__init__.py`
**Action:**
- `llm/missions/__init__.py`: Export `run_mission_pipeline`, `PipelineResult`, `CategoryMissionsResponse`, `ValidationResult`, `generate_category_missions`
- `llm/__init__.py`: Add `MissionGenerationError` to exports, add `run_mission_pipeline` and `PipelineResult`
**Verify:** `from unfamiliar.llm import run_mission_pipeline, MissionGenerationError` works.
**Done when:** Clean public API surface with all new exports.

### Task 9: Lint and type-check all new files

**Files:** All files created/modified in tasks 1–8
**Action:** Run `ruff check src/unfamiliar/llm/` on all new and modified files. Fix any issues. Validate syntax via `py_compile`.
**Verify:** Zero errors from ruff. All files compile successfully.
**Done when:** All new code passes linting and syntax validation.

---

## Interface Contracts

### `generate_category_missions`

```python
def generate_category_missions(
    analysis: AnalysisResult,
    briefing_result: BriefingResult,
    category: InvestigationCategory,
    model: BaseChatModel,
    previous_missions: list[str] = [],
    token_budget: int = DEFAULT_TOKEN_BUDGET,
) -> CategoryMissionsResponse:
```

**Purpose:** Generate missions for a single investigation category via one LLM call.

**Invariants:**
- Makes exactly 1 LLM call (plus retries on transient failure)
- Returns structured output matching `CategoryMissionsResponse`
- All missions in the response belong to the specified `category`
- At least 1 mission is returned per category

**Valid inputs:**
- `analysis`: Complete `AnalysisResult` from `analyze_repository()`
- `briefing_result`: Complete `BriefingResult` from `generate_briefing()`
- `category`: Any `InvestigationCategory` enum value
- `model`: Any `BaseChatModel` instance
- `previous_missions`: title_plain values from previously completed categories (empty for first category)
- `token_budget`: Positive integer (total budget, divided by 7 per category)

**Error conditions:**
- `AuthenticationError` — 401/403 from provider (no retry, immediate)
- `MissionGenerationError` — LLM call failed after max retries (includes category name)
- `MissionGenerationError` — structured output parsing failed

**Edge cases:**
- Very small codebase (< 5 files) → fewer missions generated, still valid
- Category with no relevant modules → LLM generates broader missions using available context
- Empty `previous_missions` (first category) → no cross-category prerequisite awareness, which is correct
- LLM returns 0 missions → raise `MissionGenerationError` (at least 1 required)

---

### `validate_file_refs`

```python
def validate_file_refs(
    file_refs: list[MissionFileRefResponse],
    repo_path: Path,
) -> tuple[list[MissionFileRefResponse], list[str]]:
```

**Purpose:** Validate file references from LLM output against the actual filesystem.

**Invariants:**
- Never raises — returns warnings instead
- File paths that don't exist are removed (mission hidden from available list per requirements)
- Line ranges are clamped to actual file bounds (auto-fix)
- File hashes are computed for valid files

**Valid inputs:**
- `file_refs`: List of file references from LLM output
- `repo_path`: Absolute path to the repository root

**Error conditions:**
- None — all issues produce warnings, not exceptions

**Edge cases:**
- Empty `file_refs` → returns empty list and no warnings
- All file paths invalid → returns empty list with warnings for each
- `line_start > line_end` → swaps them (auto-fix, logged as warning)
- `line_start > file_line_count` → clamps to last line (auto-fix)
- `line_start = 0` → set to 1 (auto-fix, lines are 1-indexed)
- Binary files or unreadable files → skip with warning

---

### `validate_prerequisites`

```python
def validate_prerequisites(
    missions: list[MissionItemResponse],
    category: InvestigationCategory,
) -> tuple[list[MissionItemResponse], list[str]]:
```

**Purpose:** Validate and fix prerequisite relationships within a category's mission set.

**Invariants:**
- No circular dependencies in output (cycles broken by removing one edge)
- Max prerequisite depth <= `MAX_PREREQUISITE_DEPTH` (3) — excess edges trimmed
- At least `MIN_ROOT_MISSIONS_PER_CATEGORY` (1) missions with zero prerequisites
- Every mission reachable from at least one root mission
- All `prerequisite_titles` reference existing missions within the set

**Valid inputs:**
- `missions`: List of mission items from LLM output for one category
- `category`: The investigation category being validated

**Error conditions:**
- None — all issues produce warnings and auto-fixes

**Edge cases:**
- Empty `missions` list → returns empty list and warning "no missions generated"
- All missions have prerequisites (no roots) → remove prerequisites from the simplest (lowest difficulty) mission to create a root
- A prerequisite_title references a non-existent mission → remove that prerequisite (auto-fix)
- Single mission → always valid (it's a root by definition)
- Depth exactly 3 → valid. Depth 4 → trim the deepest edge

---

### `validate_category_missions`

```python
def validate_category_missions(
    response: CategoryMissionsResponse,
    repo_path: Path,
) -> ValidationResult:
```

**Purpose:** Orchestrate all validation passes on a category's generated missions.

**Invariants:**
- Always returns a `ValidationResult`, never raises
- `valid` is True if no critical issues remain after auto-fix
- `warnings` list is comprehensive — every auto-fix and every issue is logged
- `fixes_applied` list documents what was changed

**Valid inputs:**
- `response`: Complete `CategoryMissionsResponse` from LLM
- `repo_path`: Absolute path to the repository root

**Error conditions:**
- None — validation is always safe

**Edge cases:**
- LLM returns empty mission list → `valid=False`, warning "no missions generated for category"
- All file refs invalid → missions remain but with empty file refs + warnings
- Chain references non-existent mission title → chain is dropped with warning

---

### `run_mission_pipeline`

```python
def run_mission_pipeline(
    analysis: AnalysisResult,
    briefing_result: BriefingResult,
    model: BaseChatModel,
    campaign_dir: Path,
    token_budget: int = DEFAULT_TOKEN_BUDGET,
) -> PipelineResult:
```

**Purpose:** Execute the full 7-category mission generation pipeline with LangGraph checkpointing.

**Invariants:**
- Processes categories in defined order (Architecture first, Technical Debt last)
- Each completed category is checkpointed — resume skips completed categories
- Thread ID is deterministic from `campaign_dir` (same dir = same thread = resume)
- Partial results are available — completed categories are in `PipelineResult` even if later categories fail
- Never makes more than 7 LLM calls (one per category, plus retries)

**Valid inputs:**
- `analysis`: Complete `AnalysisResult`
- `briefing_result`: Complete `BriefingResult`
- `model`: Any `BaseChatModel` instance
- `campaign_dir`: Path to campaign storage directory (for checkpoint SQLite file)
- `token_budget`: Positive integer

**Error conditions:**
- `AuthenticationError` — immediate stop, partial results returned in `PipelineResult`
- `MissionGenerationError` — category-level failure after retries; error logged in `errors` list, pipeline continues to next category
- Pipeline returns partial results if any categories fail (completed categories are valid)

**Edge cases:**
- All 7 categories succeed → `PipelineResult` with 7 entries, empty `errors`
- First category fails → `PipelineResult` with 0 completed, 1 error
- Resume after failure → checkpointer restores state, skips completed categories, retries the failed one
- Campaign dir doesn't exist → created automatically
- Checkpoint DB is corrupted → fall back to `InMemorySaver`, log warning, start fresh

---

### `MissionGenerationError`

```python
class MissionGenerationError(LLMError):
    def __init__(
        self, message: str, category: str = "", attempts: int = 1
    ) -> None: ...
```

**Purpose:** Raised when mission generation fails for a specific category.

**Invariants:**
- Always includes the category name (which category failed)
- Always includes the attempt count
- Inherits from `LLMError`

---

### `PipelineResult`

```python
@dataclass
class PipelineResult:
    category_missions: dict[str, CategoryMissionsResponse]
    category_validations: dict[str, ValidationResult]
    completed_categories: list[str]
    errors: list[str]
```

**Purpose:** Complete output of the mission generation pipeline.

**Invariants:**
- `completed_categories` contains only categories that succeeded
- `category_missions` and `category_validations` have matching keys
- `errors` contains one entry per failed category
- len(completed_categories) + len(errors) <= 7

---

### `ValidationResult`

```python
@dataclass
class ValidationResult:
    missions: list[MissionItemResponse]
    warnings: list[str]
    fixes_applied: list[str]
    valid: bool
```

**Purpose:** Result of validating a category's generated missions.

**Invariants:**
- `valid` is True only if at least 1 mission exists and no critical issues remain
- `warnings` includes every issue found, whether auto-fixed or not
- `fixes_applied` lists only the auto-fix actions taken
- `missions` is the post-fix version (not the original LLM output)

---

## Property-Based Testing Candidates (for Phase 8)

1. **`validate_prerequisites` cycle detection**: For any set of missions with arbitrary prerequisite_titles, the output never contains cycles. (Generate random DAGs with injected cycles, verify they're broken.)

2. **`validate_prerequisites` depth enforcement**: For any set of missions, the maximum prerequisite chain depth in the output is <= `MAX_PREREQUISITE_DEPTH`.

3. **`validate_prerequisites` root guarantee**: For any non-empty mission set, at least `MIN_ROOT_MISSIONS_PER_CATEGORY` missions in the output have zero prerequisites.

4. **`validate_prerequisites` reachability**: For any non-empty mission set, every mission in the output is reachable from at least one root mission.

5. **`validate_file_refs` line clamping**: For any `line_start` and `line_end` and any valid file, the output line range is always `1 <= line_start <= line_end <= file_line_count`.

6. **`validate_file_refs` idempotence**: Validating an already-validated set of file refs produces identical output with no new warnings.

7. **Pipeline category ordering**: `completed_categories` in `PipelineResult` is always a prefix of `CATEGORY_ORDER` (categories complete in order).

8. **Pipeline partial results**: For any N categories completing before a failure, `len(completed_categories) == N` and all N entries are present in `category_missions`.

---

## Security Checklist

### Input Handling
- [x] LLM output is validated before use (file paths checked against filesystem)
- [x] No user input flows directly into LLM prompts (all context from static analysis)
- [x] File paths from LLM are validated as relative paths within repo (no path traversal)
- [x] Line numbers from LLM are clamped to valid ranges (no out-of-bounds reads)

### API Key Handling
- [x] No new API key handling — reuses Phase 6's provider factory
- [x] API keys never appear in logs, errors, or checkpointed state

### File Access
- [x] File reads during validation use explicit UTF-8 encoding with error handling
- [x] Binary files are skipped gracefully
- [x] Only files within `repo_path` are accessed (no path traversal from LLM output)

### Checkpoint Security
- [x] Checkpoint SQLite file stored in campaign directory (user-controlled location)
- [x] No sensitive data (API keys) stored in checkpoint state
- [x] Checkpoint state contains only mission data and category progress

### Error Handling
- [x] Auth errors propagate immediately (no retry)
- [x] Category-level failures don't crash the pipeline (logged, continue to next)
- [x] Validation never raises — always returns results with warnings
