<!-- STATUS: COMPLETE -->

## Verification Complete

**Test Results:**
- Total tests: 780 (55 new tests added)
- All tests pass: ✅
- Warnings: 4 (pre-existing PytestCollectionWarning for TestInfo dataclass)

**New Test Files Created:**
- `tests/test_llm_structured.py` — 35 tests for TruncationResult, detect_truncation, _extract_json, create_structured_chain

**Tests Added to Existing Files:**
- `tests/test_llm_provider.py` — +14 tests (8 for get_retry_delay, 6 for get_effective_token_budget)
- `tests/test_mission_generator.py` — +6 tests for generate_category_missions_multi

# Phase 12: Test Reset & Coverage

## Goal

Close test gaps for the LLM module by writing tests from interface contracts (contracts.md), ensuring all public functions have comprehensive coverage including edge cases and error conditions.

## Dependencies

- **Phase 11** (Interface contracts) — contracts.md defines expected behavior
- **my-style/testing.md** — Test rules and patterns
- **contracts.md** — Single source of truth for expected behavior

## Context

This is a **test phase** running in a fresh session. Tests must be written from the interface contracts in `contracts.md`, NOT from reading implementation code. This ensures tests verify behavior, not implementation details.

**Major gaps identified:**
1. `llm/structured.py` — NO tests exist (0% coverage)
2. `get_retry_delay()` in `llm/providers/factory.py` — missing tests
3. `get_effective_token_budget()` in `llm/providers/factory.py` — missing tests
4. `generate_category_missions_multi()` in `llm/missions/generator.py` — missing tests

**Test rules from my-style/testing.md:**
1. Tests from interface contracts, not implementation
2. Field Coverage — Assert ALL fields on returned objects
3. Exact Counts — Use `==` for counts, never `>=` or `> 0`
4. Negative Test Ratio — Minimum 1 error/edge case per 2 happy path tests
5. No Mirror Tests — Never reimplement production logic in test
6. No Tautologies — Assert computed values, not setup data

---

## Environment Constraints

From container CLAUDE.md:
- **No sudo/chmod/chown** — blocked at kernel level
- **No apt-get** — require sudo
- **No pip install** — ephemeral container, changes lost
- **PYTHONPATH** — auto-configured for project
- **Tests run via:** `pytest` (pre-installed)

---

## Interface Contracts

**Source:** `_planning/phases/12-test-reset-coverage/contracts.md`

### Key Functions Requiring Tests

| Function | Module | Current State |
|----------|--------|---------------|
| `detect_truncation` | `llm/structured.py` | NO TESTS |
| `TruncationResult` | `llm/structured.py` | NO TESTS |
| `create_structured_chain` | `llm/structured.py` | NO TESTS |
| `_extract_json` | `llm/structured.py` | NO TESTS |
| `get_retry_delay` | `llm/providers/factory.py` | MISSING |
| `get_effective_token_budget` | `llm/providers/factory.py` | MISSING |
| `generate_category_missions_multi` | `llm/missions/generator.py` | MISSING |

---

## Tasks

### Task 1: Verify test environment [DONE]

**Action:**
1. Run `pytest --collect-only -q` to verify test collection works
2. Run `pytest --tb=short -q` to verify all 725 existing tests pass

**Verify:**
- Test collection succeeds ✓
- All existing tests pass ✓

**Done when:**
- Environment confirmed working ✓

---

### Task 2: Create test_llm_structured.py — TruncationResult and detect_truncation [DONE]

**File:** `tests/test_llm_structured.py`

**Action:**
Write tests for `TruncationResult` dataclass and `detect_truncation()` function based on contracts:

**From contracts:**
- `detect_truncation(text, schema) -> TruncationResult`
- Never raises — always returns TruncationResult
- Recovery attempts only for mission-style schemas (has "missions" field)

**Test classes:**
1. `TestTruncationResult` — dataclass field validation
2. `TestDetectTruncationCompleteResponse` — complete JSON handling
3. `TestDetectTruncationEmptyInput` — empty/whitespace inputs
4. `TestDetectTruncationTruncatedResponse` — truncated JSON recovery
5. `TestDetectTruncationInvalidInput` — no JSON, invalid structure

**Edge cases from contracts:**
| Input | Expected Result |
|-------|-----------------|
| Empty text | `TruncationResult(None, False, None, None)` |
| Whitespace only | `TruncationResult(None, False, None, None)` |
| Complete JSON | `TruncationResult(parsed, False, None, None)` |
| JSON in markdown fence | Extracted and parsed |
| Truncated mid-mission | `TruncationResult(partial, True, last_idx, hint)` |
| No JSON at all | `TruncationResult(None, True, None, "No JSON object found")` |

**Verify:**
```bash
pytest tests/test_llm_structured.py -v
```

**Done when:**
- All tests pass
- All edge cases from contracts covered

---

### Task 3: Add tests for _extract_json internal function [DONE]

**File:** `tests/test_llm_structured.py`

**Action:**
Write tests for `_extract_json()` internal function:

**From contracts:**
- Returns only the JSON portion (no surrounding text)
- Handles markdown code fences
- Attempts recovery for truncated JSON

**Test class:** `TestExtractJson`

**Edge cases from contracts:**
| Input | Output |
|-------|--------|
| Pure JSON object | Unchanged |
| JSON in ```json fence | Content extracted |
| JSON in ``` fence (no lang) | Content extracted |
| Unclosed fence (truncated) | Content extracted, recovery attempted |
| JSON embedded in text | Object extracted by brace matching |
| Truncated JSON | Recovery attempted |
| No JSON found | `ValueError: Could not extract JSON...` |

**Verify:**
```bash
pytest tests/test_llm_structured.py::TestExtractJson -v
```

**Done when:**
- All tests pass
- Error condition tested (ValueError)

---

### Task 4: Add tests for create_structured_chain [DONE]

**File:** `tests/test_llm_structured.py`

**Action:**
Write tests for `create_structured_chain()` function:

**From contracts:**
- Return value's `.invoke(input_data)` always returns instance of schema
- Provider-agnostic: works with any BaseChatModel
- For models without tool calling: prompt-based JSON + manual parsing

**Test class:** `TestCreateStructuredChain`

**Provider behavior from contracts:**
| Provider | Method |
|----------|--------|
| Anthropic | `model.with_structured_output(schema)` |
| OpenAI | `model.with_structured_output(schema)` |
| GLM (ChatZhipuAI) | Prompt-based JSON + manual parsing |

**Error conditions:**
- JSON parse failure → `ValueError: Could not extract JSON from LLM response...`
- Schema validation failure → `ValueError: LLM response did not match expected schema...`

**Verify:**
```bash
pytest tests/test_llm_structured.py::TestCreateStructuredChain -v
```

**Done when:**
- Tool-calling models use `with_structured_output`
- Non-tool-calling models use prompt-based fallback
- Error conditions tested

---

### Task 5: Add get_retry_delay tests to test_llm_provider.py [DONE]

**File:** `tests/test_llm_provider.py`

**Action:**
Write tests for `get_retry_delay()` function:

**From contracts:**
```python
def get_retry_delay(provider_name: str) -> float
```

**Invariants:**
- Always returns positive float
- Unknown providers return default (1.0)

**Expected returns from contracts:**
| Input | Return |
|-------|--------|
| `"anthropic"` | `1.0` |
| `"ANTHROPIC"` | `1.0` (case-insensitive) |
| `"glm"` | `30.0` |
| `"GLM"` | `30.0` (case-insensitive) |
| `"unknown"` | `1.0` |
| `"any_other_string"` | `1.0` |

**Edge cases from contracts:**
| Input | Return |
|-------|--------|
| Empty string | `1.0` |
| Whitespace string | `1.0` (stripped, unknown) |
| `" glm "` | `30.0` (stripped) |

**Test class:** `TestGetRetryDelay`

**Verify:**
```bash
pytest tests/test_llm_provider.py::TestGetRetryDelay -v
```

**Done when:**
- All provider cases covered
- Case insensitivity tested
- Edge cases tested

---

### Task 6: Add get_effective_token_budget tests to test_llm_provider.py [DONE]

**File:** `tests/test_llm_provider.py`

**Action:**
Write tests for `get_effective_token_budget()` function:

**From contracts:**
```python
def get_effective_token_budget(config: UnfamiliarConfig) -> int
```

**Invariants:**
- Return value ≤ `config.token_budget`
- Return value ≤ provider's `max_token_budget` (if set)
- If provider has no limit, returns `config.token_budget` unchanged

**Expected returns from contracts:**
| Scenario | Return |
|----------|--------|
| Anthropic with budget 100k | `100000` (no cap) |
| GLM with budget 100k | `8000` (capped) |
| GLM with budget 5000 | `5000` (under cap) |
| GLM with budget 8000 | `8000` (exactly at cap) |
| Unknown provider | `config.token_budget` unchanged |

**Test class:** `TestGetEffectiveTokenBudget`

**Verify:**
```bash
pytest tests/test_llm_provider.py::TestGetEffectiveTokenBudget -v
```

**Done when:**
- Provider capping tested
- Under-cap scenario tested
- No-cap provider tested

---

### Task 7: Add generate_category_missions_multi tests to test_mission_generator.py [DONE]

**File:** `tests/test_mission_generator.py`

**Action:**
Write tests for `generate_category_missions_multi()` function:

**From contracts:**
```python
def generate_category_missions_multi(
    analysis: AnalysisResult,
    briefing_result: BriefingResult,
    category: InvestigationCategory,
    model: BaseChatModel,
    previous_missions: list[str] | None = None,
    token_budget: int = DEFAULT_TOKEN_BUDGET,
    provider_name: str | None = None,
    missions_per_call: int = 3,
    max_calls: int = 3,
) -> CategoryMissionsResponse
```

**Invariants:**
- Makes 1-3 LLM calls depending on truncation
- Returns at least 1 mission (or raises MissionGenerationError)
- No duplicate mission titles in result
- Total missions ≤ `missions_per_call * max_calls` (9 default)

**Multi-call behavior from contracts:**
1. Call 1: Request 3 missions
2. If truncated → Extract complete missions, call 2 with "continue from X" hint
3. If still truncated → Call 3
4. Merge all missions, deduplicate by `title_plain`

**Edge cases from contracts:**
| Scenario | Expected Behavior |
|----------|-------------------|
| All calls fail but have partial missions | Return partial results with warning |
| All calls fail, no missions | `MissionGenerationError` |
| Duplicate titles across calls | Keep first occurrence only |
| Truncation detected with resume_hint | Hint passed to next call |

**Test class:** `TestGenerateCategoryMissionsMulti`

**Verify:**
```bash
pytest tests/test_mission_generator.py::TestGenerateCategoryMissionsMulti -v
```

**Done when:**
- Multi-call truncation behavior tested
- Deduplication tested
- Error conditions tested

---

## Verify Criteria

- [ ] All new test files created
- [ ] All existing tests still pass (725 → more)
- [ ] All functions from contracts have tests
- [ ] All edge cases from contracts covered with expected return values
- [ ] Error conditions tested with correct exception types
- [ ] No mirror tests (tests don't reimplement logic)
- [ ] Negative test ratio ≥ 1:2

---

## Out of Scope

- Code changes to implementation
- Tests for other modules (analysis, db, etc.)
- Property-based tests (no pure calculations in target functions)
- Integration tests with real LLM calls

---

## Full Suite Check

After all tasks complete, run:

```bash
pytest --tb=short -q
```

Report:
- Total pass/fail count
- Any regressions from existing tests
- New test count added

---

## Expected Test Count

| Task | New Tests (approx) |
|------|-------------------|
| Task 2: TruncationResult/detect_truncation | 15-20 |
| Task 3: _extract_json | 8-10 |
| Task 4: create_structured_chain | 6-8 |
| Task 5: get_retry_delay | 8-10 |
| Task 6: get_effective_token_budget | 5-7 |
| Task 7: generate_category_missions_multi | 8-10 |
| **Total** | **50-65 new tests** |
