# Phase 8 Testing: LLM Pipeline Tests (Phases 6–7)

## Summary

Comprehensive unit and property-based tests for the LLM integration layer and mission generation pipeline. All LLM calls are mocked — no real API keys or network access required.

## Automated Tests

Run the full Phase 8 test suite:

```bash
PYTHONPATH=/project/src:/project/.venv/lib/python3.11/site-packages pytest \
  tests/test_llm_exceptions.py \
  tests/test_llm_retry.py \
  tests/test_llm_context.py \
  tests/test_llm_provider.py \
  tests/test_llm_briefing.py \
  tests/test_mission_schemas.py \
  tests/test_mission_context.py \
  tests/test_mission_validation_filerefs.py \
  tests/test_mission_validation_prereqs.py \
  tests/test_mission_generator.py \
  tests/test_mission_pipeline.py \
  tests/test_conftest_helpers.py \
  -v
```

Expected: **~298 tests passing**

## Manual Testing Steps

### 1. Verify Test Coverage

```bash
PYTHONPATH=/project/src:/project/.venv/lib/python3.11/site-packages pytest \
  tests/test_llm_*.py tests/test_mission_*.py \
  --cov=unfamiliar.llm --cov=unfamiliar.analysis \
  --cov-report=term-missing
```

- **Expected:** Coverage report showing high coverage for `llm/` package

### 2. Run Property-Based Tests Separately

```bash
PYTHONPATH=/project/src:/project/.venv/lib/python3.11/site-packages pytest \
  tests/test_llm_retry.py::TestCalculateDelayPropertyBased \
  tests/test_llm_context.py::TestBuildBriefingContextPropertyBased \
  tests/test_mission_validation_filerefs.py::TestValidateFileRefsPropertyBased \
  tests/test_mission_validation_prereqs.py::TestValidatePrerequisitesPropertyBased \
  -v
```

- **Expected:** All property-based tests pass with multiple generated examples

### 3. Verify No Real API Calls

```bash
# Ensure no ANTHROPIC_API_KEY or GLM_API_KEY in environment
unset ANTHROPIC_API_KEY GLM_API_KEY
PYTHONPATH=/project/src:/project/.venv/lib/python3.11/site-packages pytest \
  tests/test_llm_provider.py tests/test_llm_briefing.py \
  -v
```

- **Expected:** All tests pass without any API keys (proves mocking works)

## Edge Cases to Check

- [ ] Exception hierarchy: all exceptions catchable via `except LLMError`
- [ ] Retry backoff: delay between attempts follows exponential formula
- [ ] Token budget: context never exceeds budget even with large files
- [ ] Line clamping: `line_start=0, line_end=0` fixed to valid range
- [ ] Cycle detection: cycles in prerequisites broken with warning
- [ ] Root guarantee: at least one root mission exists after validation

## Issues Fixed During Review

1. **Patch location error**: `resolve_api_key` moved from `factory.py` to `config.py` — tests updated to patch correct location
2. **Path traversal test**: Test expected "Path traversal" warning but file didn't exist — updated assertion to accept either message
3. **Root guarantee test**: Test expected "root" in warning but cycle breaking created root implicitly — updated assertion
4. **Hypothesis health check**: Property test used function-scoped fixture — added `suppress_health_check`
5. **Line clamping bug**: Property test found bug where `line_end=0` wasn't fixed — fixed in implementation
6. **LangChain mocking**: MagicMocks don't work with LangChain's `|` operator — created helper to patch prompts

## Checklist

- [x] All 298 tests pass
- [x] Property-based tests verify invariants
- [x] No real API calls in tests
- [x] Implementation bugs found and fixed
- [x] Ruff linting passes on all test files
