# Phase 9 Summary: Analysis Filtering

## What Was Built

Implemented aggressive file filtering at the discovery phase to reduce token bloat and prevent 429 errors. The system now filters out non-code files (docs, data, config) before they reach the LLM pipeline, while still respecting `.gitignore` patterns and allowing per-repo customization.

**Key components:**
- `FileFilter` class with 5-layer filter order
- `.gitignore` integration via pathspec library
- Per-repo config loading (`~/.unfamiliar/repos/<hash>/config.yaml`)
- `excluded_count` tracking in `AnalysisResult`

## Key Decisions

1. **Filter at discovery, not orchestrator** — Single-pass efficiency; files are filtered once before any processing
2. **PathSpec with "gitignore" pattern style** — Ensures compatibility with `.gitignore` syntax; fixed pathspec deprecation warning by using `"gitignore"` instead of `GitWildMatchPattern`
3. **Tuple return from `discover_files()`** — Returns `(files, excluded_count)` instead of wrapping in a new type; simpler API

## Non-Obvious Logic

### Filter Order (critical for correctness)
```
1. ALWAYS_EXCLUDE → exclude (cannot override)
2. gitignore → exclude
3. DEFAULT_EXCLUDE → exclude UNLESS ALWAYS_INCLUDE or include_patterns matches
4. include_patterns → include (override DEFAULT_EXCLUDE)
5. exclude_patterns → exclude (final check)
```

The interaction between layers 3-4 is the trickiest: `ALWAYS_INCLUDE` patterns like `pyproject.toml` are exempted from `DEFAULT_EXCLUDE`, and user `include_patterns` can also override defaults.

### Edge Cases in `should_exclude()`
- Empty path or absolute path (`/etc/passwd`) → excluded (invalid format)
- `file_filter=None` → no filtering (backward compatible)
- Missing `.gitignore` → filter works with defaults only

## Connections

- **Phase 6** (LLM integration): Benefits from reduced token usage
- **Phase 10** (Campaign orchestration): Will use `excluded_count` for reporting
- **Future**: Per-repo configs allow teams to customize filtering for monorepos with unusual structures
