# Codebase Overview

Generated: 2026-03-24 (after Phase 20)

## Stack

- **Language:** Python 3.11+
- **CLI Framework:** Typer
- **TUI Framework:** Textual
- **Database:** SQLite (via repository layer)
- **LLM Integration:** LangChain + LangGraph
- **Parsing:** stdlib ast (Python), Tree-sitter (multi-language)
- **Validation:** Pydantic v2

## Project Structure

```
unfamiliar/
├── pyproject.toml               # Package config, dependencies, tool settings
├── src/unfamiliar/
│   ├── __init__.py              # Package version
│   ├── __main__.py              # python -m unfamiliar entry
│   ├── cli.py                   # Typer CLI commands
│   ├── config.py                # Config loading, API keys, campaign dirs
│   ├── constants.py             # Domain enums and defaults
│   ├── models/                  # Pydantic domain models
│   │   ├── __init__.py          # Re-exports all models
│   │   ├── campaign.py          # Campaign, Briefing
│   │   ├── mission.py           # Mission, MissionChain, junction models
│   │   └── analysis.py          # Module, FileSnapshot
│   ├── db/                      # SQLite repository layer
│   │   ├── __init__.py          # open_campaign_db entry point
│   │   ├── schema.py            # SCHEMA_SQL, SCHEMA_VERSION
│   │   ├── connection.py        # open_database, init_database
│   │   └── repositories/
│   │       ├── __init__.py      # Re-exports all repos
│   │       ├── campaign.py      # CampaignRepository
│   │       ├── module.py        # ModuleRepository
│   │       ├── mission.py       # MissionRepository
│   │       ├── chain.py         # ChainRepository
│   │       ├── file_snapshot.py # FileSnapshotRepository
│   │       ├── mission_file_ref.py   # MissionFileRefRepository
│   │       └── mission_link.py       # MissionPrerequisite, ChainMission repos
│   ├── analysis/                # Static analysis pipeline
│   │   ├── __init__.py          # Re-exports analyze_repository, AnalysisResult
│   │   ├── models.py            # Dataclasses for pipeline data structures
│   │   ├── discovery.py         # File discovery via git ls-files
│   │   ├── orchestrator.py      # analyze_repository() entry point
│   │   ├── scoring.py           # File priority scoring
│   │   ├── testing.py           # Test file detection, coverage parsing
│   │   ├── adapters/            # Language-specific AST adapters
│   │   │   ├── __init__.py      # create_default_registry, PythonAdapter
│   │   │   ├── protocol.py      # LanguageAdapter protocol, AdapterRegistry
│   │   │   ├── python_adapter.py     # Python ast-based adapter
│   │   │   ├── javascript_adapter.py # JavaScript Tree-sitter adapter
│   │   │   ├── typescript_adapter.py # TypeScript Tree-sitter adapter
│   │   │   ├── _ts_imports.py        # TS import resolution helpers
│   │   │   ├── _ts_entry_points.py   # TS entry point detection
│   │   │   ├── _ts_definitions.py    # TS definition extraction
│   │   │   └── _treesitter_utils.py  # Shared Tree-sitter utilities
│   │   ├── filters/             # File filtering logic
│   │   │   ├── __init__.py      # Filter exports
│   │   │   ├── gitignore.py     # .gitignore pattern matching
│   │   │   └── patterns.py      # Exclude patterns (vendor, node_modules, etc.)
│   │   ├── graph/               # Import graph, module clustering
│   │   │   ├── __init__.py      # Re-exports graph functions
│   │   │   ├── import_graph.py  # build_import_graph, get_hub_files
│   │   │   └── clustering.py    # detect_modules via label propagation
│   │   └── security/            # Secret detection, CVE scanning
│   │       ├── __init__.py      # Security analysis exports
│   │       ├── secrets.py       # detect-secrets integration
│   │       └── vulnerabilities.py # CVE scanning (pip-audit, npm audit)
│   ├── llm/                     # LLM integration
│   │   ├── __init__.py          # Public API exports
│   │   ├── exceptions.py        # Exception hierarchy (LLMError, ProviderError, etc.)
│   │   ├── retry.py             # Exponential backoff retry utilities
│   │   ├── context.py           # AnalysisResult → LLM context transformer
│   │   ├── structured.py        # Structured output via tool calling or JSON fallback
│   │   ├── providers/           # Provider abstraction
│   │   │   ├── __init__.py      # create_chat_model export
│   │   │   └── factory.py       # create_chat_model() for Anthropic, GLM
│   │   ├── briefing/            # Codebase briefing generation
│   │   │   ├── __init__.py      # generate_briefing, BriefingResult exports
│   │   │   ├── schemas.py       # BriefingResponse, ModuleDescription
│   │   │   ├── prompts.py       # BRIEFING_PROMPT template
│   │   │   └── generator.py     # generate_briefing() orchestrator
│   │   └── missions/            # Mission generation and validation
│   │       ├── __init__.py      # Mission generation exports
│   │       ├── schemas.py       # MissionResponse, CategoryMissions
│   │       ├── prompts.py       # MISSION_PROMPT template
│   │       ├── context.py       # Mission context building
│   │       ├── generator.py     # generate_missions() orchestrator
│   │       ├── validation.py    # Mission validation and deduplication
│   │       └── pipeline.py      # Full mission generation pipeline
│   ├── campaign/                # Campaign lifecycle orchestration
│   │   ├── __init__.py          # Campaign creation exports
│   │   ├── types.py             # CampaignExistsError
│   │   ├── discovery.py         # discover_campaigns()
│   │   ├── orchestrator.py      # create_campaign(), CampaignOrchestrationError
│   │   ├── persister.py         # persist_pipeline_data()
│   │   ├── progress.py          # get_campaign_progress()
│   │   └── missions.py          # Mission status management, reopen_campaign()
│   ├── tui/                     # Textual TUI application
│   │   ├── __init__.py          # run_tui() entry point
│   │   ├── app.py               # Main TUI application (UnfamiliarApp)
│   │   ├── views.py             # View composition
│   │   ├── commands.py          # Command handling (complete, abandon, etc.)
│   │   ├── queries.py           # Database queries for TUI
│   │   ├── screens/             # Screen definitions
│   │   │   ├── __init__.py
│   │   │   └── confirm.py       # Confirmation dialogs
│   │   ├── widgets/             # Custom widgets
│   │   │   ├── __init__.py
│   │   │   ├── header.py        # Header widget
│   │   │   ├── content.py       # Content rendering
│   │   │   ├── command_input.py # Command input widget
│   │   │   └── command_ref.py   # Command reference widget
│   │   └── themes/              # Theme system
│   │       ├── __init__.py      # Theme exports
│   │       ├── base.py          # Theme protocol and types
│   │       ├── registry.py      # ThemeRegistry
│   │       └── film_noir/       # Film Noir theme (default)
│   │           ├── __init__.py
│   │           └── theme.py     # FILM_NOIR_THEME definition
└── tests/                       # Test suite
    ├── conftest.py              # Shared fixtures
    ├── test_*.py                # Unit and integration tests
    └── ...
```

## Key Files

| File | Purpose |
|------|---------|
| `cli.py` | Entry point for all CLI commands. Validates config and git repo on entry. |
| `config.py` | Loads `~/.unfamiliar/config.toml`. Resolves API keys from environment. |
| `constants.py` | All domain enums: InvestigationCategory, MissionStatus, CampaignStatus, etc. |
| `models/__init__.py` | Re-exports all Pydantic models for convenient importing. |
| `db/__init__.py` | `open_campaign_db()` entry point returns connection + all repos. |
| `db/schema.py` | Complete DDL for 11 tables with indexes. Schema version "1.0". |
| `analysis/__init__.py` | `analyze_repository()`, `AnalysisResult`, `AnalysisError` entry points. |
| `analysis/adapters/__init__.py` | `create_default_registry()`, adapters for Python/JS/TS. |
| `llm/__init__.py` | `create_chat_model()`, `generate_briefing()`, exceptions for LLM layer. |
| `llm/missions/pipeline.py` | Full mission generation pipeline with retry handling. |
| `tui/app.py` | Main TUI application with key bindings and screen management. |
| `tui/themes/registry.py` | Theme registration and lookup. |

## CLI Commands

| Command | Purpose |
|---------|---------|
| `investigate <repo>` | Start or resume an investigation |
| `campaigns` | List all saved campaigns |
| `resume` | Resume most recent active campaign |
| `reopen <repo>` | Reopen a closed campaign (takes repo path) |
| `upgrade [campaign]` | Upgrade campaign(s) to current schema |

## Configuration

Location: `~/.unfamiliar/config.toml`

```toml
[general]
provider = "anthropic"  # or "glm"
token_budget = 50000
theme = "film_noir"

[llm]
model = ""              # Optional: override default model
max_tokens = 4096       # Optional: max output tokens
base_url = ""           # Optional: custom API endpoint
```

API key resolution: **Environment variables only**
- `ANTHROPIC_API_KEY` for Anthropic
- `ZHIPUAI_API_KEY` for GLM/Zhipu
- `UNFAMILIAR_API_KEY` as generic fallback

## Data Model

Core entities:

### Primary Entities

| Model | Purpose | Key Fields |
|-------|---------|------------|
| `Campaign` | An investigation of a codebase | repo_path, status, theme, mode |
| `Mission` | A single investigation task | title, category, difficulty, status |
| `Module` | A logical code unit | name, path, category_relevance (JSON) |
| `Briefing` | LLM-generated codebase summary | summary, tech_stack (JSON), security_posture |
| `MissionChain` | Ordered learning path through missions | name, category, sort_order |
| `FileSnapshot` | File state at analysis time | file_path, file_hash, last_analyzed |

### Junction Models

| Model | Purpose |
|-------|---------|
| `MissionFileRef` | Links missions to specific files/lines |
| `MissionPrerequisite` | DAG of mission dependencies |
| `ModuleMissionLink` | Many-to-many: modules ↔ missions |
| `ChainMissionLink` | Many-to-many: chains ↔ missions (ordered) |

### Database Schema

- **11 tables:** meta, campaign, briefing, module, mission, mission_file_ref, mission_prerequisite, module_mission, mission_chain, chain_mission, file_snapshot
- **WAL mode** for concurrent reads during TUI sessions
- **Foreign keys** enabled with CASCADE deletes
- **Indexes** on foreign keys and common query patterns (status, category)
- **JSON columns** for list fields (tech_stack, category_relevance)

## Dependencies

### Runtime
- langchain, langchain-core, langchain-anthropic, langchain-community, langchain-openai, langgraph, langgraph-checkpoint-sqlite
- textual, typer[all]
- pydantic, pydantic-settings
- tree-sitter, tree-sitter-javascript, tree-sitter-typescript
- detect-secrets, pip-audit, pathspec

### Development
- pytest, pytest-cov, pytest-asyncio
- hypothesis (property-based testing)
- ruff (linting/formatting)
- mypy (type checking)
- mutmut (mutation testing)

## Conventions

- **Line length:** 88 characters (Ruff default)
- **Type hints:** Required on all public functions (mypy strict)
- **Imports:** Sorted by isort (via Ruff)
- **Docstrings:** Module-level for all packages, function-level for public APIs
- **Database:** No ORM — direct sqlite3 with parameterized queries
- **Timestamps:** ISO 8601 text format (SQLite has no native datetime)

## Static Analysis Pipeline

### Entry Point

```python
from unfamiliar.analysis import analyze_repository, AnalysisResult

result = analyze_repository(Path("/path/to/repo"))
```

### Pipeline Stages

1. **File Discovery** — git ls-files, SHA-256 hashing, language classification
2. **Filtering** — gitignore patterns, exclude vendor/node_modules
3. **Parsing** — Language adapters extract imports, definitions, entry points
4. **Import Graph** — Build directed graph, compute in-degree centrality
5. **Module Detection** — Label propagation clustering on import graph
6. **Test Detection** — Naming convention matching, coverage.xml parsing
7. **Security Scan** — Secret detection, CVE scanning
8. **File Scoring** — Priority scoring for LLM token budget allocation

### Analysis Data Structures

| Dataclass | Purpose |
|-----------|---------|
| `FileInfo` | Discovered file metadata (path, language, size, hash) |
| `ParsedFile` | Parse results (imports, definitions, entry points, complexity) |
| `ImportGraph` | Directed import relationships with centrality |
| `DetectedModule` | Clustered module with confidence score |
| `TestInfo` | Test files, source-to-test mapping, coverage |
| `ScoredFile` | File with priority score and reasons |
| `AnalysisResult` | Complete pipeline output for LLM consumption |

### Language Adapters

Adapters implement the `LanguageAdapter` protocol:

- `supported_extensions()` — Returns frozenset of file extensions
- `can_handle(path)` — Checks if adapter handles the file
- `parse_file(path, content)` — Returns ParsedFile with structure
- `resolve_import(source, target, repo_files)` — Maps imports to repo files

**Python Adapter:** Uses stdlib `ast` module.
**JavaScript/TypeScript Adapters:** Use Tree-sitter for parsing.

### File Scoring

Scoring criteria (additive):

| Signal | Score | Condition |
|--------|-------|-----------|
| Entry point | +100 | File contains entry point (main, CLI, API) |
| High centrality | +50 | in-degree >= 3 (hub file) |
| High complexity | +30 | Cyclomatic complexity >= 15 |
| Test file | +20 | Matches test naming convention |

## LLM Integration Layer

### Entry Point

```python
from unfamiliar.llm import create_chat_model, generate_briefing, BriefingResult
from unfamiliar.config import load_config

config = load_config()
model = create_chat_model(config)
result = generate_briefing(analysis_result, model)
```

### Provider Factory

`create_chat_model(config)` creates LangChain chat models via `init_chat_model()`:

| Provider | Implementation | Notes |
|----------|---------------|-------|
| **Anthropic** | `init_chat_model(provider="anthropic")` | Default |
| **GLM (Zhipu)** | `init_chat_model(provider="openai", base_url=...)` | OpenAI-compatible endpoint |

Default settings: temperature=0.2, max_retries=2

### Structured Output

`create_structured_chain()` provides unified structured output:
- **Anthropic**: Uses tool-calling-based structured output
- **GLM**: Uses JSON text mode (tool-calling API has rate limits)

### Exception Hierarchy

```
Exception (base)
├── ConfigError                     — Config loading/parsing errors
├── APIKeyError                     — Missing/invalid API key
├── DatabaseError                   — SQLite errors
├── AnalysisError                   — Static analysis failures
├── CampaignExistsError             — Campaign already exists
├── CampaignOrchestrationError      — Campaign creation failures
├── MissionStatusError              — Invalid mission state transition
├── CampaignStatusError             — Invalid campaign state transition
└── LLMError (base for LLM layer)
    ├── RetryableError              — Base for transient, retryable failures
    │   ├── BriefingGenerationError — Briefing failed after retries
    │   └── MissionGenerationError  — Mission generation failed after retries
    ├── ProviderError               — Unknown provider, missing package
    └── AuthenticationError         — 401/403 from provider
```

### LLM Package Structure

```
llm/
├── __init__.py           # Public API exports
├── exceptions.py         # Exception hierarchy
├── retry.py              # with_llm_retry() exponential backoff
├── context.py            # build_briefing_context()
├── structured.py         # create_structured_chain()
├── providers/
│   ├── __init__.py       # create_chat_model export
│   └── factory.py        # create_chat_model() implementation
├── briefing/
│   ├── __init__.py       # generate_briefing, BriefingResult exports
│   ├── schemas.py        # BriefingResponse, ModuleDescription
│   ├── prompts.py        # BRIEFING_PROMPT template
│   └── generator.py      # generate_briefing() implementation
└── missions/
    ├── __init__.py       # Mission generation exports
    ├── schemas.py        # MissionResponse, CategoryMissions
    ├── prompts.py        # MISSION_PROMPT template
    ├── context.py        # Mission context building
    ├── generator.py      # generate_missions() with multi-call support
    ├── validation.py     # Mission validation and deduplication
    └── pipeline.py       # Full pipeline orchestrator
```

## TUI Architecture

### Entry Point

```python
from unfamiliar.tui import run_tui

run_tui(campaign_dir, campaign_id)
```

### Key Components

| Component | File | Purpose |
|-----------|------|---------|
| `UnfamiliarApp` | `tui/app.py` | Main Textual application |
| `ThemeRegistry` | `tui/themes/registry.py` | Theme registration and lookup |
| `queries.py` | `tui/queries.py` | Database queries for display |
| `commands.py` | `tui/commands.py` | Command handling (complete, abandon, etc.) |

### Theme System

Themes implement the `Theme` protocol with colors, styles, and category-specific formatting. The default `film_noir` theme provides a noir detective aesthetic.

### Key Bindings

| Key | Action |
|-----|--------|
| `j/k` | Navigate missions |
| `Enter` | Select mission |
| `c` | Complete mission |
| `a` | Abandon mission |
| `q` | Quit |
