# kb — Knowledge Base CLI

Give Claude Code persistent memory across sessions. `kb` indexes meeting transcripts, memory files, and entity records into a hybrid search engine so agents can find people, decisions, and context without manual file browsing. SQLite + LanceDB + FTS5 + Qwen3 embeddings.

IMPORTANT: Prefer reading docs/ and source files over guessing. The index below tells you where everything lives.

## Setup

```bash
uv sync --all-extras && uv run pre-commit install   # first time
uv run pytest -x -q --cov && uv run mypy src/       # verify
uv run kb --help                                      # CLI self-documentation (includes agent playbook)
```

## Common Commands

```bash
uv run pytest tests/test_search.py::test_name -x -v  # single test
uv run pytest -n auto -x -q --cov                     # parallel (xdist)
uv run ruff check src/ --fix                           # lint + autofix
uv run ruff format src/                                # format
uv run kb mcp                                          # start MCP server (stdio)
```

Pre-commit hooks enforce everything (ruff, mypy, bandit, pytest+cov). Trust the hooks.

**Quick CI check locally:**
```bash
make ci                          # mirror exact GitHub CI pipeline
make fix                         # auto-fix lint + format issues
./scripts/ci-local.sh            # same as make ci, with --fix flag available
./scripts/ci-local.sh --fix      # auto-fix then run full checks
```

## TDD Workflow

Every new feature must have:
1. **A happy-path integration test** — exercises the real SQLite + LanceDB layer using the test fixture DB (`KB_DATA_DIR` isolation). No mocking of internal modules — test the actual pipeline end-to-end.
2. **A happy-path E2E test** — a subprocess smoke test invoking `kbx <command>` as a real process and asserting on stdout/exit code (see `tests/test_cli.py` for the pattern).

**TDD order is mandatory:**
- Write integration and E2E tests first (red), then implement until they pass (green)
- When fixing a bug: write a test that reproduces the bug first, then fix the code
- Never write implementation code before a failing test exists

**Exceptions (must be explicitly noted in a comment, not silently skipped):**
- Pure infrastructure (migrations, NFC normalization, cache warming) — unit test only is acceptable
- MCP handler functions — handler unit tests are sufficient; do not test FastMCP transport wiring (that's FastMCP's responsibility)
- ML model behaviour (embedding quality, search tuning) — covered by `tests/eval_queries.py`, not pytest

Quick commands:
```bash
uv run pytest tests/test_search.py::test_name -x -v  # single test
uv run pytest -n auto -x -q --cov                     # parallel (xdist)
uv run pytest -m "not slow" -x -q                     # skip embedding model tests
```
Then: `uv run mypy src/` — clean before committing.

## Architecture

`sources/` (walk_*) → `chunker.py` (parse + chunk) → `indexer.py` (store + embed + link) → `search.py` (FTS5 + vector + RRF)

**Write-through principle:** Markdown files are the source of truth. All data writes go to flat files first; the DB is a derived index rebuilt from those files. Never write to the DB without a corresponding file.

**The boundary rule:** Sources yield `ParsedDocument`, indexer stores to SQLite + LanceDB, search returns `SearchResponse`. Types in `types.py` (Pydantic strict).

## Index

```
docs/|root: ./docs
|architecture.md — system design, data flow, component relationships
|chunking.md — markdown-aware chunking strategy (notes by ##, transcripts by ¶)
|cli.md — CLI commands reference
|context.md — compressed entity index for AI agents (~2K tokens)
|entities.md — entity system: seeding, regex linking, CRUD
|indexing.md — walk → chunk → embed → store pipeline
|integration.md — Granola ingest, import/export
|mcp.md — MCP server mode (stdio transport)
|output.md — render pipeline (table/json/jsonl/csv + fields + jq)
|search.md — FTS5 + vector + RRF fusion + recency weighting
|testing.md — test strategy, fixtures, markers

src/kb/|root: ./src/kb
|api.py — KnowledgeBase service class (public Python API for external consumers)
|cli.py — Click commands, all output via kb_output()
|config.py — project root detection, get_db() singleton
|types.py — Pydantic v2 strict models (ParsedDocument, SearchResult, Entity…)
|db.py — SQLite + LanceDB wrapper, migrations, NFC path normalization
|indexer.py — orchestrates: walk → chunk → embed → store → link entities
|search.py — hybrid search: FTS5 + vector + RRF fusion + recency weighting
|embeddings.py — Qwen3-Embedding-0.6B (MLX on Apple Silicon, PyTorch fallback)
|entities.py — entity seeding from memory/ + regex-based mention linking + source-ID linking
|matching.py — task-to-project matching (TypedDicts: TaskInput, ProjectInput) + sources extraction/formatting
|chunker.py — markdown-aware chunking (notes by ##, transcripts by ¶)
|context.py — compressed entity index for AI agents
|output.py — render pipeline (table/json/jsonl/csv + fields + jq)
|crud.py — entity CRUD with markdown file sync
|writeback.py — DB → markdown file sync (atomic writes)
|staleness.py — auto-reindex changed memory files on next search
|glossary.py — glossary term CRUD (memory/glossary.md)
|dateparse.py — natural date parsing ("since January", "last 7 days")
|mcp_server.py — MCP server mode (stdio transport)
|sources/{meetings,memory}.py — walk meetings/ and memory/ directories
|sync/granola.py — Granola API sync + attendee matching
|sync/ydoc.py — ProseMirror JSON → Yjs ydoc state (pure-Python via pycrdt, no Node.js)
```

## Conventions

- mypy strict, Pydantic v2 (`strict=True`; `StrictFrozen` for immutable, `StrictMutable` for models mutated at runtime), Python 3.10+
- Line length 100 (ruff enforced)
- Coverage minimum 90% — enforced by pre-commit
- **`kb --help` is the LLM API contract** — any CLI change MUST update `_AGENT_PLAYBOOK` in `cli.py`
- All file writes atomic (temp file → rename)
- NFC Unicode normalization on all paths (macOS compat)

## Claude Code Sandbox

The Claude Code sandbox injects SOCKS proxy env vars (`ALL_PROXY=socks5h://localhost:...`) that break `httpx` (requires `socksio` package). This affects any HTTP call — Granola sync, API calls, HuggingFace model downloads.

**How it's handled:**
- **CLI startup** (`cli.py`): Strips `ALL_PROXY`, `all_proxy`, `FTP_PROXY`, `ftp_proxy`, `GRPC_PROXY`, `grpc_proxy`, `RSYNC_PROXY` before any imports. Covers all CLI commands.
- **Tests** (`conftest.py`): Session-scoped autouse fixture strips the same vars + sets `HF_HUB_OFFLINE=1` so tests never hit the network.
- **Programmatic use**: If using `kb` as a library (e.g., `from kb.sync.granola import GranolaClient`), callers must strip proxy vars themselves — the CLI-level fix doesn't apply.

**Other sandbox notes:**
- DNS resolution can fail even for allowlisted hosts when running `uv run python3 -c "..."` — may need `dangerouslyDisableSandbox` for direct API calls outside the CLI entry point.
- GPG signing requires the passphrase to be pre-cached in `gpg-agent` from a regular terminal — `pinentry-mac` can't show a dialog from Claude Code subprocesses.
- Embedding model tests are skipped when the Qwen3 model isn't in the local HF cache (`~/.cache/huggingface/hub/models--Qwen--Qwen3-Embedding-0.6B/`).

## Claude Code Hooks

`.claude/settings.json` configures two hooks (matching gm's pattern):
- **PreToolUse** (`block-sensitive-files.sh`): Blocks Edit/Write to `.env*` and `uv.lock`
- **PostToolUse** (`post-edit-lint.sh`): Auto-runs `ruff check --fix` + `ruff format` on `.py` files after Edit/Write

Scripts in `.claude/hooks/` read stdin JSON for `tool_input.file_path` and `cwd`.

## MCP Server

- **27 tools** with full MCP tool annotations (`readOnlyHint`, `destructiveHint`, `idempotentHint`)
- **Handler pattern**: testable `handle_*` functions (take `db`/`project_root` directly) + thin MCP wrappers (call `get_db()`/`find_project_root()` then delegate)
- **Error shape**: `{"error": str, "suggestion": str | null}` — all tools
- **List shape**: `{"results": [...], "meta": {"total": N, "limit": N}}` — all list tools. `total` is the true count, not capped by limit.
- **Tool annotations**: 16 read-only, 6 mutating-idempotent, 3 mutating, 2 destructive. Defined as `_READ_ONLY`, `_MUTATING`, `_MUTATING_IDEMPOTENT`, `_DESTRUCTIVE` presets.
- **`find_project_root()` with global config**: When XDG config has absolute memory path, derives project root from `memory_path.parent`. Uses `Path.is_relative_to()` for safe XDG detection.
- **Tag filtering**: Only works on memory notes (`memory_note`, `memory_doc`). Meeting docs don't have tags — use `doc_type` filter instead.

## Gotchas

- **MPS memory** — embedding batches: 32 on MPS, 16 on CPU; texts truncated at 8K chars. `torch.mps.empty_cache()` between batches
- **Model cache** — ~1.1GB at `~/.config/kbx/model/`. First run downloads automatically
- **`KB_DATA_DIR`** — env var overrides default `~/.config/kbx/`. Tests use this to isolate
- **Incremental indexing** — skips unchanged files (by `content_hash`). Use `--full` to force
- **LanceDB lazy import** — only loaded for vector ops. `--no-embed` skips entirely
- **Slow tests** — `@pytest.mark.slow` on ML model tests; pre-commit skips them (`-m "not slow"`). Run all with `uv run pytest`
