---
phase: 02-search-core
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - src/corpus_analyzer/store/schema.py
  - src/corpus_analyzer/config/schema.py
  - src/corpus_analyzer/search/__init__.py
  - tests/search/__init__.py
  - tests/search/test_engine.py
  - tests/search/test_classifier.py
  - tests/search/test_summarizer.py
  - tests/search/test_formatter.py
  - tests/ingest/test_indexer.py
autonomous: true
requirements:
  - SEARCH-01
  - SEARCH-02
  - SEARCH-03
  - SEARCH-04
  - SEARCH-05
  - CLASS-01
  - CLASS-02
  - CLASS-03
  - SUMM-01
  - SUMM-02
  - SUMM-03
  - CLI-01

must_haves:
  truths:
    - "ChunkRecord schema has nullable construct_type and summary columns added in-place (no table rebuild)"
    - "SourceConfig has a summarize: bool = True field"
    - "All Wave 0 test files exist with failing tests for search, classifier, summarizer, formatter, and indexer extensions"
    - "The tests/search/ package is discoverable by pytest"
  artifacts:
    - path: "src/corpus_analyzer/store/schema.py"
      provides: "ChunkRecord with construct_type and summary Optional[str] fields + ensure_schema_v2()"
      contains: "ensure_schema_v2"
    - path: "src/corpus_analyzer/config/schema.py"
      provides: "SourceConfig with summarize: bool = True field"
      contains: "summarize"
    - path: "src/corpus_analyzer/search/__init__.py"
      provides: "search package init"
    - path: "tests/search/__init__.py"
      provides: "test package init"
    - path: "tests/search/test_engine.py"
      provides: "failing test stubs for search engine (SEARCH-01 through SEARCH-05, CLASS-02, CLASS-03, CLI-02, CLI-03, CLI-04)"
    - path: "tests/search/test_classifier.py"
      provides: "failing test stubs for construct classifier (CLASS-01)"
    - path: "tests/search/test_summarizer.py"
      provides: "failing test stubs for summarizer (SUMM-01, SUMM-03)"
    - path: "tests/search/test_formatter.py"
      provides: "failing test stubs for snippet extraction (SEARCH-05)"
    - path: "tests/ingest/test_indexer.py"
      provides: "extended with test_index_adds_construct_type and test_index_skips_summary_on_unchanged_file"
  key_links:
    - from: "src/corpus_analyzer/store/schema.py"
      to: "src/corpus_analyzer/ingest/indexer.py"
      via: "ensure_schema_v2 called in CorpusIndex.open()"
      pattern: "ensure_schema_v2"
    - from: "src/corpus_analyzer/config/schema.py"
      to: "src/corpus_analyzer/ingest/indexer.py"
      via: "SourceConfig.summarize checked before summary generation"
      pattern: "source\\.summarize"
---

<objective>
Extend the Phase 1 schema and config for Phase 2, and scaffold all Wave 0 test files with failing tests before any search implementation begins.

Purpose: Phase 2 implementation plans (02-02, 02-03, 02-04) are TDD plans that require red tests to exist. Schema extension must happen first so the new columns are available to all Phase 2 modules. This plan has no implementation dependencies and unblocks all downstream plans.

Output: Extended ChunkRecord and SourceConfig schemas; search package init; test scaffolds for engine, classifier, summarizer, formatter, and indexer extensions — all in failing (RED) state.
</objective>

<execution_context>
@/home/ollie/.claude/get-shit-done/workflows/execute-plan.md
@/home/ollie/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/02-search-core/02-CONTEXT.md
@.planning/phases/02-search-core/02-RESEARCH.md
@src/corpus_analyzer/store/schema.py
@src/corpus_analyzer/config/schema.py
@src/corpus_analyzer/ingest/indexer.py
@tests/ingest/test_indexer.py
</context>

<tasks>

<task type="auto">
  <name>Task 1: Extend ChunkRecord schema and SourceConfig for Phase 2</name>
  <files>
    src/corpus_analyzer/store/schema.py
    src/corpus_analyzer/config/schema.py
    src/corpus_analyzer/search/__init__.py
  </files>
  <action>
**src/corpus_analyzer/store/schema.py** — Add two Optional[str] fields to ChunkRecord and an `ensure_schema_v2` function:

1. Add `from __future__ import annotations` is already present.
2. Add `Optional` import: change `from __future__ import annotations` to also enable `Optional` via `from typing import Optional` (or use `str | None` with `from __future__ import annotations` already present — prefer `str | None`).
3. Add two fields to ChunkRecord after `indexed_at`:

```python
# --- Phase 2 fields (nullable, added via schema evolution) ---
construct_type: str | None = None
"""Agent construct type: skill, prompt, workflow, agent_config, code, documentation.

Set to None for chunks indexed before Phase 2. Populated during corpus index
after Phase 2 lands. Treat None as 'documentation' in search display.
"""

summary: str | None = None
"""1-2 sentence AI summary of the file generated at index time via Ollama.

Set to None for chunks indexed before Phase 2 or when summarize=False on source.
"""
```

4. Add `ensure_schema_v2` function below `make_chunk_id`:

```python
def ensure_schema_v2(table: lancedb.table.Table) -> None:
    """Add Phase 2 nullable columns to an existing chunks table if they don't exist.

    Uses LanceDB's add_columns API for in-place schema evolution — no data loss
    and no table rebuild required. Safe to call on every CorpusIndex.open() call.

    Args:
        table: The LanceDB chunks table to upgrade.
    """
    existing_cols = {field.name for field in table.schema}
    if "construct_type" not in existing_cols:
        table.add_columns({"construct_type": "cast(NULL as string)"})
    if "summary" not in existing_cols:
        table.add_columns({"summary": "cast(NULL as string)"})
```

5. Add the `lancedb` import at top (it is already imported in indexer.py but not schema.py — add `import lancedb  # type: ignore[import-untyped]` to schema.py for the `ensure_schema_v2` type annotation).

**src/corpus_analyzer/config/schema.py** — Add `summarize` field to SourceConfig:

Add after the `exclude` field:

```python
summarize: bool = True
"""Whether to generate AI summaries for files in this source at index time.

Set to False for large repos where LLM summary generation cost is prohibitive.
Defaults to True. When False, summary column remains None for all chunks.
"""
```

**src/corpus_analyzer/search/__init__.py** — Create empty package init:

```python
"""Search module for corpus-analyzer.

Provides hybrid BM25+vector search, construct type classification,
AI summary generation, and result formatting.
"""
```

All three files require full type annotations and PEP 257 docstrings per project style. Run `uv run mypy src/corpus_analyzer/store/schema.py src/corpus_analyzer/config/schema.py src/corpus_analyzer/search/__init__.py` and fix any errors.
  </action>
  <verify>
    <automated>uv run pytest tests/store/test_schema.py -x -q && uv run pytest tests/config/test_schema.py -x -q && uv run mypy src/corpus_analyzer/store/schema.py src/corpus_analyzer/config/schema.py</automated>
    <manual>Check that existing store and config tests still pass after adding the new fields.</manual>
    <sampling_rate>run after this task, before Task 2</sampling_rate>
  </verify>
  <done>
    - ChunkRecord has construct_type: str | None = None and summary: str | None = None fields
    - ensure_schema_v2() exists and guards add_columns with column-existence checks
    - SourceConfig has summarize: bool = True field
    - search/__init__.py exists
    - All existing store and config tests pass
    - mypy passes on modified files
  </done>
</task>

<task type="auto">
  <name>Task 2: Scaffold Wave 0 test files (RED state)</name>
  <files>
    tests/search/__init__.py
    tests/search/test_engine.py
    tests/search/test_classifier.py
    tests/search/test_summarizer.py
    tests/search/test_formatter.py
    tests/ingest/test_indexer.py
  </files>
  <action>
Create test scaffolds in RED (failing) state. Use `pytest.mark.xfail(strict=True, reason="not yet implemented")` on stubs — or simply `raise NotImplementedError` in the test body so pytest reports them as ERROR (not passing). The goal: tests that unambiguously fail until the implementation exists.

**tests/search/__init__.py** — Empty file with package docstring:
```python
"""Tests for the corpus-analyzer search module."""
```

**tests/search/test_engine.py** — Covers SEARCH-01 through SEARCH-05, CLASS-02, CLASS-03, SUMM-02, CLI-02, CLI-03, CLI-04:

```python
"""Tests for CorpusSearch hybrid search engine.

Tests cover: hybrid search returns results, RRF fusion, filters by source/type/construct,
snippet extraction integration, construct_type stored per chunk, summary in results,
end-to-end search, all filter flags, and status command.
"""
from __future__ import annotations

import pytest

# --- Shared fixture (implement in GREEN phase) ---
# All tests use a tmp_path LanceDB fixture with pre-seeded chunks


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_hybrid_search_returns_results() -> None:
    """SEARCH-01: hybrid_search() returns a non-empty list for a matching query."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_hybrid_uses_rrf() -> None:
    """SEARCH-02: hybrid_search() uses RRFReranker (_relevance_score present in results)."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_filter_by_source() -> None:
    """SEARCH-03: source filter limits results to chunks with matching source_name."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_filter_by_file_type() -> None:
    """SEARCH-04: file_type filter limits results to chunks with matching file_type."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_construct_type_stored() -> None:
    """CLASS-02: construct_type column is populated after index_source() with classification."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_filter_by_construct() -> None:
    """CLASS-03: construct filter limits results to chunks with matching construct_type."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_summary_in_results() -> None:
    """SUMM-02: summary field is present and non-empty in search results for indexed file."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_search_end_to_end() -> None:
    """CLI-02: hybrid_search() returns list of dicts with file_path, snippet, _relevance_score."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_all_filters() -> None:
    """CLI-03: source + file_type + construct filters chain as AND predicates."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_status_returns_stats() -> None:
    """CLI-04: status() returns dict with files, chunks, last_indexed, model keys."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="CorpusSearch not yet implemented")
def test_zero_results_no_crash() -> None:
    """SEARCH-01: hybrid_search() with no matching query returns empty list (not error)."""
    raise NotImplementedError
```

**tests/search/test_classifier.py** — Covers CLASS-01:

```python
"""Tests for ConstructClassifier rule-based and LLM fallback classification."""
from __future__ import annotations

import pytest
from pathlib import Path


@pytest.mark.xfail(strict=True, reason="ConstructClassifier not yet implemented")
def test_classify_skill_by_path() -> None:
    """CLASS-01: file under skills/ path segment classifies as 'skill'."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="ConstructClassifier not yet implemented")
def test_classify_prompt_by_path() -> None:
    """CLASS-01: file under prompts/ path segment classifies as 'prompt'."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="ConstructClassifier not yet implemented")
def test_classify_workflow_by_path() -> None:
    """CLASS-01: file under workflows/ path segment classifies as 'workflow'."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="ConstructClassifier not yet implemented")
def test_classify_code_by_extension() -> None:
    """CLASS-01: .py / .ts / .js files classify as 'code'."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="ConstructClassifier not yet implemented")
def test_classify_agent_config_by_frontmatter() -> None:
    """CLASS-01: MD with name + description + tools frontmatter classifies as 'agent_config'."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="ConstructClassifier not yet implemented")
def test_classify_skill_by_frontmatter() -> None:
    """CLASS-01: MD with name + description (no tools) frontmatter classifies as 'skill'."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="ConstructClassifier not yet implemented")
def test_llm_fallback_called_when_rules_unclear() -> None:
    """CLASS-01: File with no rule signals triggers LLM fallback (mock ollama.generate)."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="ConstructClassifier not yet implemented")
def test_fallback_to_documentation_when_llm_disabled() -> None:
    """CLASS-01: When LLM fallback is disabled, unclear files classify as 'documentation'."""
    raise NotImplementedError
```

**tests/search/test_summarizer.py** — Covers SUMM-01, SUMM-03:

```python
"""Tests for Summarizer AI summary generation."""
from __future__ import annotations

import pytest


@pytest.mark.xfail(strict=True, reason="Summarizer not yet implemented")
def test_generate_summary() -> None:
    """SUMM-01: generate_summary() returns a non-empty string (mock ollama.generate)."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="Summarizer not yet implemented")
def test_summary_is_one_or_two_sentences() -> None:
    """SUMM-01: generate_summary() result contains 1-2 sentences."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="Summarizer not yet implemented")
def test_skip_when_summarize_false() -> None:
    """SUMM-03: generate_summary() returns None when source.summarize is False."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="Summarizer not yet implemented")
def test_skip_when_content_unchanged() -> None:
    """SUMM-01: generate_summary() skips generation when file hash matches stored hash."""
    raise NotImplementedError
```

**tests/search/test_formatter.py** — Covers SEARCH-05:

```python
"""Tests for snippet extraction and Rich result formatting."""
from __future__ import annotations

import pytest


@pytest.mark.xfail(strict=True, reason="extract_snippet not yet implemented")
def test_extract_snippet_short_text() -> None:
    """SEARCH-05: text with <= max_lines lines is returned in full."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="extract_snippet not yet implemented")
def test_extract_snippet_finds_best_line() -> None:
    """SEARCH-05: snippet window centers around the line containing the most query terms."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="extract_snippet not yet implemented")
def test_extract_snippet_truncates_at_word_boundary() -> None:
    """SEARCH-05: snippet longer than 200 chars is truncated at word boundary with ellipsis."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="extract_snippet not yet implemented")
def test_extract_snippet_empty_query() -> None:
    """SEARCH-05: empty query falls back to first max_lines lines without error."""
    raise NotImplementedError
```

**tests/ingest/test_indexer.py** — Append two new failing tests to the existing file. Read the existing file first and append at the bottom:

```python
# --- Phase 2 extension tests ---

@pytest.mark.xfail(strict=True, reason="index_source classify+summarize not yet implemented")
def test_index_adds_construct_type(tmp_path: Path) -> None:
    """CLI-01: After index_source(), chunks have a non-None construct_type value."""
    raise NotImplementedError


@pytest.mark.xfail(strict=True, reason="index_source classify+summarize not yet implemented")
def test_index_skips_summary_on_unchanged_file(tmp_path: Path) -> None:
    """CLI-01: Second index_source() call on unchanged file does not re-generate summary."""
    raise NotImplementedError
```

After creating all files, run the full test suite to confirm they are xfail (not accidentally passing):
`uv run pytest tests/search/ tests/ingest/test_indexer.py -v --no-header 2>&1 | tail -30`

The xfail tests should show as `XFAIL` in the output (expected failure = correct RED state).
  </action>
  <verify>
    <automated>uv run pytest tests/search/ tests/ingest/test_indexer.py -v --no-header 2>&1 | grep -E "(XFAIL|XPASS|ERROR|PASSED|FAILED)" | head -40</automated>
    <manual>All new tests should show XFAIL (not PASSED, FAILED, or ERROR). Any ERROR means a syntax issue in the test file that must be fixed.</manual>
    <sampling_rate>run after this task commits</sampling_rate>
  </verify>
  <done>
    - tests/search/__init__.py exists
    - tests/search/test_engine.py has 11 xfail test stubs
    - tests/search/test_classifier.py has 8 xfail test stubs
    - tests/search/test_summarizer.py has 4 xfail test stubs
    - tests/search/test_formatter.py has 4 xfail test stubs
    - tests/ingest/test_indexer.py has 2 new xfail stubs appended
    - All new tests report XFAIL when run
    - Existing tests continue to pass
  </done>
</task>

</tasks>

<verification>
Run the full test suite to confirm no regressions from schema changes:
`uv run pytest -v --no-header 2>&1 | tail -20`

Run lint and type check on modified source files:
`uv run ruff check src/corpus_analyzer/store/schema.py src/corpus_analyzer/config/schema.py src/corpus_analyzer/search/__init__.py && uv run mypy src/corpus_analyzer/store/ src/corpus_analyzer/config/ src/corpus_analyzer/search/`
</verification>

<success_criteria>
- ChunkRecord has construct_type: str | None = None and summary: str | None = None
- ensure_schema_v2() is importable from corpus_analyzer.store.schema
- SourceConfig has summarize: bool = True
- All Wave 0 test files exist and are importable
- All new tests are in XFAIL state (red, not passing)
- All existing tests continue to pass
- mypy and ruff pass on modified files
</success_criteria>

<output>
After completion, create `.planning/phases/02-search-core/02-01-SUMMARY.md` with:
- Schema changes made (fields added, functions added)
- Test files created and their test count
- Any deviations from plan
- Import paths for ensure_schema_v2, ConstructClassifier, Summarizer (for downstream plans to reference)
</output>
