---
phase: 06-frontmatter-classification
plan: 02
type: execute
wave: 2
depends_on:
  - 06-01
files_modified:
  - src/corpus_analyzer/store/schema.py
  - src/corpus_analyzer/ingest/indexer.py
  - tests/ingest/test_indexer.py
autonomous: true
requirements:
  - CLASS-04
  - CLASS-05

must_haves:
  truths:
    - "The chunks table stores `classification_source` (e.g. 'frontmatter' or 'rule_based') and `classification_confidence` (float) per chunk"
    - "Running `corpus index` on a source with a file containing `type: skill` writes classification_source='frontmatter' and classification_confidence=0.95 to the index"
    - "Existing tables without these columns are upgraded automatically on next index open (no data loss)"
    - "Files classified by rule-based signals continue to index correctly with source='rule_based' and confidence=0.6"
    - "The full test suite remains green after wiring"
  artifacts:
    - path: "src/corpus_analyzer/store/schema.py"
      provides: "classification_source and classification_confidence nullable fields on ChunkRecord; ensure_schema_v3()"
      contains: "classification_source"
    - path: "src/corpus_analyzer/ingest/indexer.py"
      provides: "Updated chunk dict building to extract .construct_type, .confidence, .source from ClassificationResult"
    - path: "tests/ingest/test_indexer.py"
      provides: "Test confirming classification_source and classification_confidence are stored correctly"
  key_links:
    - from: "indexer.py index_source()"
      to: "classifier.classify_file()"
      via: "ClassificationResult destructuring into chunk_dict keys"
      pattern: "classify_result\\.construct_type|classify_result\\.confidence|classify_result\\.source"
    - from: "ensure_schema_v3()"
      to: "CorpusIndex.open()"
      via: "called after ensure_schema_v2 on every table open"
      pattern: "ensure_schema_v3"
---

<objective>
Wire ClassificationResult into the LanceDB schema and indexer pipeline.

Purpose: Plan 01 changed classify_file() to return a ClassificationResult rather than a plain str. This plan updates the storage layer (ChunkRecord schema) and the indexer (chunk dict construction) to persist the new confidence and source fields. This completes the end-to-end frontmatter classification feature.

Output: Updated store/schema.py (new fields + migration), updated ingest/indexer.py (uses ClassificationResult), and a regression/integration test.
</objective>

<execution_context>
@/home/ollie/.claude/get-shit-done/workflows/execute-plan.md
@/home/ollie/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/06-frontmatter-classification/06-CONTEXT.md
@.planning/phases/06-frontmatter-classification/06-01-SUMMARY.md
@src/corpus_analyzer/store/schema.py
@src/corpus_analyzer/ingest/indexer.py
@src/corpus_analyzer/search/classifier.py
</context>

<tasks>

<task type="auto">
  <name>Task 1: Add classification_source and classification_confidence to ChunkRecord schema</name>
  <files>
    src/corpus_analyzer/store/schema.py
  </files>
  <action>
    1. Add two nullable fields to the `ChunkRecord` LanceModel class after the existing Phase 2 fields:

       ```python
       # --- Phase 6 fields (nullable, added via schema evolution) --------------
       classification_source: str | None = None
       """Source of construct type classification: 'frontmatter', 'rule_based', or 'llm'.

       Set to None for chunks indexed before Phase 6. Populated during corpus index
       after Phase 6 lands.
       """

       classification_confidence: float | None = None
       """Confidence score for construct type classification (0.0–1.0).

       Set to None for chunks indexed before Phase 6. Frontmatter matches → 0.95,
       tags matches → 0.70, rule-based → 0.60, LLM → 0.80.
       """
       ```

    2. Add a new `ensure_schema_v3(table)` function directly below `ensure_schema_v2()`:

       ```python
       def ensure_schema_v3(table: lancedb.table.Table) -> None:
           """Add Phase 6 nullable columns to an existing chunks table if they don't exist.

           Uses LanceDB's add_columns API for in-place schema evolution — no data loss
           and no table rebuild required. Safe to call on every CorpusIndex.open() call.

           Args:
               table: The LanceDB chunks table to upgrade.
           """
           existing_cols = {field.name for field in table.schema}
           if "classification_source" not in existing_cols:
               table.add_columns({"classification_source": "cast(NULL as string)"})
           if "classification_confidence" not in existing_cols:
               table.add_columns({"classification_confidence": "cast(NULL as double)"})
       ```

    3. Update the docstring for the schema lock-in warning at the top of the module to mention
       that classification_source and classification_confidence were added in Phase 6.

    Do NOT change any existing fields. The `construct_type` field stays as-is.
  </action>
  <verify>
    <automated>uv run mypy src/corpus_analyzer/store/schema.py && uv run ruff check src/corpus_analyzer/store/schema.py</automated>
    <manual>Confirm ChunkRecord has classification_source and classification_confidence fields. Confirm ensure_schema_v3 is defined.</manual>
  </verify>
  <done>
    ChunkRecord has two new nullable fields: classification_source (str | None) and classification_confidence (float | None).
    ensure_schema_v3() function exists and correctly adds both columns.
    mypy and ruff clean on schema.py.
  </done>
</task>

<task type="auto">
  <name>Task 2: Wire ClassificationResult into indexer and call ensure_schema_v3</name>
  <files>
    src/corpus_analyzer/ingest/indexer.py
    tests/ingest/test_indexer.py
  </files>
  <action>
    ## indexer.py changes

    1. Update the `ensure_schema_v2` import to also import `ensure_schema_v3`:

       ```python
       from corpus_analyzer.store.schema import ChunkRecord, ensure_schema_v2, ensure_schema_v3, make_chunk_id
       ```

    2. In `CorpusIndex.open()`, call `ensure_schema_v3(table)` immediately after the existing
       `ensure_schema_v2(table)` call:

       ```python
       ensure_schema_v2(table)
       ensure_schema_v3(table)
       ```

    3. In `index_source()`, update the classification call to capture the full `ClassificationResult`:

       Change:
       ```python
       construct_type = classify_file(
           file_path,
           full_text,
           model=self._embedder.model,
           use_llm=source.use_llm_classification,
       )
       ```

       To:
       ```python
       classify_result = classify_file(
           file_path,
           full_text,
           model=self._embedder.model,
           use_llm=source.use_llm_classification,
       )
       ```

    4. In the chunk dict construction, replace `"construct_type": construct_type` with:

       ```python
       "construct_type": classify_result.construct_type,
       "classification_source": classify_result.source,
       "classification_confidence": classify_result.confidence,
       ```

    ## tests/ingest/test_indexer.py changes

    Add a test `test_indexer_stores_frontmatter_classification` that:
    1. Creates a temp source directory with one `.md` file containing `type: skill` frontmatter
    2. Creates a mock `OllamaEmbedder` (same pattern as existing indexer tests)
    3. Calls `index_source()` with `use_llm_classification=False`
    4. Queries the table and asserts that the stored chunk has:
       - `construct_type == "skill"`
       - `classification_source == "frontmatter"`
       - `classification_confidence == 0.95`

    Follow the existing test patterns in `tests/ingest/test_indexer.py` exactly — use `tmp_path`,
    real LanceDB table, mocked embedder.
  </action>
  <verify>
    <automated>uv run pytest tests/ingest/test_indexer.py -v && uv run mypy src/corpus_analyzer/ingest/indexer.py && uv run ruff check src/corpus_analyzer/ingest/indexer.py</automated>
    <manual>Confirm the new test passes and no existing indexer tests regress.</manual>
  </verify>
  <done>
    indexer.py calls ensure_schema_v3() on open.
    Chunk dicts include classification_source and classification_confidence from ClassificationResult.
    New integration test passes: frontmatter `type: skill` → stored with source="frontmatter", confidence=0.95.
    Full test suite green: uv run pytest -v.
    mypy and ruff clean on indexer.py.
  </done>
</task>

</tasks>

<verification>
uv run pytest -v
uv run ruff check .
uv run mypy src/
</verification>

<success_criteria>
1. ChunkRecord has classification_source (str | None) and classification_confidence (float | None) fields
2. ensure_schema_v3() adds these columns to existing tables without data loss
3. indexer.py correctly unpacks ClassificationResult and writes all three fields to chunk dicts
4. Integration test confirms end-to-end: md file with frontmatter type: skill → stored classification_source="frontmatter", classification_confidence=0.95
5. Full test suite green (uv run pytest -v)
6. ruff and mypy clean across all modified files
</success_criteria>

<output>
After completion, create `.planning/phases/06-frontmatter-classification/06-02-SUMMARY.md` summarizing:
- Schema fields added and migration function
- Indexer changes (lines changed, what was updated)
- Test added
- Commits created
- Confirmation that Phase 6 is complete (both CLASS-04 and CLASS-05 satisfied)
</output>
