---
phase: 16-integration-hardening
plan: 02
type: tdd
wave: 2
depends_on:
  - 16-01
files_modified:
  - src/corpus_analyzer/ingest/ts_chunker.py
autonomous: true
requirements:
  - IDX-08
  - IDX-09

must_haves:
  truths:
    - "A TypeScript file exceeding 50,000 characters falls back to chunk_lines() — no chunk_name in output"
    - "chunk_typescript() completes normally when _get_cached_parser raises ImportError — returns chunk_lines() output"
    - "ImportError and Exception are separate except clauses — not merged into except (ImportError, Exception)"
    - "tree_sitter imports are no longer at module level — importing ts_chunker succeeds when tree-sitter is absent"
    - "All 320 tests pass (318 pre-existing + 2 new tests from 16-01)"
  artifacts:
    - path: "src/corpus_analyzer/ingest/ts_chunker.py"
      provides: "Size guard: if len(source) > 50_000: return chunk_lines(path)"
      contains: "50_000"
    - path: "src/corpus_analyzer/ingest/ts_chunker.py"
      provides: "ImportError branch: separate except ImportError clause before except Exception"
      contains: "except ImportError"
    - path: "src/corpus_analyzer/ingest/ts_chunker.py"
      provides: "Lazy import of get_parser inside _get_cached_parser() body (not at module level)"
      contains: "from tree_sitter_language_pack import get_parser"
  key_links:
    - from: "src/corpus_analyzer/ingest/ts_chunker.py::chunk_typescript"
      to: "corpus_analyzer.ingest.chunker.chunk_lines"
      via: "size guard: len(source) > 50_000 early return"
      pattern: "50_000"
    - from: "src/corpus_analyzer/ingest/ts_chunker.py::chunk_typescript"
      to: "corpus_analyzer.ingest.chunker.chunk_lines"
      via: "except ImportError branch after _get_cached_parser() call"
      pattern: "except ImportError"
    - from: "src/corpus_analyzer/ingest/ts_chunker.py::_get_cached_parser"
      to: "tree_sitter_language_pack.get_parser"
      via: "lazy function-local import inside _get_cached_parser body"
      pattern: "from tree_sitter_language_pack import get_parser"
---

<objective>
TDD GREEN phase: restructure ts_chunker.py so that tree-sitter imports become lazy (function-local inside _get_cached_parser), then add the size guard (IDX-08) and the ImportError fallback branch (IDX-09) to chunk_typescript(). Both new tests from 16-01 must turn GREEN. All 320 tests must pass.

Purpose: Make the TypeScript chunker production-safe against minified files (size guard) and missing tree-sitter installs (ImportError fallback).
Output: src/corpus_analyzer/ingest/ts_chunker.py restructured with both guards implemented.
</objective>

<execution_context>
@/home/ollie/.claude/get-shit-done/workflows/execute-plan.md
@/home/ollie/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/16-integration-hardening/16-CONTEXT.md
@.planning/phases/16-integration-hardening/16-RESEARCH.md
@.planning/phases/15-core-ast-chunker/15-02-SUMMARY.md
@src/corpus_analyzer/ingest/ts_chunker.py
@tests/ingest/test_chunker.py
</context>

<tasks>

<task type="tdd">
  <name>Task 1: Restructure ts_chunker.py — lazy imports + size guard + ImportError branch</name>
  <files>src/corpus_analyzer/ingest/ts_chunker.py</files>
  <action>
Apply three surgical changes to `src/corpus_analyzer/ingest/ts_chunker.py`. The locked decisions from CONTEXT.md are NON-NEGOTIABLE:

**Change 1 — Move tree-sitter imports from module level to function scope:**

REMOVE these two module-level imports (lines 14-15 currently):
```python
from tree_sitter import Parser
from tree_sitter_language_pack import get_parser
```

The `Parser` type annotation is only used in `_get_cached_parser`'s return type. Since `from __future__ import annotations` is already at line 8 (all annotations are strings at runtime), add a `TYPE_CHECKING` guard:

```python
from typing import TYPE_CHECKING, Any

if TYPE_CHECKING:
    from tree_sitter import Parser
```

Move `from tree_sitter_language_pack import get_parser` into the body of `_get_cached_parser()`:

```python
@lru_cache(maxsize=8)
def _get_cached_parser(dialect: str) -> "Parser":
    """Return a cached tree-sitter Parser for the given grammar dialect.

    Args:
        dialect: One of "typescript", "tsx", or "javascript".

    Returns:
        Configured tree_sitter.Parser instance (loaded once per dialect per process).
    """
    from tree_sitter_language_pack import get_parser  # lazy: allows ImportError catch at call site
    return get_parser(dialect)  # type: ignore[arg-type]
```

**Change 2 — Add size guard (IDX-08) inside chunk_typescript(), after the empty-source check:**

Insert immediately after the `if not source.strip(): return []` line:

```python
# IDX-08: size guard — minified/generated files bypass AST parse
if len(source) > 50_000:
    return chunk_lines(path)
```

**Change 3 — Add explicit ImportError branch (IDX-09) as a separate except clause:**

The existing try/except wrapping `_get_cached_parser` and `parser.parse` currently has only `except Exception:`. Split into TWO explicit branches per the locked decision:

BEFORE:
```python
    try:
        parser = _get_cached_parser(dialect)
        tree = parser.parse(source.encode("utf-8"))
    except Exception:
        return chunk_lines(path)
```

AFTER (separate clauses — do NOT merge into `except (ImportError, Exception):`):
```python
    # IDX-09: ImportError branch — tree-sitter not installed; separate from has_error fallback
    try:
        parser = _get_cached_parser(dialect)
        tree = parser.parse(source.encode("utf-8"))
    except ImportError:
        return chunk_lines(path)
    except Exception:
        return chunk_lines(path)
```

**mypy considerations:**
- `"Parser"` as a string annotation for `_get_cached_parser` return type works because `from __future__ import annotations` is present.
- If mypy complains about the return type of `_get_cached_parser` (since `Parser` is now only in TYPE_CHECKING), change the return type annotation to `Any` and add `# type: ignore[return-value]` or just use `Any`. The existing `# type: ignore[arg-type]` on `get_parser(dialect)` already suppresses Literal mismatch.
- Alternative: keep `"Parser"` string annotation and add `from tree_sitter import Parser` under `if TYPE_CHECKING:` — mypy uses this for type checking but it's not imported at runtime.

**After changes, the complete imports section of ts_chunker.py should look like:**
```python
from __future__ import annotations

from functools import lru_cache
from pathlib import Path
from typing import TYPE_CHECKING, Any

if TYPE_CHECKING:
    from tree_sitter import Parser
```

**Verification steps to run during implementation:**
```bash
# Run only the two new RED tests — they must now be GREEN
uv run pytest tests/ingest/test_chunker.py::TestChunkTypeScript::test_large_file_falls_back_to_chunk_lines tests/ingest/test_chunker.py::TestChunkTypeScript::test_import_error_falls_back_to_chunk_lines -v

# Run full test suite — all 320 must pass
uv run pytest --tb=short -q

# Lint check — must exit 0
uv run ruff check src/corpus_analyzer/ingest/ts_chunker.py

# Type check — must exit 0
uv run mypy src/
```

**TDD commit after all pass:**
`feat(16-02): add size guard and ImportError fallback to chunk_typescript`
  </action>
  <verify>
    <automated>uv run pytest tests/ingest/test_chunker.py::TestChunkTypeScript::test_large_file_falls_back_to_chunk_lines tests/ingest/test_chunker.py::TestChunkTypeScript::test_import_error_falls_back_to_chunk_lines -v && uv run pytest --tb=no -q 2>&1 | tail -3 && uv run ruff check src/corpus_analyzer/ingest/ts_chunker.py && uv run mypy src/</automated>
    <manual>Confirm both new tests GREEN; confirm no module-level tree_sitter import remains in ts_chunker.py; confirm separate except ImportError and except Exception clauses exist</manual>
  </verify>
  <done>Both test_large_file_falls_back_to_chunk_lines and test_import_error_falls_back_to_chunk_lines PASS. All 320 tests pass. ruff exits 0. mypy exits 0. ts_chunker.py has no module-level tree_sitter imports. Two separate except clauses (ImportError, Exception) exist in chunk_typescript().</done>
</task>

</tasks>

<verification>
After completing this plan:

```bash
# Both guard tests GREEN
uv run pytest tests/ingest/test_chunker.py::TestChunkTypeScript::test_large_file_falls_back_to_chunk_lines tests/ingest/test_chunker.py::TestChunkTypeScript::test_import_error_falls_back_to_chunk_lines -v

# Full suite green
uv run pytest --tb=no -q 2>&1 | tail -3

# Lint clean
uv run ruff check .

# Type clean
uv run mypy src/

# Confirm no module-level tree_sitter import
grep -n "^from tree_sitter" src/corpus_analyzer/ingest/ts_chunker.py | wc -l  # must be 0

# Confirm separate except clauses exist
grep -n "except ImportError\|except Exception" src/corpus_analyzer/ingest/ts_chunker.py
```
</verification>

<success_criteria>
1. `test_large_file_falls_back_to_chunk_lines` passes — 50,001-char file returns chunk_lines output (no chunk_name)
2. `test_import_error_falls_back_to_chunk_lines` passes — patched _get_cached_parser raises ImportError, chunk_typescript returns non-empty list without chunk_name
3. All 320 tests pass
4. `uv run ruff check .` exits 0
5. `uv run mypy src/` exits 0
6. No module-level `from tree_sitter` import in ts_chunker.py
7. Two separate `except ImportError:` and `except Exception:` clauses exist (NOT merged)
8. TDD GREEN commit created: `feat(16-02): add size guard and ImportError fallback to chunk_typescript`
</success_criteria>

<output>
After completion, create `.planning/phases/16-integration-hardening/16-16-02-SUMMARY.md`
</output>
