# GLiNER2 Integration — Cycle 4: Architecture Design

**Status:** Historical design document. Implemented in v1.2.0.
**Note:** Remote API tier (PIONEER_API_KEY) was removed. Extractor uses local inference only: `fastino/gliner2-multi-v1` via `[ner]` extras.
**Target:** extractor v1.2.0, schema v1.1.0

---

## 1. Module Structure Overview

```
src/extractor/
  ner.py              # NEW: NERStep class, model singleton, batching, element filter
  schema.py           # MODIFIED: EntityAnnotation model, entities field on BaseElement/ChunkRecord
  chunker.py          # MODIFIED: entity rollup from elements to chunks
  cli.py              # MODIFIED: --entities, --entities-model, --entities-types, --entities-threshold flags
  writer.py           # MODIFIED: NER stats accumulation for manifest
  pipeline.py         # NO CHANGE to internals; NER wraps the output stream
```

---

## 2. New Module: `src/extractor/ner.py`

### 2.1 Full Class Design

```python
"""
extractor.ner — GLiNER2 named entity recognition annotator.

This module is an OPTIONAL feature. Import is guarded: `gliner` is only
imported inside NERStep.__init__() to avoid ImportError for users who
did not install extractor[ner].

Usage:
    from extractor.ner import NERStep, preload_model
    step = NERStep(labels=["organization", "person", "date"], threshold=0.50)
    elements = step.annotate(elements)  # lazy generator
"""

from __future__ import annotations

import logging
from collections.abc import Generator, Iterable
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from extractor.schema import BaseElement

logger = logging.getLogger(__name__)

# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------

DEFAULT_REMOTE_MODEL_ID = "gliner-xl-1b"   # served at gliner.pioneer.ai (PIONEER_API_KEY)
DEFAULT_LOCAL_MODEL_ID  = "fastino/gliner2-multi-v1"  # fallback when no API key

DEFAULT_LABELS: list[str] = [
    "organization",
    "person",
    "date",
    "location",
    "product",
    "technology",
    "event",
    "quantity",
]

# Element types on which NER must NOT run (categorical skip).
# Checked against element.element_type before any inference.
NER_SKIP_TYPES: frozenset[str] = frozenset({
    "code:block",
    "code:inline",
    "scientific:equation_display",
    "scientific:equation_inline",
    "scientific:citation",
    "scientific:theorem",
    "scientific:proof",
    "scientific:algorithm",
    "structural:page_header",
    "structural:page_footer",
    "structural:page_break",
    "structural:subtitle",
    "media:image",
    "media:figure",
    "media:diagram",
    "media:chart",
    "text:uncategorized",
    "meta:document_title",
    "meta:author",
    "meta:date",
    "meta:address",
    "meta:email",
    "meta:url",
    "meta:page_number",
})

# Element types where a higher per-type minimum threshold applies.
NER_HIGH_THRESHOLD_TYPES: dict[str, float] = {
    "structural:section_header": 0.80,
    "structural:title": 0.80,
    "structural:subsection_header": 0.80,
}

# Element types where a lower per-type minimum threshold applies.
NER_LOW_THRESHOLD_TYPES: dict[str, float] = {
    "table:simple": 0.55,
    "table:complex": 0.55,
    "table:continuation": 0.55,
}

# Maximum cl100k tokens to feed to GLiNER2.
# cl100k 256 tokens ≈ 300–350 BERT wordpiece tokens — safely under BERT 512 limit.
# Silent truncation is not acceptable; use W_NER_INPUT_TRUNCATED warning.
NER_MAX_CL100K_TOKENS: int = 256

# Batch sizes by token-length bucket (cl100k tokens).
BATCH_SIZES: dict[tuple[int, int], int] = {
    (0, 64): 16,
    (65, 128): 8,
    (129, 256): 4,
    (257, 512): 2,
}

# ---------------------------------------------------------------------------
# Module-level singleton
# ---------------------------------------------------------------------------

_model_singleton: "NERModel | None" = None  # type: ignore[type-arg]


def preload_model(use_fp16: bool = True) -> None:
    """
    Eagerly initialize the GLiNER2 model singleton.

    Automatically selects remote API (if PIONEER_API_KEY is set) or local
    model fallback (if [ner] extras installed). Call before a document
    processing loop to eliminate cold-start latency on the first document.
    """
    global _model_singleton
    if _model_singleton is None:
        _model_singleton = _load_model(use_fp16=use_fp16)
        logger.info("NER model preloaded")


def _load_model(use_fp16: bool = True) -> "NERModel":  # type: ignore[type-arg]
    """
    Load GLiNER2 model using the two-tier strategy:

    Tier 1 — Remote API (default, preferred):
        Requires PIONEER_API_KEY env var.
        Uses the 1B XL model served at gliner.pioneer.ai via GLiNER2.from_api().
        No local install of torch/gliner needed for this path.

    Tier 2 — Local fallback:
        Used automatically when PIONEER_API_KEY is not set.
        Requires: pip install extractor[ner]
        Uses fastino/gliner2-multi-v1 (205M, CPU-first, fp16).
        W_NER_MODEL_UNAVAILABLE emitted if [ner] extras also not installed.
    """
    import os
    api_key = os.environ.get("PIONEER_API_KEY")

    if api_key:
        try:
            from gliner2 import GLiNER2  # type: ignore[import-untyped]
            logger.info("NER: remote API (gliner.pioneer.ai, XL 1B model)")
            return GLiNER2.from_api()
        except ImportError as exc:
            raise ImportError(
                "GLiNER2 remote API client requires: pip install gliner2"
            ) from exc
    else:
        # Local fallback — requires [ner] extras
        try:
            from gliner import GLiNER  # type: ignore[import-untyped]
        except ImportError as exc:
            raise ImportError(
                "No PIONEER_API_KEY set and local gliner not installed. "
                "Set PIONEER_API_KEY (recommended) or run: pip install extractor[ner]"
            ) from exc
        logger.info("NER: local model %s (PIONEER_API_KEY not set)", DEFAULT_LOCAL_MODEL_ID)
        model = GLiNER.from_pretrained(DEFAULT_LOCAL_MODEL_ID)
        if use_fp16:
            try:
                model = model.half()
            except Exception:
                logger.warning("fp16 not available on this platform; using fp32")
        return model  # type: ignore[return-value]


# ---------------------------------------------------------------------------
# NERStep
# ---------------------------------------------------------------------------

class NERStep:
    """
    Named Entity Recognition annotator for the extractor element stream.

    Wraps the element stream as a lazy generator. Elements are batched,
    sent through GLiNER2, and emitted with entity annotations attached.
    Elements on the NER_SKIP_TYPES list are passed through unmodified.

    Args:
        labels: Entity type labels to extract. Uses DEFAULT_LABELS if not provided.
        threshold: Global minimum confidence score. Spans below threshold are dropped.
        model_id: GLiNER2 model ID (HuggingFace hub or local path).
        use_fp16: Use half-precision. Default True.
        batch_size: Override automatic batch size selection.
    """

    def __init__(
        self,
        labels: list[str] | None = None,
        threshold: float = 0.50,
        model_id: str = DEFAULT_MODEL_ID,
        use_fp16: bool = True,
        batch_size: int | None = None,
    ) -> None:
        self.labels = labels or DEFAULT_LABELS
        self.threshold = threshold
        self.model_id = model_id
        self.use_fp16 = use_fp16
        self._batch_size_override = batch_size

        # Validate labels
        for label in self.labels:
            if not label.islower() or "_" in label or not label.replace(" ", "").isalpha():
                logger.warning(
                    "Entity label '%s' does not follow canonical naming convention "
                    "(lowercase singular noun, no underscores). Results may vary.",
                    label,
                )

        # Lazy model load — deferred until first call to annotate()
        self._model: object | None = None

        # Statistics accumulated during annotation
        self.stats: NERStats = NERStats()

    def _ensure_model(self) -> None:
        global _model_singleton
        if self._model is None:
            if _model_singleton is not None and _model_singleton._model_id == self.model_id:  # type: ignore[attr-defined]
                self._model = _model_singleton
            else:
                self._model = _load_model(self.model_id, self.use_fp16)

    def annotate(
        self,
        elements: Iterable["BaseElement"],
    ) -> Generator["BaseElement", None, None]:
        """
        Annotate an element stream with entity annotations.

        Returns a lazy generator. NER inference is deferred until the generator
        is consumed. Elements are yielded in original order with `entities` field
        populated (or absent if element type is skipped).

        Failure isolation: if inference fails for a batch, affected elements are
        yielded with entities=[] and a warning is logged. Extraction never fails
        due to NER errors.
        """
        self._ensure_model()

        # Collect the full stream to enable batching by length.
        # Memory cost: all elements in RAM — this is already the case in the
        # current pipeline (chunker collects elements). Not a new constraint.
        all_elements: list["BaseElement"] = list(elements)

        # Partition into eligible and skip
        eligible_indices: list[int] = []
        for i, elem in enumerate(all_elements):
            if elem.element_type not in NER_SKIP_TYPES:
                eligible_indices.append(i)

        # Run inference on eligible elements
        self._run_inference_on_indices(all_elements, eligible_indices)

        # Yield all elements in original order
        yield from all_elements

    def _get_ner_text(self, element: "BaseElement") -> tuple[str, bool]:
        """
        Extract the text to run NER on for a given element.

        For table elements: use structured row cell text concatenated,
        NOT the pipe-delimited markdown text (markdown adds noise).
        For all others: use element.text.

        Returns:
            (text, was_truncated): text to run NER on, and whether it was truncated.
        """
        if element.element_type in ("table:simple", "table:complex", "table:continuation"):
            if hasattr(element, "table") and element.table is not None:  # type: ignore[attr-defined]
                rows = getattr(element.table.structured, "rows", None) if hasattr(element.table, "structured") else None  # type: ignore[attr-defined]
                if rows:
                    cell_texts = " | ".join(
                        " ".join(str(cell) for cell in row) for row in rows
                    )
                    return self._truncate_text(cell_texts)
        return self._truncate_text(element.text)

    def _truncate_text(self, text: str) -> tuple[str, bool]:
        """
        Truncate text to NER_MAX_CL100K_TOKENS cl100k tokens.

        Uses a character-count heuristic (4 chars per token) to avoid the
        tiktoken import overhead per element. For precise truncation, replace
        with tiktoken.encode() if the dependency is already loaded.

        Returns:
            (text, was_truncated)
        """
        # Heuristic: 4 chars per cl100k token on average
        max_chars = NER_MAX_CL100K_TOKENS * 4
        if len(text) <= max_chars:
            return text, False
        # Truncate at last space before limit to avoid mid-word cuts
        truncated = text[:max_chars].rsplit(" ", 1)[0]
        return truncated, True

    def _get_effective_threshold(self, element: "BaseElement") -> float:
        """Get the effective confidence threshold for this element type."""
        elem_type = element.element_type
        if elem_type in NER_HIGH_THRESHOLD_TYPES:
            return max(self.threshold, NER_HIGH_THRESHOLD_TYPES[elem_type])
        if elem_type in NER_LOW_THRESHOLD_TYPES:
            return min(self.threshold, NER_LOW_THRESHOLD_TYPES[elem_type])
        return self.threshold

    def _auto_batch_size(self, token_count: int) -> int:
        if self._batch_size_override is not None:
            return self._batch_size_override
        for (lo, hi), size in BATCH_SIZES.items():
            if lo <= token_count <= hi:
                return size
        return 2  # default for very long texts

    def _run_inference_on_indices(
        self,
        all_elements: list["BaseElement"],
        eligible_indices: list[int],
    ) -> None:
        """
        Run batched GLiNER2 inference on eligible elements.

        Batching strategy:
        1. Sort eligible elements by estimated token count (length-bucketing)
        2. Group into batches of similar length
        3. Run inference per batch
        4. Write EntityAnnotation objects back to elements in-place
        5. On batch failure: log warning, set entities=[] on affected elements
        """
        from extractor.schema import EntityAnnotation  # avoid circular import

        if not eligible_indices:
            return

        # Sort by text length (proxy for token count) — minimizes padding waste
        sorted_indices = sorted(
            eligible_indices,
            key=lambda i: len(all_elements[i].text),
        )

        # Build batches
        batches: list[list[int]] = []
        current_batch: list[int] = []
        current_batch_size = self._auto_batch_size(len(all_elements[sorted_indices[0]].text) // 4)

        for idx in sorted_indices:
            elem = all_elements[idx]
            elem_token_estimate = len(elem.text) // 4
            ideal_batch_size = self._auto_batch_size(elem_token_estimate)

            if len(current_batch) >= ideal_batch_size:
                batches.append(current_batch)
                current_batch = [idx]
                current_batch_size = ideal_batch_size
            else:
                current_batch.append(idx)

        if current_batch:
            batches.append(current_batch)

        # Process batches
        for batch_indices in batches:
            texts: list[str] = []
            truncation_flags: list[bool] = []

            for idx in batch_indices:
                elem = all_elements[idx]
                text, was_truncated = self._get_ner_text(elem)
                texts.append(text)
                truncation_flags.append(was_truncated)
                if was_truncated:
                    self.stats.truncated_elements += 1
                    logger.debug(
                        "NER input truncated for element %s (type=%s)",
                        elem.id,
                        elem.element_type,
                    )

            try:
                # GLiNER2 batch predict
                # Returns: list[list[dict]] — outer list per text, inner list per entity
                batch_results = self._model.batch_predict_entities(  # type: ignore[union-attr]
                    texts,
                    self.labels,
                    threshold=0.0,  # apply our own threshold logic
                )

                for elem_pos, (idx, raw_entities) in enumerate(
                    zip(batch_indices, batch_results)
                ):
                    elem = all_elements[idx]
                    effective_threshold = self._get_effective_threshold(elem)

                    annotations: list[EntityAnnotation] = []
                    for ent in raw_entities:
                        score = float(ent.get("score", 0.0))
                        if score < effective_threshold:
                            continue
                        annotation = EntityAnnotation(
                            entity_type=ent["label"],
                            text=ent["text"],
                            start=ent.get("start", 0),
                            end=ent.get("end", len(ent["text"])),
                            score=round(score, 4),
                            extraction_method=f"ner:{self.model_id.split('/')[-1]}",
                        )
                        annotations.append(annotation)

                    # Set entities on element (in-place mutation)
                    elem.entities = annotations

                    # Update stats
                    self.stats.elements_annotated += 1
                    self.stats.total_spans += len(annotations)
                    for ann in annotations:
                        self.stats.by_type[ann.entity_type] = (
                            self.stats.by_type.get(ann.entity_type, 0) + 1
                        )
                    if annotations:
                        self.stats.elements_with_entities += 1

            except Exception as exc:
                logger.warning(
                    "NER inference failed for batch of %d elements: %s. "
                    "Setting entities=[] for affected elements.",
                    len(batch_indices),
                    exc,
                )
                for idx in batch_indices:
                    all_elements[idx].entities = []
                self.stats.inference_errors += 1

    @property
    def manifest_stats(self) -> dict:
        """Return manifest-compatible stats block for this NER run."""
        return {
            "model": self.model_id,
            "labels": self.labels,
            "threshold": self.threshold,
            "elements_annotated": self.stats.elements_annotated,
            "elements_with_entities": self.stats.elements_with_entities,
            "total_spans": self.stats.total_spans,
            "by_type": self.stats.by_type,
            "truncated_elements": self.stats.truncated_elements,
            "inference_errors": self.stats.inference_errors,
            "mean_score": self.stats.mean_score,
        }


class NERStats:
    """Mutable stats accumulator for a single NER annotation pass."""

    def __init__(self) -> None:
        self.elements_annotated: int = 0
        self.elements_with_entities: int = 0
        self.total_spans: int = 0
        self.by_type: dict[str, int] = {}
        self.truncated_elements: int = 0
        self.inference_errors: int = 0
        self._scores: list[float] = []

    @property
    def mean_score(self) -> float | None:
        if not self._scores:
            return None
        return round(sum(self._scores) / len(self._scores), 4)
```

---

## 3. Schema Changes — `src/extractor/schema.py`

### 3.1 New Model: `EntityAnnotation`

```python
class EntityAnnotation(BaseModel):
    """
    A single named entity annotation on an element or chunk.

    On element records: `start` and `end` are character offsets into element.text.
    On chunk records (rollup): `start`/`end` are absent; `source_element_id` is present.

    Stability: experimental (schema v1.1.0). Promoted to stable in v1.2.0.
    """
    model_config = ConfigDict(extra="forbid")

    entity_type: str
    """Canonical entity label. Must match a registered label from the canonical label set."""

    text: str
    """Verbatim surface form as it appears in the source text."""

    start: int | None = None
    """Inclusive character offset into element.text. Present on element records only."""

    end: int | None = None
    """Exclusive character offset into element.text. Present on element records only."""

    score: float
    """GLiNER2 confidence score in [0.0, 1.0]."""

    extraction_method: str
    """Follows extractor token convention: 'ner:gliner2-multi-v1'."""

    source_element_id: str | None = None
    """
    Element ID that produced this entity. Present on chunk records (rollup) only.
    Absent on element records (the element itself is the source).
    """
```

### 3.2 Modifications to `BaseElement`

Add `entities` as an optional field:

```python
class BaseElement(BaseModel):
    # ... existing fields unchanged ...

    entities: list[EntityAnnotation] | None = None
    """
    Named entity annotations for this element.

    ABSENT (None, serialized as field omission) when NER was not run.
    Empty list [] when NER was run but no entities found above threshold.
    Never null in JSON — use field omission pattern.

    Stability: experimental (schema v1.1.0).
    """

    model_config = ConfigDict(
        # Use exclude_none=True at serialization to omit absent fields
        extra="ignore",
    )
```

Serialization rule: when serializing to JSON, `entities: None` must be **omitted entirely** (not serialized as `null`). This distinguishes "NER not run" from "NER ran, no entities found." Use `model.model_dump(exclude_none=True)` at the writer layer.

### 3.3 Modifications to `ChunkMetadata`

```python
class ChunkMetadata(BaseModel):
    # ... existing fields unchanged ...

    entity_types: list[str] | None = None
    """
    Deduplicated sorted list of entity_type values from all entities in this chunk.
    Example: ["date", "organization", "person"]

    ABSENT when NER was not run.
    Empty list [] when NER ran but no entities found.

    Enables metadata-only filtering without unpacking the full entities array.
    Mirrors the has_table/has_code pattern.

    Stability: experimental (schema v1.1.0).
    """
```

### 3.4 Modifications to `ChunkRecord`

```python
class ChunkRecord(BaseModel):
    # ... existing fields unchanged ...

    entities: list[EntityAnnotation] | None = None
    """
    Rolled-up entity annotations from all contributing elements.

    Deduplication: entities with identical (entity_type, text) pairs are merged.
    When merging, the highest score is kept. source_element_ids are unioned
    into a list (but EntityAnnotation.source_element_id holds the primary
    source — the element with the highest score for that span).

    start/end fields are NOT present on chunk-level EntityAnnotation objects
    (offsets into concatenated chunk text are not computed — too expensive
    and of limited value compared to element-level offsets).

    Stability: experimental (schema v1.1.0).
    """
```

---

## 4. Chunker Modifications — `src/extractor/chunker.py`

When flushing a chunk buffer, aggregate element-level entities into the chunk record:

```python
def _build_chunk_entities(
    contributing_elements: list[BaseElement],
) -> list[EntityAnnotation] | None:
    """
    Aggregate element-level entity annotations into a chunk-level rollup.

    Returns None if no contributing element has an entities field set
    (i.e., NER was not run).

    Deduplication: entities with the same (entity_type, text) are merged.
    The entity with the highest score is kept. source_element_id is set
    to the element that produced the highest-score span.
    """
    any_ner_run = any(
        elem.entities is not None for elem in contributing_elements
    )
    if not any_ner_run:
        return None  # NER was not run; field will be absent

    # Merge by (entity_type, normalized_text) key
    seen: dict[tuple[str, str], EntityAnnotation] = {}
    for elem in contributing_elements:
        if elem.entities is None:
            continue
        for ann in elem.entities:
            key = (ann.entity_type, ann.text.lower())
            if key not in seen or ann.score > seen[key].score:
                chunk_ann = EntityAnnotation(
                    entity_type=ann.entity_type,
                    text=ann.text,
                    score=ann.score,
                    extraction_method=ann.extraction_method,
                    source_element_id=elem.id,
                    # start/end omitted at chunk level
                )
                seen[key] = chunk_ann

    return list(seen.values()) if seen else []


def _build_chunk_entity_types(
    entities: list[EntityAnnotation] | None,
) -> list[str] | None:
    """Build the deduplicated sorted entity_types summary."""
    if entities is None:
        return None
    return sorted(set(e.entity_type for e in entities))
```

These functions are called in the chunk flush path, after contributing elements are collected.

---

## 5. CLI Changes — `src/extractor/cli.py`

### New Flags on `run` Command

```python
@app.command()
def run(
    # ... existing parameters unchanged ...

    entities: bool = typer.Option(
        True,
        "--entities/--no-entities",
        help=(
            "Extract named entities using GLiNER2 (default: enabled). "
            "When gliner2 is not installed, extraction proceeds normally with a "
            "W_NER_MODEL_UNAVAILABLE manifest warning — no hard failure. "
            "Use --no-entities to disable explicitly. "
            "Adds ~2–6s overhead per 100-page document (fp16, CPU). "
            "Model is loaded once and cached for the process lifetime."
        ),
    ),
    entities_model: str = typer.Option(
        "fastino/gliner2-multi-v1",
        "--entities-model",
        help="GLiNER2 model ID (HuggingFace hub or local path).",
    ),
    entities_types: str | None = typer.Option(
        None,
        "--entities-types",
        help=(
            "Comma-separated entity labels to extract. "
            "Default: organization,person,date,location,product,technology,event,quantity. "
            "Example: --entities-types 'organization,person,date'"
        ),
    ),
    entities_threshold: float = typer.Option(
        0.50,
        "--entities-threshold",
        min=0.0,
        max=1.0,
        help="Minimum GLiNER2 confidence score. Spans below threshold are dropped.",
    ),
) -> None:
    ...
```

### Pipeline Integration in `run()`

```python
# After pipeline.run(), before writer:
ner_step: NERStep | None = None
if entities:
    try:
        from extractor.ner import NERStep, preload_model
        labels = [l.strip() for l in entities_types.split(",")] if entities_types else None
        ner_step = NERStep(
            labels=labels,
            threshold=entities_threshold,
            model_id=entities_model,
        )
        preload_model(model_id=entities_model)
    except ImportError:
        # Neither remote API key nor local [ner] extras available.
        # Graceful fallback — W_NER_MODEL_UNAVAILABLE written to manifest.
        typer.echo(
            "Warning: entity extraction skipped. "
            "Set PIONEER_API_KEY for remote inference (recommended), "
            "or run: pip install extractor[ner] for local inference.",
            err=True,
        )
        ner_step = None

element_stream = pipeline.run(path=target, ...)
if ner_step is not None:
    element_stream = ner_step.annotate(element_stream)

writer.write(element_stream, ...)
```

---

## 6. BERT Token Limit Handling Strategy

### The Problem
GLiNER2 uses a BERT-family encoder. BERT has a hard 512 wordpiece token limit. The extractor default `max_chunk_tokens=512` uses cl100k tokenization (GPT-4). The relationship:
- 512 cl100k tokens ≈ 600–768 BERT wordpiece tokens (ratio ~1.2–1.5x due to different tokenization)
- A 512 cl100k chunk regularly exceeds the BERT 512 limit
- GLiNER2's behavior on inputs that exceed 512 BERT tokens is model-dependent: some truncate silently, some produce garbage output for trailing tokens

### The Solution
NER input is truncated to `NER_MAX_CL100K_TOKENS = 256` cl100k tokens before inference:
- 256 cl100k tokens ≈ 300–384 BERT wordpiece tokens — well within the 512 BERT limit
- Character-count heuristic: `max_chars = 256 * 4 = 1024 chars` (avoids tiktoken import overhead per element)
- Truncation at last word boundary (rsplit on space)
- `stats.truncated_elements` counter in manifest for observability
- Debug-level log per truncated element (not warning-level — truncation is expected for long elements)

### Why 256 and Not 512?
- The 1.5x tokenizer ratio is a conservative upper bound; the safe upper bound for cl100k is floor(512 / 1.5) = 341
- 256 provides a 33% safety margin against unusual tokenization ratios in domain-specific text (technical jargon, legal terminology, chemical names all have higher BERT-to-cl100k ratios)
- Entity-rich content is concentrated in the first N tokens of a paragraph — truncating later content has low entity recall impact in practice

### Future Option: Two-Pass NER
For v1.3.0, implement BERT-token-aware splitting: count actual BERT wordpiece tokens, split at 480-token boundary, run two inference passes, merge results with deduplication. Not for v1.2.0 — the complexity/benefit ratio is poor given that entity recall after truncation at 256 cl100k is already high.

---

## 7. Element Type Filter — Exact Implementation

The `NER_SKIP_TYPES` frozenset in `ner.py` is the canonical implementation. Checked as:

```python
if element.element_type in NER_SKIP_TYPES:
    # Pass through unmodified; entities field remains absent (None)
    continue
```

For `table:*` types: NER runs on `table.structured.rows` cell text (concatenated), NOT on `element.text` (which is the pipe-delimited markdown representation — noisy for NER).

For `scientific:reference_entry`: NER runs normally, but only `person`, `organization`, and `date` labels should produce meaningful results. The caller can subset labels; the implementation does not apply per-element label filtering (that would require per-element label configuration — too complex for v1.2.0).

---

## 8. Batching Algorithm — Full Pseudocode

```
function run_inference_batched(elements, eligible_indices, model, labels, threshold):

    # Step 1: Sort by text length (proxy for BERT token count)
    sorted_indices = sort(eligible_indices, key=lambda i: len(elements[i].text))

    # Step 2: Group into length-homogeneous batches
    batches = []
    current_batch = []
    for idx in sorted_indices:
        token_estimate = len(elements[idx].text) / 4  # chars / 4 ≈ cl100k tokens
        ideal_batch_size = lookup_batch_size(token_estimate)
        if len(current_batch) >= ideal_batch_size:
            batches.append(current_batch)
            current_batch = [idx]
        else:
            current_batch.append(idx)
    if current_batch:
        batches.append(current_batch)

    # Step 3: Process each batch
    for batch in batches:
        texts = []
        for idx in batch:
            text, truncated = get_ner_text(elements[idx])  # truncate to 256 tokens
            texts.append(text)

        try:
            results = model.batch_predict_entities(texts, labels, threshold=0.0)
            for (idx, entity_list) in zip(batch, results):
                effective_threshold = get_effective_threshold(elements[idx])
                annotations = [
                    EntityAnnotation(entity)
                    for entity in entity_list
                    if entity.score >= effective_threshold
                ]
                elements[idx].entities = annotations  # in-place mutation
        except Exception as e:
            log.warning("NER batch failed: %s", e)
            for idx in batch:
                elements[idx].entities = []  # empty, not absent — NER was attempted
            stats.inference_errors += 1

function lookup_batch_size(token_estimate):
    if token_estimate <= 64: return 16
    if token_estimate <= 128: return 8
    if token_estimate <= 256: return 4
    return 2
```

---

## 9. Model Singleton Pattern

```python
# Module-level singleton (process-scoped)
_model_singleton: GLiNER | None = None

def preload_model(model_id: str = DEFAULT_MODEL_ID, use_fp16: bool = True) -> None:
    global _model_singleton
    if _model_singleton is None:
        _model_singleton = _load_model(model_id, use_fp16)

class NERStep:
    def _ensure_model(self) -> None:
        global _model_singleton
        if self._model is None:
            if _model_singleton is not None:
                self._model = _model_singleton
            else:
                self._model = _load_model(self.model_id, self.use_fp16)
                _model_singleton = self._model  # cache for future NERStep instances
```

**Properties:**
- First `NERStep` instantiation in a process: cold load (~3–5s)
- Subsequent `NERStep` instances: reuse cached singleton (0ms)
- `preload_model()` before document loop: eliminates cold start on first document
- Multiple workers (v1.2.0 batch API): each process has its own singleton — documented
- Memory cost: ~600 MB RSS per worker process at fp16

---

## 10. Manifest Changes

### `run_config` additions

```json
"run_config": {
  "extract_entities": true,
  "entities_model": "fastino/gliner2-multi-v1",
  "entities_labels": ["organization", "person", "date", "location", "product", "technology", "event", "quantity"],
  "entities_threshold": 0.50
}
```

### `parser_chain` addition (when `--entities` is active)

```json
{
  "step": 6,
  "name": "ner_annotator",
  "impl": "gliner:gliner2-multi-v1",
  "duration_ms": 3240
}
```

### `stats.entities` block

```json
"stats": {
  "entities": {
    "total_spans": 847,
    "elements_annotated": 312,
    "elements_with_entities": 218,
    "truncated_elements": 14,
    "inference_errors": 0,
    "mean_score": 0.82,
    "by_type": {
      "organization": 124,
      "person": 89,
      "date": 67,
      "location": 45,
      "product": 38,
      "technology": 31,
      "event": 22,
      "quantity": 431
    }
  }
}
```

### New Warning Codes

| Code | Trigger | Description |
|---|---|---|
| `W_NER_MODEL_UNAVAILABLE` | Neither `PIONEER_API_KEY` set nor `[ner]` extras installed | Entity extraction skipped. Set `PIONEER_API_KEY` (remote, recommended) or `pip install extractor[ner]` (local fallback). Extraction continues normally. |
| `W_NER_INFERENCE_FAILED` | Exception in batch inference | NER batch failed; entities=[] for affected elements |
| `W_NER_INPUT_TRUNCATED` | Element text exceeded 256 cl100k tokens | Entity recall may be incomplete for long elements |

---

## 11. Integration Point Summary

```
pipeline.run(path) → Iterator[BaseElement]
         ↓
[if --entities]
NERStep.annotate(elements) → Iterator[BaseElement with .entities populated]
         ↓
[if --mode chunks]
chunker.chunk(elements) → Iterator[ChunkRecord with .entities rollup]
         ↓
writer.write(records) → JSONL output
         ↓
manifest.write() → manifest.json with stats.entities block
```

No changes to `pipeline.py` internals. No changes to parser chain. No changes to any existing element serialization. The NER step is a transparent wrapper that adds an optional field.
