# Writing a Custom Parser

Extractor's parser system is plugin-based. Any class that extends `BaseParser` and registers itself with the parser registry can handle a new file format.

## 1. Understand the data model

Parsers produce a stream of `RawElement` objects. Each becomes a `BaseElement` after normalization by the pipeline.

**`RawElement`** (from `extractor.parsers.base`):
- `element_type: str` — must be one of `ELEMENT_TYPES` (see `extractor.schema`)
- `text: str` — the primary text content
- `page: int | None` — 1-based page number
- `heading_level: int | None` — 1–6 for headings
- `detection_tier: int` — 1=native structure, 2=heuristic, 3=fallback, 4=raw
- `extra: dict` — arbitrary metadata; `extraction_method` and `confidence` are read by the pipeline
- `table_data: dict | None` — structured table data (see `TableData` schema)
- `figure_data`, `equation_data`, `image_data`, etc.

## 2. Create your parser class

```python
# mypackage/parsers/rtf.py
from extractor.parsers.base import BaseParser, FileRecord, RawElement
from typing import Iterator

class RTFParser(BaseParser):
    handles = [
        ("application/rtf", None),   # (mime_type, subtype) — subtype None = catch-all
    ]

    def parse(self, file_record: FileRecord) -> Iterator[RawElement]:
        # file_record.path — pathlib.Path to the file
        # file_record.sha256 — hex SHA-256 for deduplication
        with open(file_record.path, "rb") as f:
            content = f.read()

        # ... parse content ...

        yield RawElement(
            element_type="structural:title",
            text="Document Title",
            heading_level=1,
            detection_tier=1,
            extra={"extraction_method": "rtf:header", "confidence": 0.95},
        )
        yield RawElement(
            element_type="text:narrative",
            text="Body paragraph text...",
            extra={"extraction_method": "rtf:paragraph", "confidence": 0.90},
        )
```

## 3. Register your parser

```python
# Call this at import time (e.g., in your package's __init__.py)
from extractor.registry import registry
from mypackage.parsers.rtf import RTFParser

registry.register(RTFParser())
```

Or add it to extractor's built-in `src/extractor/parsers/register.py`.

## 4. Supported MIME types

Use `extractor info` to see all currently registered MIME types, or call:
```python
from extractor.registry import registry
print(registry.list_supported())
```

## 5. Element types

The full list of valid `element_type` values is in `extractor.schema.ELEMENT_TYPES`. Key types:

| Type | Use for |
|------|---------|
| `structural:title` | Document title (heading level 1) |
| `structural:section_header` | Section headings (levels 2–6) |
| `text:narrative` | Body paragraphs |
| `list:item` | Unordered list items |
| `list:item_ordered` | Ordered list items |
| `table:simple` | Tables with no merged cells |
| `table:complex` | Tables with colspan/rowspan |
| `code:block` | Fenced code blocks |
| `media:figure` | Figures with captions |
| `media:image` | Standalone images |
| `scientific:equation_display` | Block equations |

## 6. Testing your parser

```python
# tests/test_my_rtf_parser.py
from pathlib import Path
from extractor import extract

def test_rtf_no_crash(tmp_path):
    rtf_file = tmp_path / "test.rtf"
    rtf_file.write_bytes(b"{\\rtf1 Hello World}")
    elements = list(extract(rtf_file, mode="elements"))
    assert len(elements) > 0

def test_rtf_element_types(tmp_path):
    from extractor.schema import ELEMENT_TYPES
    rtf_file = ...
    for el in extract(rtf_file, mode="elements"):
        assert el.element_type in ELEMENT_TYPES
```
