# `.to_llm()` Implementation Plan

## Test PDFs

| PDF | Type | Text | Lines | Rects | Key features |
|-----|------|------|-------|-------|-------------|
| `01-practice.pdf` | Inspection report | 44 words, 3 sizes | 10h, 5v | 8 | Grid lines, bold labels, mixed sizes |
| `cookbook/vendor_invoice.pdf` | Invoice | 42 words, 4 sizes | 7h, 5v | 1 | Bold headers, line-item table, few rects |
| `cookbook/scanned_form.pdf` | Application form | 34 words, 5 sizes | 10h, 0v | 5 | Horizontal lines only, form fields, checkboxes likely |
| `needs-ocr.pdf` | Scanned doc | 0 words | 0, 0 | 0 | Full-page image, 100% coverage |
| `multicolumn.pdf` | Multi-column data | 174 words, 1 size | 0, 0 | 662 | No lines but many rects, single font size |

## Stages

### Stage 1: Service wiring + TEXT LAYER section
- Create `ToLLMService` with `@register_delegate`
- Wire into `PDFContext`, `ServiceNamespace`
- Add `to_llm()` method to Page, Region, ElementCollection, Element, PDF, PageCollection
- Implement TEXT LAYER: word count by source, image count + coverage
- **Test on all 5 PDFs, show output**
- **Consensus: Is the wiring clean? Is TEXT LAYER output correct and useful?**

### Stage 2: STYLES & CONTENT section
- Group text elements by `(round(size, 0), bold, italic)`
- Sort tiers by size descending
- Per tier: count, x-position clustering (sort-and-group, ±3pt tolerance), text samples (top 5 per cluster, sorted by y then x)
- Generate selector strings from tier properties
- **Test on all 5 PDFs, show output**
- **Consensus: Are the tiers sensible? Are the clusters right? Are selectors valid?**

### Stage 3: LINES + RECTANGLES sections
- LINES: Count by orientation, cluster positions (±2pt tolerance), detect regular spacing (gap variance < 20%), compute spans (min x0, max x1 per horizontal cluster; min top, max bottom per vertical cluster)
- RECTANGLES: Count, size-bucket (small <20pt, medium 20-100pt, large >100pt), detect "square-ish" clusters (|w-h| < 3pt), position clusters
- **Test on all 5 PDFs, show output**
- **Consensus: Are line clusters meaningful? Do rectangle groups make sense?**

### Stage 4: ALIGNMENT section
- Global x-clustering across all text elements (sort-and-group, ±3pt tolerance)
- Report clusters with >3% of total elements
- Include y-range per cluster
- **Test on all 5 PDFs, show output**
- **Consensus: Do clusters match visible column structure?**

### Stage 5: LAYOUT PREVIEW section
- Call `extract_text(layout=True)`, take first N lines (15 default, configurable)
- Handle empty pages gracefully
- **Test on all 5 PDFs, show output**
- **Consensus: Does the preview add value alongside structured sections?**

### Stage 6: Full page.to_llm() assembly
- Combine all sections with section headers
- Implement `detail` parameter (brief/standard/full)
- Implement `include_text` flag
- Implement `include_hints` flag (none/descriptive/api)
- Apply hard caps on all variable-length lists
- **Test on all 5 PDFs at all detail levels, show output**
- **Consensus: Token counts, overall effectiveness, missing pieces?**

### Stage 7: Region + Element + ElementCollection levels
- `region.to_llm()`: layout text within bounds + local styles + local lines
- `element.to_llm()`: element details + nearby elements (within 50pt) + local lines + same-style summary
- `collection.to_llm()`: extent, style distribution, samples, alignment, spacing
- **Test: find elements on 01-practice.pdf and vendor_invoice.pdf, show region/element output**
- **Consensus: Does the local context help for targeted extraction?**

### Stage 8: PDF / PageCollection level
- Per-page digest: word count, style tier count, line counts, top anchors (from largest/boldest text)
- Cross-page patterns: repeated text at similar y-positions (header/footer detection)
- **Test on multi-page PDFs (use multipage-table-african-recipes.pdf or Atlanta PDF)**
- **Consensus: Does the routing view help agents pick the right pages?**

### Stage 9: Include-if-computed sections
- If `analyze_layout()` was run: show detected regions
- If `detect_checkboxes()` was run: show checkbox count/states
- If `detect_form_cells()` was run: show cell count
- **Test: run analyze_layout() on a page, then show to_llm() with results**
- **Consensus: Clean integration? Useful?**

## Implementation notes

- Clustering algorithm: sort values, walk sorted list, start new cluster when gap > tolerance. O(n log n).
- Font size rounding: round to nearest 0.5pt to handle Word-exported PDF drift.
- Text sampling: per tier, per x-cluster, take first 5 by y-position. Cap at 5 per cluster, 3 clusters per tier.
- Selector generation: use tier's min/max size ±1pt for range, add :bold/:italic as appropriate.
- Line span computation: per y-cluster of horizontal lines, report min(x0) to max(x1). Same for vertical.
- Square detection: |width - height| < 3pt AND max(width, height) < 20pt.
