# Changelog

## 1.0.0 (2026-03-05)

### Added
- **LangChain loader** (`integrations/langchain.py`) — `PDFMuxLoader(path, quality)` wraps `load_llm_context()`, returns `list[Document]` with metadata. Lazy import of langchain-core.
- **LlamaIndex reader** (`integrations/llamaindex.py`) — `PDFMuxReader(quality)` wraps `load_llm_context()`, returns `list[Document]`. Lazy import of llama-index-core.
- **Security hardening** — file size limit (500MB), page count limit (10,000), configurable via `PDFMUX_MAX_FILE_SIZE_MB` and `PDFMUX_MAX_PAGES` env vars.
- **API stability tests** — signature-level tests lock function parameters, JSON schema fields, and `__all__` exports for the 1.x series.
- **New optional extras**: `pdfmux[langchain]`, `pdfmux[llamaindex]`, updated `pdfmux[all]`.
- 19 new tests: API stability (10 tests), integrations (6 tests), security (3 tests).

### Changed
- **Interface locked** — public API signatures frozen for 1.x (additive-only keyword args allowed). JSON schema frozen at 1.0.0.
- Classifier upgraded to "Production/Stable".
- JSON schema version bumped to `1.0.0`.
- Total test count: 151.

## 0.9.0 (2026-03-05)

### Added
- **Region OCR** (`regions.py`) — surgical OCR of image regions within pages. `detect_weak_regions()` finds images without overlapping text. `ocr_region()` clips and OCRs just the image area. `merge_region_text()` inserts results at correct y-position. "Bad" pages now preserve good text while recovering image content.
- **Enhanced MCP server** — 2 new tools: `analyze_pdf` (quick triage — classify + audit without extraction) and `batch_convert` (process entire directories). Error responses now include structured error codes.
- **`WeakRegion` type** (`types.py`) — frozen dataclass with `page_num`, `bbox`, `reason`. Exported from `pdfmux`.
- **Example scripts** (`examples/`) — `basic_usage.py`, `batch_processing.py`, `mcp_agent.py` with self-contained usage examples.
- 14 new tests: region OCR (8 tests) and enhanced MCP (6 tests).

### Changed
- Multi-pass pipeline now tries region OCR on "bad" pages before falling back to full-page OCR.
- JSON schema version bumped to `0.9.0`.
- Total test count: 132.

## 0.8.0 (2026-03-05)

### Added
- **Column detection** (`detect.py`) — `detect_layout(page)` detects multi-column PDFs by clustering text block x-positions with gap detection. Returns `PageLayout` with column count, boundaries, and reading order.
- **Layout-aware extraction** (`fast.py`) — `_needs_reorder()` samples first 5 pages; if multi-column detected, `_extract_with_layout()` reorders blocks column-by-column. Single-column PDFs take existing fast path with zero overhead.
- **Block-level scoring** (`audit.py`) — `score_block(text)` applies 3 lightweight quality checks (alphabetic ratio, word structure, encoding quality) at individual text block granularity.
- **`PageLayout` type** (`types.py`) — frozen dataclass with `columns`, `column_boundaries`, `reading_order`. Exported from `pdfmux`.
- 8 new tests: layout detection (5 tests) and block scoring (3 tests).

### Changed
- JSON schema version bumped to `0.8.0`.
- Total test count: 118.

## 0.7.0 (2026-03-05)

### Added
- **Structured error codes** — every exception now has a `.code` class attribute (`PDF_NOT_FOUND`, `PDF_CORRUPTED`, `EXTRACTION_ERROR`, `PARTIAL_EXTRACTION`, `NO_EXTRACTOR`, `FORMAT_ERROR`, `AUDIT_ERROR`, `OCR_TIMEOUT`). Backward-compatible: existing catch blocks still work.
- **`OCRTimeoutError`** — new exception for OCR timeout scenarios. Exported from `pdfmux`.
- **Provenance on chunks** — `Chunk` dataclass now carries `extractor` and `ocr_applied` fields. Propagated through `chunk_by_sections()` and `format_llm()` output.
- **JSON `error_code` field** — JSON output includes `error_code` (null on success). Per-page `ocr` boolean flag in pages array.
- **CLI logging** — `--verbose` (INFO), `--debug` (DEBUG), `--quiet` (ERROR only) flags on `pdfmux convert`.
- 17 new tests: error codes (13 tests) and provenance (4 tests).

### Changed
- `FileError` and `ExtractionError` accept optional `code=` keyword for specific error codes.
- LLM format output now includes `extractor` and `ocr_applied` per chunk.
- JSON schema version bumped to `0.7.0`.
- Total test count: 110.

## 0.6.0 (2026-03-05)

### Added
- **Parallel OCR dispatch** (`parallel.py`) — OCR re-extraction now runs across 4 threads via `ThreadPoolExecutor`. ONNX runtime releases the GIL during inference, giving real parallelism. Per-page timing and error isolation via `PageOCRResult` frozen dataclass.
- **OCR budget control** — Standard mode caps OCR at 30% of document pages. Prioritizes "bad" pages (some text) over "empty" pages. `quality=high` ignores the budget. Override with `PDFMUX_OCR_BUDGET` env var.
- **Windowed audit** — `audit_document()` now processes pages in windows of 50 instead of loading the entire document at once. Bounds memory on 500+ page PDFs.
- 7 new tests: parallel OCR dispatch (4 tests) and budget control logic (3 tests).

### Changed
- Multi-pass pipeline uses parallel OCR dispatch instead of serial page-by-page loop.
- JSON schema version bumped to `0.6.0`.
- Total test count: 93.

## 0.5.0 (2026-03-05)

### Added
- **Typed architecture** — 6 frozen dataclasses and enums in `types.py` (`Quality`, `OutputFormat`, `PageQuality`, `PageResult`, `DocumentResult`, `Chunk`). Every data flow in the pipeline now passes through typed, immutable objects.
- **Error hierarchy** — flat exception tree in `errors.py`: `PdfmuxError` base with `FileError`, `ExtractionError`, `ExtractorNotAvailable`, `FormatError`, `AuditError`. All exported from `import pdfmux`.
- **Streaming extractors** — all 5 extractors yield `Iterator[PageResult]`, one page at a time. Memory stays bounded even on 500-page PDFs (~135MB peak vs unbounded before).
- **Extractor protocol + registry** — `Extractor` Protocol class + `@register(name, priority)` decorator for auto-registration. Programmatic access via `get_extractor()`, `available_extractors()`, `extractor_names()`.
- **5-check confidence scoring** — per-page confidence computed from character density, alphabetic ratio, word structure, whitespace sanity, and encoding quality (mojibake detection). Content-weighted document average replaces the old heuristic scorer.
- **Concurrent batch processing** — `process_batch()` with `ThreadPoolExecutor`. Error isolation per file — one failure doesn't stop the batch. Used by CLI batch conversion.

### Changed
- JSON schema version bumped to `0.5.0`.
- All public types and errors exported from the top-level `pdfmux` package (`pdfmux.PageResult`, `pdfmux.FileError`, etc.).
- Extractors conform to a common `Extractor` protocol and register via decorator instead of hardcoded lookup.
- Confidence scoring is now deterministic and auditable — 5 named checks with individual scores, content-weighted aggregation.
- Extractor names simplified: `"pymupdf4llm (fast)"` → `"pymupdf4llm"`, `"docling (tables)"` → `"docling"`, etc.
- `detect.py` now raises `FileError` instead of `FileNotFoundError`/`ValueError` for consistency with the error hierarchy.

### Fixed
- Memory usage no longer scales linearly with page count during extraction (streaming architecture).

## 0.4.0 (2026-03-04)

### Added
- **Public Python API** — three importable functions: `extract_text()`, `extract_json()`, `load_llm_context()`. No more CLI-only usage.
- **Section-aware chunking** (`chunking.py`) — splits Markdown at heading boundaries with per-chunk page tracking and token estimates (chars/4). Powers `load_llm_context()` and `--format llm`.
- **LLM output format** — `pdfmux report.pdf -f llm` outputs chunked JSON with `{title, text, page_start, page_end, tokens, confidence}` per section. Designed for RAG pipelines and context windows.
- **`pdfmux analyze`** — per-page extraction breakdown showing page type (digital/graphical/scanned), quality (good/bad/empty), char count, confidence, and extractor used.
- **Locked JSON schema** — JSON output now includes `schema_version: "0.4.0"` and `ocr_pages` field for downstream stability.

### Changed
- **JSON output** now includes `schema_version` and `ocr_pages` fields in every response.
- **`--format` option** accepts `llm` in addition to `markdown`, `json`, `csv`.

## 0.3.0 (2026-03-04)

### Added
- **Multi-pass extraction** — fast extract → per-page audit → selective OCR → merge. All standard-mode PDFs now go through this pipeline. Zero overhead when all pages are good.
- **RapidOCR extractor** — lightweight OCR using PaddleOCR v4 models via ONNX runtime. ~200MB install, CPU-only, Apache 2.0 license. Replaces Surya as default `pdfmux[ocr]`.
- **Per-page quality auditing** (`audit.py`) — classifies each page as "good", "bad", or "empty" based on text density and image presence. Drives selective re-extraction.
- **Smart OCR comparison** — for "bad" pages (some text), only uses OCR if it extracts MORE text than fast extraction. For "empty" pages, any OCR text >10 chars is accepted.
- **OCR fallback chain** — RapidOCR → Surya → Gemini Flash LLM. Each step only processes pages the previous step couldn't recover.
- **`ocr_pages` tracking** — `ConversionResult` now reports which pages were re-extracted with OCR.
- **Multi-pass in bench** — `pdfmux bench` now includes a "Multi-pass" row showing the full pipeline result.
- **RapidOCR in doctor** — `pdfmux doctor` now checks for RapidOCR installation.

### Changed
- **`pdfmux[ocr]`** now installs RapidOCR + onnxruntime (~200MB) instead of Surya (~5GB). Surya moved to `pdfmux[ocr-heavy]`.
- **Routing simplified** — removed `_handle_graphical_pdf()` and `_handle_mixed_pdf()`. Multi-pass handles all PDF types uniformly.
- **Graphical + tables routing** — graphical PDFs no longer route to Docling even if table heuristics trigger. Multi-pass OCR is more valuable than table formatting for image-heavy content.
- **Confidence scoring** — OCR-recovered pages get a small penalty for OCR noise (max 15%) instead of the large "extraction_limited" penalty.

### Fixed
- Pitch decks and slide exports now get 70-85% confidence with OCR installed (was 30-55%)
- Digital PDFs maintain identical confidence and zero overhead through multi-pass fast path
- RapidOCR logging noise suppressed (model paths, engine info no longer pollute output)

## 0.2.2 (2026-03-04)

### Added
- **Graphical PDF detection** — detects image-heavy PDFs (pitch decks, infographics) and routes to OCR/LLM instead of fast extraction
- **Honest confidence scoring** — confidence now reflects actual extraction quality, not just text presence. Graphical PDFs with missing image content score lower.
- **Actionable warnings** — clear messages when extraction is limited, with specific `pip install pdfmux[ocr]` or `pdfmux[llm]` suggestions
- **MCP server quality metadata** — AI agents now receive confidence score and warnings alongside extracted text
- **Spaced-text cleanup** — fixes common PDF artifact where text renders as "W i t h  o v e r" → "With over"

### Fixed
- Detection no longer classifies image-heavy "digital" PDFs as fully extractable
- Confidence no longer reports 100% on graphical PDFs where image content was missed
- FastExtractor now falls back to raw fitz when pymupdf4llm returns empty (fixes certain PDF encodings)
- bench command now shows honest confidence that matches pipeline routing

## 0.2.1 (2026-03-04)

### Added
- **`pdfmux doctor`** — check installed extractors, versions, and API keys
- **`pdfmux bench`** — benchmark all available extractors on a PDF side by side

### Fixed
- Suppressed upstream pymupdf4llm "Consider using pymupdf_layout" noise from all commands

## 0.2.0 (2026-03-03)

First public release.

### Added
- **Smart routing** — auto-detect PDF type and pick the best extractor
- **PyMuPDF extractor** — digital PDFs at 0.01s/page
- **Docling extractor** — 97.9% table accuracy (optional: `pdfmux[tables]`)
- **Surya OCR extractor** — scanned PDF support (optional: `pdfmux[ocr]`)
- **Gemini Flash extractor** — complex layout fallback (optional: `pdfmux[llm]`)
- **Mixed PDF handling** — digital pages + scanned pages merged automatically
- **Output formats** — Markdown, JSON, CSV
- **Quality presets** — fast, standard, high
- **Batch conversion** — convert entire directories
- **MCP server** — built-in stdio server for AI agents
- **Confidence scoring** — text completeness, encoding quality, structure checks
- **Graceful fallback** — missing extractors fall back silently to next best
