# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

pdfalive is a Python (3.13) library and CLI tool that uses LLMs to enhance PDF files. It provides:
- **Automatic Table of Contents (bookmarks) generation** for PDFs using LLM inference, with optional postprocessing refinement
- **OCR processing** for scanned PDFs using Tesseract integration
- **Intelligent file renaming** using LLM inference for batch renaming with confidence scoring

## Development Commands

```bash
# Install dependencies (uses uv package manager)
uv sync

# Run the CLI - TOC generation
uv run pdfalive generate-toc examples/example.pdf output.pdf --force

# Run the CLI - OCR text extraction
uv run pdfalive extract-text input.pdf output.pdf

# Run the CLI - Intelligent file renaming
uv run pdfalive rename -q "Rename to '[Author] - [Title].pdf'" *.pdf

# Linting
uv run ruff check .
uv run ruff format .

# Type checking
uv run mypy pdfalive

# Run tests
uv run pytest
```

## Architecture

The codebase follows a processor pattern where document operations are encapsulated in processor classes.

```
pdfalive/
├── cli.py                 # Click-based CLI entry point
├── tokens.py              # Token counting utilities
├── prompts.py             # LLM system prompts
├── config/
│   ├── __init__.py        # Config module exports
│   ├── models.py          # Pydantic models for config validation
│   └── loader.py          # TOML loading, path resolution, default_map conversion
├── models/
│   ├── toc.py             # TOC, TOCEntry, TOCFeature models
│   ├── page_content.py    # PageContent model
│   └── rename.py          # RenameOp, RenameResult, ApplyRenamesResult models
├── processors/
│   ├── toc_generator.py   # TOCGenerator processor
│   ├── ocr_processor.py   # OCRProcessor for text extraction
│   ├── ocr_detection.py   # OCR detection strategies
│   └── rename_processor.py # RenameProcessor for file renaming
└── tests/                 # Unit tests
```

**CLI Commands:**
- `generate-toc` - Main command for TOC generation (with optional automatic OCR). Supports `--inplace` for modifying the input file directly, `--postprocess` for LLM-based TOC refinement, and `--ocr-output` to include the OCR text layer in the output PDF.
- `extract-text` - OCR-only command for text extraction from scanned PDFs. Supports `--inplace` for modifying the input file directly.
- `rename` - Intelligent file renaming using LLM inference. Supports reading paths from a file via `-f`/`--input-file` option for handling many files or long filenames. Use `-y`/`--yes` to skip the confirmation prompt.

**Processor Classes:**
- `TOCGenerator` - Extracts font/text features from PDF pages, sends to LLM for TOC inference, writes bookmarks back to PDF. Supports multiprocessing, intelligent batching for large documents, retry logic with exponential backoff, and optional postprocessing to refine the TOC using reference text from the document's first pages.
- `OCRProcessor` - Performs OCR on scanned PDFs using PyMuPDF's Tesseract integration. Supports multiprocessing for parallel page processing.
- `OCRDetectionStrategy` / `NoTextDetectionStrategy` - Strategy pattern for determining if a document needs OCR.
- `RenameProcessor` - Uses LLM to generate intelligent file rename suggestions based on user instructions. Supports batch renaming with confirmation preview.

**Model Classes:**
- `TOC` / `TOCEntry` - Pydantic models for structured LLM output with confidence scores. `TOC.sanitize_hierarchy()` clamps levels for PyMuPDF compatibility. `TOC.sort_by_page()` enforces page number monotonicity after postprocessing.
- `TOCFeature` - Compact representation of page features sent to the LLM
- `PageContent` - Data model for page representation
- `RenameOp` / `RenameResult` - Pydantic models for file rename operations with confidence scores and reasoning
- `ApplyRenamesResult` / `RenameError` - Dataclasses tracking successful and failed rename operations

**Key Integration Points:**
- PyMuPDF (`pymupdf`) for PDF reading/writing and OCR via Tesseract
- LangChain for LLM abstraction with `init_chat_model()` and `with_structured_output()` for typed responses
- LangSmith for observability/tracing via `@traceable` decorator on CLI commands
- Tenacity for retry logic with exponential backoff
- Rich for terminal progress indicators and table output
- Default model: `gpt-5.2` (configurable via `--model-identifier`)

**TOC Generation Strategy:**
The `TOCGenerator._extract_features()` method extracts font metadata (name, size) and text snippets from the first few blocks/lines of each page (Phase 1), then scans remaining blocks for heading candidates (Phase 2, with stricter font size threshold of 1.2x body size and a cap of 3 candidates per page, checking only the first span per block). Font subset prefixes (e.g. `ABCDEF+`) are stripped for deduplication. Features are serialized into a compact pipe-delimited format with a font table (`serialize_features_compact()`), which is ~50% more compact than Python `str()` — font names are mapped to short IDs (F0, F1, ...), spans are grouped by page (`P42:\nF0|14|Chapter Title|.12|B`), and redundant fields like `text_length` are omitted. This format is sent to the LLM which identifies chapter/section headings and returns structured `TOCEntry` objects with confidence scores. For large documents, features are batched with overlap for context continuity (default 100K tokens per batch). Token counting uses tiktoken (`o200k_base` encoding) for accurate tokenizer-based counts, eliminating the need for character-based ratio estimation. LLM calls use retry logic that only retries transient errors (rate limits, server errors); client errors like context overflow are raised immediately. When `--postprocess` is enabled, a second LLM pass refines the TOC by cross-referencing against any printed table of contents found in the document's first pages (up to 20 pages scanned, with pages 4+ filtered to keep only TOC-like lines: dot-leader entries, section numbers, and short title lines), removing duplicates, fixing typos, and adding missing entries. The features summary for the postprocessor prioritizes heading-like spans (80% budget) over body text, capped at 80 entries. The postprocessor prompt explicitly instructs the LLM to output PDF page numbers (not printed page numbers), with a conversion formula when front matter offset is detected. After postprocessing, a deterministic correction step restores original PDF page numbers for matched entries (using best-match fuzzy matching with a coverage ratio threshold to avoid false substring matches) and applies sanity-check offset correction for new entries. Front matter title detection uses a regex pattern (`_FRONT_MATTER_TITLE_PATTERN`) that recognizes titles like "Introduction" and "Preface to the Second Edition" as front matter, while correctly excluding subject-specific titles like "Introduction to Ito-Calculus". After correction, `TOC.sort_by_page()` enforces page number monotonicity, and finally `TOC.sanitize_hierarchy()` clamps hierarchy levels to satisfy PyMuPDF constraints (first entry must be level 1, no level jumps > 1).

**In-place Editing:**
Both `generate-toc` and `extract-text` support `--inplace` mode which modifies the input file directly. This uses a temporary file strategy (write to temp, then replace original) to work around PyMuPDF's limitation of not being able to save to the same file it has open. Original file permissions are preserved.

**OCR Integration:**
When `--ocr` is enabled (default), `generate-toc` automatically detects if a PDF needs OCR by checking for extractable text (using configurable `min_text_coverage` threshold, default 25%), performs OCR if needed, then proceeds with TOC generation on the text layer. By default the OCR text layer is discarded from the final output (used only for TOC inference); use `--ocr-output` to include it.

**Configuration System:**
The CLI supports TOML configuration files (`pdfalive.toml` or `.pdfalive.toml`) for setting default option values. The config module (`pdfalive/config/`) handles:
- `models.py` - Pydantic models for validating config structure with kebab-case alias support
- `loader.py` - File discovery (cwd > home > ~/.config/pdfalive/), TOML parsing, and conversion to Click's `default_map` format

The config is loaded via an eager callback on the `--config` option in `cli.py`. Global settings apply to all LLM-using commands, and command-specific settings override globals. CLI arguments always take precedence over config file values.

## Development Guidelines

- After making major changes to functionality, CLI options, or project architecture, always review and update all documentation to stay in sync: `README.md`, `docs/usage.md`, and `CLAUDE.md`. Docs should accurately reflect current defaults, option names, and feature descriptions.
- When adding or removing CLI configuration options, also update the example `pdfalive.toml` snippets in `README.md` and `docs/usage.md`, as well as the config models in `pdfalive/config/models.py`.
- always prefer placing imports at top of files rather than inline. Especially when writing unit-test. only do otherwise to avoid circular dependencies in rare cases. In those cases, mention explicitly why you are doing this in a comment on the relevant code line.
- When writing unit-tests, use variables and/or pytest fixtures (e.g. via conftest.py and `@pytest.fixture` decorator) for fixture values and objects, rather than repeating literal values in test setup and assertions. Prefer using pytest's `@pytest.mark.parametrize` decorator when you wish to test different values or combinations of values rather than creating repetitive standalone test cases.
- When making changes, always make sure formatting, linting, type checks, and tests work afterwards. We use ruff, mypy and pytest for these, and can run them via uv, e.g. `uv run ruff ...`, `uv run mypy`, etc.
- When finished making substantial changes to functionality and/or API (e.g. CLI usage) make sure to update documentation - README.md, CLAUDE.md and docs/ markdown files should all be kept up to date. Changing any CLI configuration options should also result in update the config/ submodule which lets us use TOML configuration files for defaults.
