---
title: "feat: Condensr Book Summarizer Library"
type: feat
status: completed
date: 2026-03-01
origin: docs/brainstorms/2026-03-01-condensr-book-summarizer-brainstorm.md
---

# feat: Condensr Book Summarizer Library

## Overview

Build Condensr, an open-source Python library that takes a PDF book and produces a structured Markdown summary. It summarizes each chapter individually (max ~500 words per chapter) using Mistral AI. Exposes a Python API (`get_chapters()`, `summarize()`) and a CLI (`condensr book.pdf`).

See brainstorm: [docs/brainstorms/2026-03-01-condensr-book-summarizer-brainstorm.md](../brainstorms/2026-03-01-condensr-book-summarizer-brainstorm.md)

## Problem Statement / Motivation

Books are long. Readers want a structured, chapter-by-chapter summary to decide whether to read the full book, or to quickly reference key points. Existing tools are either manual, proprietary, or don't preserve chapter structure. Condensr automates this with a simple API and CLI.

## Proposed Solution

A layered Python package with three modules (see brainstorm):

- `condensr/parser.py` — PDF parsing + chapter detection (heuristic, no AI calls)
- `condensr/summarizer.py` — Mistral AI integration + prompt templates
- `condensr/cli.py` — CLI entry point via Click
- `condensr/__init__.py` — Public API: `get_chapters()` and `summarize()`

## Technical Considerations

### Gap Resolutions from SpecFlow Analysis

These gaps were identified during spec analysis. Resolutions are captured here as binding decisions for implementation.

**G1: `get_chapters()` does NOT call Mistral.**
The brainstorm says "no API calls made." The AI fallback for chapter detection is only used inside `summarize()`. `get_chapters()` relies solely on TOC extraction + heading heuristics. This keeps it free, fast, and usable without an API key.

**G2: `get_chapters()` returns `[]` when no chapters are found.**
`summarize()` handles the empty-list case by treating the entire book as a single unit and summarizing it in one page.

**G3: Chapter text boundaries.**
When using TOC, page numbers define boundaries (chapter N starts at its page, ends at chapter N+1's page minus one). When using heading heuristics, text between consecutive detected headings constitutes a chapter. Text before the first heading is skipped. Text after the last heading is included in the last chapter.

**G4: Context window overflow.**
`mistral-small-latest` has a 128k token context window — most chapters (20-50 pages, ~10-25k tokens) fit comfortably. For v1: if a chapter's text exceeds 100k characters (~25k tokens), truncate with a warning to stderr. No recursive summarization for v1 (see Out of Scope).

**G5: Prompt template — specified below in Phase 3.**

**G6: Book title.**
Use `doc.metadata.get("title")`. Fall back to filename without extension (e.g., `book.pdf` → `Book`).

**G7: Front/back matter.**
No special handling. If detected as a chapter by TOC or heuristics, it gets summarized. If not detected, it's silently skipped.

**G8: Output file already exists → overwrite silently** (standard Unix behavior).

**G9: Partial output on CLI failure → leave partial file in place.** User can see what completed.

**G10: `on_chapter` fires before `yield`.** Callback first, then yield.

**G11: `on_chapter` exception → propagates to caller, halts generator.**

**G12: Encrypted PDFs → raise clear error:** `"PDF is password-protected. Condensr does not support encrypted PDFs."`

**G13: Scanned/image PDFs → raise clear error** if total extracted text is < 100 characters: `"No text content found. The PDF may be scanned/image-based."`

**G14: Invalid path → let PyMuPDF's FileNotFoundError/ValueError propagate naturally.** No custom wrapping needed.

**G15: Book structure with Parts > Chapters → flatten.** `get_toc()` returns levels. For v1, treat all level-1 and level-2 entries as chapters. Don't nest.

**G16: Resource cleanup.** Each parser function (`detect_chapters`, `extract_chapter_text`, etc.) opens and closes the PDF internally. No generator-level cleanup needed. Slightly less efficient (reopens per chapter) but avoids the partial-consumption cleanup problem entirely.

**G17: Data privacy notice.** Add a note in README that book content is sent to Mistral's API servers.

### Architecture

```
User Code / CLI
      │
      ▼
condensr/__init__.py     ← Public API: get_chapters(), summarize()
      │
      ├──► parser.py     ← PDF → list of Chapter(title, pages) + text extraction
      │
      └──► summarizer.py ← Chapter text → Mistral API → summary markdown
```

### Internal Data Model

```python
# Internal only — not part of public API
@dataclass
class Chapter:
    title: str
    start_page: int  # 0-indexed
    end_page: int     # 0-indexed, inclusive
```

`parser.py` produces `list[Chapter]`. `get_chapters()` returns `[c.title for c in chapters]`. `summarize()` uses the full `Chapter` objects to extract text by page range.

---

## Implementation Phases

### Phase 1: Project Scaffolding

Set up the Python package infrastructure.

**Files to create:**

#### `pyproject.toml`

```toml
[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[project]
name = "condensr"
version = "0.1.0"
description = "Summarize books chapter by chapter using AI"
readme = "README.md"
license = "AGPL-3.0-or-later"
requires-python = ">=3.10"
dependencies = [
    "pymupdf>=1.24",
    "mistralai>=1.0,<2.0",
    "click>=8.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-mock",
]

[project.scripts]
condensr = "condensr.cli:main"
```

#### `.gitignore`

Standard Python gitignore: `__pycache__/`, `*.egg-info/`, `dist/`, `.venv/`, `*.pyc`.

#### `LICENSE`

AGPL v3 full text (see brainstorm: license decision driven by PyMuPDF's AGPL license).

#### Package directory structure

```
condensr/
    __init__.py    # empty initially
    parser.py      # empty initially
    summarizer.py  # empty initially
    cli.py         # empty initially
tests/
    __init__.py
    test_parser.py
    test_summarizer.py
```

- [x] Create `pyproject.toml` with metadata, dependencies, and CLI entry point
- [x] Create `.gitignore` for Python
- [x] Add `LICENSE` (AGPL v3)
- [x] Scaffold empty package modules and test files

---

### Phase 2: PDF Parser Module — `condensr/parser.py`

The parser handles PDF opening, chapter detection, and text extraction. No Mistral calls.

**Public functions:**

```python
def detect_chapters(pdf_path: str) -> list[Chapter]:
    """Detect chapter boundaries in a PDF. Returns list of Chapter dataclasses.

    Detection strategy (in order):
    1. TOC extraction via doc.get_toc()
    2. Heading heuristics (bold text, large font, "Chapter N" patterns)
    3. No chapters found → returns empty list
    """

def extract_chapter_text(pdf_path: str, chapter: Chapter) -> str:
    """Extract plain text for a specific chapter by page range."""

def extract_full_text(pdf_path: str) -> str:
    """Extract all text from the PDF (used when no chapters detected)."""

def get_book_title(pdf_path: str) -> str:
    """Return book title from PDF metadata, or cleaned filename as fallback."""
```

**Internal helpers:**

```python
def _detect_from_toc(doc) -> list[Chapter]:
    """Use doc.get_toc() to build chapter list with page boundaries."""
    # get_toc() returns [[level, title, page_num], ...]
    # Filter to level 1 (and optionally level 2) entries
    # Convert 1-based page numbers to 0-based
    # Each chapter runs from its start page to the next chapter's start page - 1

def _detect_from_headings(doc) -> list[Chapter]:
    """Scan pages for heading-like text using font metadata."""
    # For each page, call page.get_text("dict")
    # Look at spans with: font size > median size * 1.3, or bold flag (flags & 16)
    # Also match regex patterns: r"^(Chapter|Part|Section)\s+\d+" (case-insensitive)
    # Build Chapter list from detected headings
```

**Key implementation details:**
- Import: `import pymupdf` (modern import, >= 1.24)
- Font flags bitmask: bold = `flags & 16`, italic = `flags & 2`
- `doc.get_toc(simple=True)` returns `[level, title, page_num]` with 1-based pages
- `page.get_text("dict")` returns blocks → lines → spans with `size`, `flags`, `font` fields
- `doc.pages(start, stop)` for page-range text extraction (0-indexed, stop exclusive)

**Validation on open:**
- Encrypted PDF → check `doc.is_encrypted`, raise `ValueError("PDF is password-protected...")`
- No text → if total extracted chars < 100, raise `ValueError("No text content found...")`

- [x] Implement `Chapter` dataclass
- [x] Implement `_detect_from_toc()` using `doc.get_toc()`
- [x] Implement `_detect_from_headings()` using font metadata
- [x] Implement `detect_chapters()` orchestrating both strategies
- [x] Implement `extract_chapter_text()` and `extract_full_text()`
- [x] Implement `get_book_title()` with metadata fallback
- [x] Add validation for encrypted and scanned PDFs

---

### Phase 3: Summarizer Module — `condensr/summarizer.py`

Handles Mistral API calls and prompt construction.

**Public function:**

```python
def summarize_text(
    text: str,
    chapter_title: str,
    model: str = "mistral-small-latest",
    api_key: str | None = None,
) -> str:
    """Summarize a chapter's text using Mistral. Returns markdown summary string."""
```

**Mistral SDK usage (v1.x):**

```python
from mistralai import Mistral

def summarize_text(text, chapter_title, model="mistral-small-latest", api_key=None):
    client = Mistral(api_key=api_key)  # falls back to MISTRAL_API_KEY env var

    response = client.chat.complete(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": USER_PROMPT.format(title=chapter_title, text=text)},
        ],
        temperature=0.3,
    )
    return response.choices[0].message.content
```

**Prompt template:**

```python
SYSTEM_PROMPT = """You are a book summarizer. You produce clear, structured summaries in Markdown format.
Rules:
- Summarize the key ideas, arguments, and conclusions
- Keep the summary under 500 words
- Use bullet points for lists of key points
- Preserve important names, dates, and figures
- Do not add information not present in the source text
- Write in the same language as the source text"""

USER_PROMPT = """Summarize the following chapter.

Chapter title: {title}

Chapter text:
{text}"""
```

**Text truncation:** If `text` exceeds 100,000 characters, truncate to 100,000 and log a warning. This is a safety valve — most chapters are well under this limit given mistral-small's 128k token context window.

- [x] Define `SYSTEM_PROMPT` and `USER_PROMPT` constants
- [x] Implement `summarize_text()` using `Mistral` client (v1.x SDK)
- [x] Add text truncation with warning for oversized chapters

---

### Phase 4: Public API — `condensr/__init__.py`

Wire parser and summarizer together into the two public functions.

```python
"""Condensr — Summarize books chapter by chapter using AI."""

from condensr.parser import detect_chapters, extract_chapter_text, extract_full_text, get_book_title
from condensr.summarizer import summarize_text


def get_chapters(pdf_path: str) -> list[str]:
    """Detect and return chapter titles from a PDF.

    Uses heuristic detection only (no API calls).
    Returns an empty list if no chapters are found.

    Example:
        >>> condensr.get_chapters("book.pdf")
        ["Introduction", "Chapter 1: Origins", "Chapter 2: Growth"]
    """
    chapters = detect_chapters(pdf_path)
    return [c.title for c in chapters]


def summarize(
    pdf_path: str,
    *,
    model: str = "mistral-small-latest",
    api_key: str | None = None,
    on_chapter=None,
):
    """Summarize a PDF book chapter by chapter.

    Yields (chapter_title, summary_markdown) tuples as each chapter completes.
    If no chapters are detected, summarizes the entire book as one unit.

    Args:
        pdf_path: Path to the PDF file.
        model: Mistral model name (default: "mistral-small-latest").
        api_key: Mistral API key. Falls back to MISTRAL_API_KEY env var.
        on_chapter: Optional callback(title, summary) fired before each yield.

    Example:
        >>> for title, summary in condensr.summarize("book.pdf"):
        ...     print(f"## {title}\\n{summary}")
    """
    chapters = detect_chapters(pdf_path)

    if not chapters:
        # No chapters detected — summarize entire book
        text = extract_full_text(pdf_path)
        title = get_book_title(pdf_path)
        summary = summarize_text(text, title, model=model, api_key=api_key)
        if on_chapter:
            on_chapter(title, summary)
        yield (title, summary)
        return

    for chapter in chapters:
        text = extract_chapter_text(pdf_path, chapter)
        summary = summarize_text(text, chapter.title, model=model, api_key=api_key)
        if on_chapter:
            on_chapter(chapter.title, summary)
        yield (chapter.title, summary)
```

**Note:** Resource cleanup — `detect_chapters`, `extract_chapter_text`, and `extract_full_text` each open/close the document internally. This is slightly less efficient (reopens the file per chapter) but simpler and avoids the generator cleanup problem entirely. For v1 this is acceptable; optimization can come later if profiling shows it matters.

- [x] Implement `get_chapters()` wrapping `detect_chapters()`
- [x] Implement `summarize()` generator with callback support
- [x] Add docstrings with usage examples

---

### Phase 5: CLI — `condensr/cli.py`

```python
import click
from condensr import summarize, get_book_title


@click.command()
@click.argument("pdf_path", type=click.Path(exists=True))
@click.option("--output", "-o", type=click.Path(), default=None,
              help="Output file path. Default: <book>-summary.md")
@click.option("--model", "-m", default="mistral-small-latest",
              help="Mistral model name.")
def main(pdf_path, output, model):
    """Summarize a PDF book chapter by chapter."""
    if output is None:
        # book.pdf → book-summary.md
        base = pdf_path.rsplit(".", 1)[0]
        output = f"{base}-summary.md"

    title = get_book_title(pdf_path)
    chapter_titles = get_chapters(pdf_path)
    total = len(chapter_titles) if chapter_titles else 1

    with open(output, "w") as f:
        f.write(f"# {title}\n\n")
        for i, (chapter_title, summary) in enumerate(summarize(pdf_path, model=model), 1):
            click.echo(f"Summarizing chapter {i}/{total}: {chapter_title}...", err=True)
            f.write(f"## {chapter_title}\n\n{summary}\n\n")

    click.echo(f"Summary written to {output}", err=True)
```

- [x] Implement `main()` Click command with `pdf_path` argument, `--output` and `--model` options
- [x] Progress output to stderr via `click.echo(..., err=True)`
- [x] Auto-name output file from input filename

---

### Phase 6: Tests

#### `tests/test_parser.py`

Test chapter detection logic with fixture PDFs. No Mistral calls needed.

```python
# Test cases:
# - PDF with TOC → detect_chapters returns correct titles and page ranges
# - PDF with bold headings but no TOC → heading heuristics find chapters
# - PDF with no chapters → returns empty list
# - PDF with "Chapter N" text patterns → detected correctly
# - Encrypted PDF → raises ValueError
# - get_book_title() → returns metadata title, or filename fallback
```

**Test fixture strategy:** Use `pymupdf` itself to programmatically create small test PDFs in a `conftest.py` fixture. No need to commit binary PDF files to the repo.

```python
# tests/conftest.py
import pymupdf
import pytest

@pytest.fixture
def pdf_with_toc(tmp_path):
    """Create a small PDF with a table of contents."""
    doc = pymupdf.open()
    # Add pages with text, set TOC
    # ...
    path = tmp_path / "book_with_toc.pdf"
    doc.save(str(path))
    doc.close()
    return str(path)
```

#### `tests/test_summarizer.py`

Test Mistral integration with mocked API calls.

```python
# Test cases:
# - summarize_text() calls Mistral with correct model and messages
# - summarize_text() returns the response content
# - Text exceeding 100k chars is truncated
# - api_key parameter is passed to Mistral client
```

- [x] Create `tests/conftest.py` with PDF fixture generators
- [x] Write `test_parser.py` — TOC detection, heading heuristics, empty list, encrypted PDF
- [x] Write `test_summarizer.py` — mocked Mistral calls, truncation, API key handling

---

### Phase 7: README and Finishing

#### `README.md`

Sections:
1. **One-line description** — what Condensr does
2. **Installation** — `pip install condensr`
3. **Quick Start** — 3-line Python example + CLI example
4. **API Reference** — `get_chapters()` and `summarize()` with parameters
5. **CLI Reference** — `condensr [OPTIONS] PDF_PATH`
6. **Configuration** — `MISTRAL_API_KEY` env var
7. **Privacy Notice** — book content is sent to Mistral's API servers
8. **License** — AGPL v3

- [x] Write README.md with installation, quickstart, API/CLI reference, and privacy notice

---

## Acceptance Criteria

- [x] `condensr.get_chapters("book.pdf")` returns chapter titles as `list[str]`
- [x] `condensr.summarize("book.pdf")` yields `(title, summary_md)` tuples
- [x] Callback `on_chapter=` fires before each yield
- [x] CLI `condensr book.pdf` writes `book-summary.md` with progress to stderr
- [x] Chapter detection works via TOC extraction and heading heuristics
- [x] No chapters → whole-book summary in one page
- [x] Encrypted PDFs → clear error message
- [x] Image-only PDFs → clear error message
- [x] `pip install -e .` succeeds and `condensr` CLI is available
- [x] `pytest` passes with all tests green

## Dependencies & Risks

| Risk | Impact | Mitigation |
|---|---|---|
| PyMuPDF TOC extraction unreliable for some PDFs | Poor chapter detection | Heading heuristics as fallback |
| Mistral API changes | SDK calls break | Pin `mistralai>=1.0,<2.0` |
| AGPL license limits corporate adoption | Smaller user base | Documented alternative (pdfplumber+pypdf) |
| Prompt quality affects summary usefulness | Poor summaries | Iterate on prompt template; easy to update |

## Sources & References

### Origin

- **Brainstorm:** [docs/brainstorms/2026-03-01-condensr-book-summarizer-brainstorm.md](../brainstorms/2026-03-01-condensr-book-summarizer-brainstorm.md) — Key decisions carried forward: AGPL license, layered architecture, heuristic+AI chapter detection, generator+callback API, Mistral-only provider, Click CLI.

### Library Documentation

- PyMuPDF: `import pymupdf`, `doc.get_toc()`, `page.get_text("dict")` for font metadata, `doc.pages(start, stop)` for ranges
- Mistral SDK v1.x: `from mistralai import Mistral`, `client.chat.complete()`, auto-reads `MISTRAL_API_KEY` env var
- Click 8.x: `@click.command()`, `click.echo(..., err=True)` for stderr
