# Condensr — Book Summarizer Library Brainstorm

**Date:** 2026-03-01
**Status:** Draft

---

## What We're Building

An open-source Python library called **Condensr** that takes a PDF book as input and produces a structured Markdown summary. It summarizes each chapter individually (max one page per chapter), using Mistral AI for the actual summarization. If the book has no detectable chapters, it summarizes the whole book in one page.

The library exposes both a **Python API** (generator + callback) and a **CLI command** (`condensr book.pdf`).

Two public functions:
- `get_chapters(pdf_path)` — returns detected chapter titles as a list of strings, no API calls made
- `summarize(pdf_path, ...)` — generator that yields `(title, summary_md)` per chapter

---

## Why This Approach

### Architecture: Layered package (`parser` / `summarizer` / `cli`)

Three focused modules with clear responsibilities:

- `condensr/parser.py` — PDF ingestion, chapter boundary detection (heuristic + AI fallback)
- `condensr/summarizer.py` — Mistral AI integration, prompt construction, per-chapter summarization
- `condensr/cli.py` — CLI entry point (`condensr` command via Click)
- `condensr/__init__.py` — Clean public API surface: `get_chapters()` and `summarize()`

**Why:** Standard idiomatic Python OSS library structure. Each layer is independently testable. Contributors can understand the codebase by reading one file at a time.

---

## Key Decisions

### 1. PDF Parsing — PyMuPDF (fitz)
- Native TOC/outline extraction (`doc.get_toc()`) — perfect for heuristic chapter detection
- Access to text with font metadata (size, flags) for fallback heading detection
- Fast, well-maintained, widely used for PDF text extraction

### 2. Chapter Detection Strategy — Heuristic first, AI fallback
1. **Try TOC extraction** via `doc.get_toc()` — if the PDF has a table of contents, use it directly
2. **Try heading heuristics** — scan for lines that are bold, larger font, or match `Chapter N` / `Part N` patterns
3. **AI fallback** — send the first ~2000 characters of each candidate section to Mistral and ask it to confirm whether it's a chapter boundary
4. **No chapters found** — treat the entire book as a single document, summarize in one page

### 3. AI Provider — Mistral only (for now)
No provider abstraction. Use the official `mistralai` Python SDK directly. Simple is better; can refactor later if multi-provider support is needed.

### 4. Streaming / Progressive Output — Generator + Callback (both)
`summarize()` is **always a generator** — it yields `(chapter_title, summary_md)` tuples as each chapter completes. If `on_chapter=` is provided, it fires the callback on each chapter *and* still yields. Callers can choose to consume the generator, use the callback, or both.

```python
# Generator only
for chapter_title, summary_md in condensr.summarize("book.pdf"):
    print(f"## {chapter_title}\n{summary_md}")

# Callback only (exhaust the generator)
def on_chapter(title, summary):
    save_to_db(title, summary)

for _ in condensr.summarize("book.pdf", on_chapter=on_chapter):
    pass

# Both simultaneously
for chapter_title, summary_md in condensr.summarize("book.pdf", on_chapter=on_chapter):
    display_live(chapter_title, summary_md)
```

The CLI exhausts the generator and streams chapter output progressively to stdout and the output file. Before each chapter, it prints a status line to stderr:

```
Summarizing chapter 3/12: Introduction...
```

No extra dependencies — plain `click.echo(..., err=True)`.

### 5. Output Format
A single Markdown file structured as:
```markdown
# Book Title

## Chapter 1: Introduction
<summary — max one page (~500 words)>

## Chapter 2: ...
```

---

## Package Structure

```
condensr/
├── __init__.py         # Public API: summarize()
├── parser.py           # PDF reading + chapter detection
├── summarizer.py       # Mistral calls + prompt templates
└── cli.py              # CLI entry point

pyproject.toml          # Build config + dependencies
README.md
tests/
  test_parser.py
  test_summarizer.py
```

---

## Dependencies

| Dependency | Purpose |
|---|---|
| `pymupdf` | PDF parsing, TOC extraction, text/font access |
| `mistralai` | Official Mistral AI Python SDK |
| `click` | CLI framework |

---

## Resolved Questions

- **Mistral model** — Default to `mistral-small`. Users can override via a `model=` parameter.
- **API key config** — Support both: read `MISTRAL_API_KEY` env var automatically, and accept `api_key=` parameter in `summarize()` (explicit wins over env var).
- **CLI output** — Auto-name: `book.pdf` → `book-summary.md` in the same directory. Add an optional `--output` flag to override.
- **Rate limiting** — Do not handle internally. Let the `mistralai` SDK raise its exception. Caller is responsible for retry logic.

---

## Open Source Conventions

### License — AGPL v3
PyMuPDF is AGPL-licensed, which propagates to Condensr. Condensr is released under **AGPL v3**. Users who distribute modified versions must share source. Fine for open-source use; corporate users who need a permissive license can swap PyMuPDF for `pdfplumber`+`pypdf` themselves.

Files to include: `LICENSE`, `NOTICE` (if needed).

### Testing — pytest
Standard Python testing framework. Tests live in `tests/` alongside the package.

- `test_parser.py` — chapter detection logic (no Mistral calls)
- `test_summarizer.py` — Mistral integration (mocked in unit tests)

No coverage target mandated for v1, but parser tests should be thorough since that logic is pure Python.

### Documentation — README + Docstrings
Minimum viable docs for launch:

- `README.md` — installation, quickstart example, CLI usage, API reference overview
- Docstrings on `get_chapters()` and `summarize()` with usage examples

No separate docs site for v1.

---

## Out of Scope (for v1)

- Multi-provider support (OpenAI, Anthropic, etc.)
- EPUB / DOCX input formats
- Web UI or hosted service
- Translation / multilingual output
- Fine-grained summary length control per chapter
