# PaperMind v1.0 Overnight Fix Plan

> 22 tasks across 6 batches. All blockers + high-impact improvements.
> Estimated: 2-3 hours of agent work. No user input needed.
> Run with: `claude -p "Read ~/Development/papermind/docs/plans/v1.0-overnight.md and execute all batches sequentially. Work from ~/Development/papermind. Use uv run for all commands. Commit after each batch. Run full test suite (uv run pytest tests/ -q) after each batch — do not proceed if tests fail."`

---

## Batch A: Critical Blockers (3 tasks)

### A1. Fix transformers dependency for [ocr] extra
**File**: `pyproject.toml`
**Problem**: `transformers>=5.0` isn't on PyPI (pre-release). `pip install papermind[ocr]` installs 4.x which breaks.
**Fix**: Change to `transformers>=4.48` (works with `trust_remote_code=True`) and add a runtime version check in `glm_ocr.py` that gives a clear error if the installed version can't load GLM-OCR.
```python
# In glm_ocr.py _ensure_model(), after imports:
import transformers
if not hasattr(transformers, "GlmOcrConfig"):
    # Model needs transformers dev branch
    try:
        processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
    except Exception as exc:
        raise ImportError(
            f"GLM-OCR model requires transformers >= 5.0 (dev branch). "
            f"Installed: {transformers.__version__}. "
            f"Install with: pip install 'transformers @ git+https://github.com/huggingface/transformers.git'"
        ) from exc
```

### A2. Fix qmd_search cwd
**File**: `src/papermind/query/qmd.py`
**Problem**: `subprocess.run(cmd, ...)` has no `cwd`, searches wrong collection.
**Fix**: Add `cwd=str(kb_path)` to the subprocess call. One line.

### A3. Guard duplicate codebase ingestion
**File**: `src/papermind/cli/ingest.py`
**Problem**: Re-ingesting same codebase creates duplicate catalog entries.
**Fix**: Before `catalog.add(entry)`, check if entry ID exists and remove old one first (same pattern as package ingestion).

**Commit message**: `fix: 3 critical blockers — transformers dep, qmd cwd, codebase dedup`

---

## Batch B: Download Pipeline Alignment (3 tasks)

### B1. Extract _build_providers to shared location
**Files**: `src/papermind/discovery/providers.py` (new), `src/papermind/cli/discover.py`, `src/papermind/cli/main.py`
**Problem**: `_build_providers` lives in `cli/discover.py`, duplicated logic in `download.py`.
**Fix**: Move to `discovery/providers.py`, import from both CLI modules.

### B2. Use _build_providers in download command
**File**: `src/papermind/cli/download.py`
**Problem**: `_run_fresh_search` hardcodes SS+Exa, missing OpenAlex.
**Fix**: Import and use `build_providers()` from the shared location.

### B3. Add Unpaywall fallback to download command
**File**: `src/papermind/cli/download.py`
**Problem**: `download` command doesn't try Unpaywall when PDF URL missing.
**Fix**: Same pattern as `fetch` — if no pdf_url but has DOI, call `resolve_pdf_url()`.

**Commit message**: `fix: align download command with fetch — OpenAlex + Unpaywall`

---

## Batch C: Code Quality (6 tasks)

### C1. Extract _resolve_kb helper
**Files**: `src/papermind/cli/utils.py` (new), `cli/ingest.py`, `cli/catalog.py`, `cli/download.py`
**Problem**: `_resolve_kb` duplicated 3 times.
**Fix**: Move to `cli/utils.py`, import everywhere.

### C2. Remove unused image_dir param from convert_pdf_glm
**File**: `src/papermind/ingestion/glm_ocr.py`
**Problem**: `convert_pdf_glm(image_dir=...)` is never called with image_dir from paper.py (images extracted separately).
**Fix**: Remove `image_dir` param and the image extraction code at the end of `convert_pdf_glm`. Keep `extract_images` as standalone.

### C3. Fix PaperMindConfig import in discover.py
**File**: `src/papermind/cli/discover.py`
**Problem**: `PaperMindConfig` used as type annotation but not imported (suppressed with `# type: ignore`).
**Fix**: Add the import, remove the type: ignore comment.

### C4. Use Path.is_relative_to() for path traversal
**Files**: `src/papermind/cli/catalog.py`, `src/papermind/mcp_server.py`
**Problem**: `str(path).startswith(str(base))` has edge cases.
**Fix**: Replace with `path.is_relative_to(base)` (Python 3.9+, we require 3.11).

### C5. Config validation
**File**: `src/papermind/config.py`
**Problem**: Unknown keys silently ignored, `ocr_dpi` not validated.
**Fix**: Add `_validate()` method — warn on unknown keys, check `ocr_dpi` in [72, 600] range.

### C6. Consistent error handling in CLI
**Files**: `src/papermind/cli/ingest.py`, `src/papermind/cli/main.py`
**Problem**: Some commands show tracebacks on unexpected errors, others catch cleanly.
**Fix**: Add top-level `except Exception` in `fetch_command` and `ingest_paper_cmd` that prints `[red]Error:[/red] {exc}` and exits 1.

**Commit message**: `refactor: code quality — shared helpers, path safety, config validation`

---

## Batch D: Missing Features (5 tasks)

### D1. Abstract fetching from OpenAlex
**File**: `src/papermind/discovery/openalex.py`
**Problem**: OpenAlex search doesn't return abstracts (`abstract=""`).
**Fix**: Add `abstract_inverted_index` to the `select` fields, reconstruct abstract text from the inverted index (OpenAlex stores abstracts as `{word: [positions]}` dicts).

### D2. fetch --dry-run flag
**File**: `src/papermind/cli/main.py`
**Problem**: No way to preview what `fetch` would do without downloading/ingesting.
**Fix**: Add `--dry-run` flag that runs discover only and prints results table (reuse discover's table renderer).

### D3. catalog show --json flag
**File**: `src/papermind/cli/catalog.py`
**Problem**: No machine-readable catalog output.
**Fix**: Add `--json` flag to `show` that prints `catalog.json` contents.

### D4. BibTeX export
**File**: `src/papermind/cli/catalog.py` (new `export` command)
**Problem**: No citation export for reference managers.
**Fix**: New `papermind --kb <path> export bibtex` command that reads catalog entries with DOIs and outputs `.bib` format.

### D5. Paper dedup by title similarity
**File**: `src/papermind/ingestion/paper.py`
**Problem**: Same paper with preprint DOI vs journal DOI creates two entries.
**Fix**: Before ingestion, check if any existing entry has >90% title similarity (using `difflib.SequenceMatcher`). If so, warn and skip (same pattern as DOI dedup).

**Commit message**: `feat: abstracts, --dry-run, --json catalog, BibTeX export, title dedup`

---

## Batch E: Tests for New Features (3 tasks)

### E1. Tests for Batch A fixes
**File**: `tests/test_e2e.py` (extend)
- Test duplicate codebase re-ingestion produces single catalog entry
- Test qmd_search with cwd (mock subprocess, verify cwd kwarg)
- Test GLM-OCR gives clear error on old transformers

### E2. Tests for Batch C fixes
**File**: `tests/test_config.py` (extend)
- Test config warns on unknown keys
- Test ocr_dpi validation (too low, too high)
- Test is_relative_to path traversal edge case

### E3. Tests for Batch D features
**Files**: `tests/test_cli_catalog.py` (extend), new `tests/test_export.py`
- Test `catalog show --json` output is valid JSON
- Test `fetch --dry-run` doesn't write files
- Test BibTeX export format
- Test title similarity dedup

**Commit message**: `test: coverage for v1.0 fixes and features`

---

## Batch F: Documentation + Tag (2 tasks)

### F1. Update README
**File**: `README.md`
- Add `fetch --dry-run` to examples
- Add `catalog show --json` to CLI reference
- Add `export bibtex` to CLI reference
- Fix transformers install instructions
- Add "Paper Discovery" section noting OpenAlex returns abstracts

### F2. Update next-steps.md and tag
**File**: `docs/next-steps.md`
- Mark completed items
- Add remaining v1.1 items
- `git tag v1.0.0`

**Commit message**: `docs: README + next-steps update, tag v1.0.0`

---

## Execution Notes

- Work from `~/Development/papermind`
- Use `uv run` for all Python commands
- Run `uv run pytest tests/ -q --tb=short` after EVERY batch
- If tests fail, fix before proceeding — never skip
- Commit after each batch with the specified message
- Do NOT push to remote
- Do NOT modify `demo_kb/`
- Total expected: ~40 new/modified files, ~15 new tests, 6 commits
