# Architecture

## Overview

`axm-bib` follows a layered architecture with clear separation of concerns:

```mermaid
graph TD
    subgraph "User Interfaces"
        CLI["CLI (cyclopts)"]
        MCP["MCP Tools"]
    end

    subgraph "Orchestration"
        Pipeline["BibPipeline (façade)"]
        Search["search.py"]
        Venue["VenueEnricher"]
        Download["download.py"]
    end

    subgraph "Processing"
        Extract["extract_pdf (PyMuPDF)"]
        Storage["FileSystemStorage"]
    end

    subgraph "API Clients"
        S2["SemanticScholarClient"]
        CR["CrossRefClient"]
        UW["UnpaywallClient"]
        DBLP["DBLPClient"]
        OR["OpenReviewClient"]
        ARC["ArXivClient"]
        OAL["OpenAlexClient"]
        HAL["HALClient"]
    end

    subgraph "HTTP Layer"
        Retry["request_with_retry"]
    end

    subgraph "External"
        S2API["Semantic Scholar API"]
        CRAPI["CrossRef API"]
        UWAPI["Unpaywall API"]
        DBLPAPI["DBLP API"]
        ORAPI["OpenReview API"]
        ARCAPI["arXiv API"]
        OALAPI["OpenAlex API"]
        HALAPI["HAL API"]
        ArXiv["arXiv.org"]
        ACL["ACL Anthology"]
    end

    CLI --> Pipeline
    MCP --> Pipeline
    Pipeline --> Search
    Pipeline --> Venue
    Pipeline --> Download
    Venue --> S2
    Venue --> DBLP
    Pipeline --> CR
    Pipeline --> UW
    Pipeline --> Extract
    Pipeline --> Storage
    Download --> Retry
    S2 --> Retry
    UW --> Retry
    DBLP --> Retry
    OR --> Retry
    ARC --> Retry
    OAL --> Retry
    HAL --> Retry
    CR --> CRAPI
    Retry --> S2API
    Retry --> UWAPI
    Retry --> DBLPAPI
    Retry --> ORAPI
    Retry --> ARCAPI
    Retry --> OALAPI
    Retry --> HALAPI
    Download -.->|"DOI-based fallback"| ArXiv
    Download -.->|"DOI-based fallback"| ACL
```

## Layers

### 1. User Interfaces

- **CLI** (`cli.py`) — `cyclopts`-based commands with input validation and formatted output
- **MCP Tools** (`tools/`) — classes satisfying the `AXMTool` protocol (from `axm.tools.base`) auto-discovered via `axm.tools` entry points

Both interfaces delegate to the pipeline layer — they handle input/output formatting only.

### 2. Pipeline

`BibPipeline` (`pipeline.py`) is a **thin façade** that orchestrates calls across clients and delegates domain logic to three focused modules:

| Module | Responsibility |
|---|---|
| `search.py` | Deduplication, filtering, and relevance ranking of search results |
| `venue.py` | `VenueEnricher` — S2/DBLP venue lookup, BibTeX type promotion |
| `download.py` | URL safety (SSRF guard), PDF fetching via `request_with_retry`, DOI fallback URLs |

**Public API:**

- **`resolve()`** — Identifier-only router (DOI, arXiv, HAL, DBLP, OpenReview → BibEntry). Titles are rejected with a hint to use `search()`. For arXiv identifiers, entries are **enriched with venue metadata** via `VenueEnricher` (e.g. `@misc` → `@inproceedings` with `booktitle = {NeurIPS}`). When DBLP arXiv-ID search returns only CoRR, a **title-based DBLP search** is attempted as a final fallback
- **`search()`** — Fans out to **6 sources in parallel** (S2, CrossRef, DBLP, OpenReview, arXiv, OpenAlex), then delegates post-processing to `search.py`: `dedup_papers()` → `filter_papers()` → `rank_by_relevance()`. Returns a `SearchResult` with `sources_queried` / `sources_failed` diagnostics
- **`download_pdf()`** — Any-ID input: `_build_pdf_urls()` (S2 OA → arXiv → Unpaywall → DOI fallbacks from `download.py`) → `fetch_pdf_content()` from `download.py` → extract → `resolve()` for BibTeX
- **`graph()`** — Citation graph traversal: S2 primary, OpenAlex fallback for citations/references

`resolve()` delegates identifier detection to the `resolver` module (see below), then routes to the appropriate client.

`OpenReviewClient`, `ArXivClient`, `OpenAlexClient`, and `HALClient` are accessed both through `resolve()` for identifier-based lookups and standalone for specific data sources.

Validation (limit clamping, error handling) lives here.

#### Resolver (`resolver.py`)

The `detect_identifier()` function classifies a free-form string into one of six `IdentifierType` variants:

| Type | Pattern | Example |
|---|---|---|
| `DOI` | `10.xxxx/...` | `10.1145/363235.363259` |
| `ARXIV` | `arXiv:...` or arxiv URL | `arXiv:1706.03762` |
| `HAL` | `hal-NNNNNNNN` | `hal-01234567` |
| `DBLP` | DBLP venue pattern | `conf/nips/VaswaniSPUJGKP17` |
| `OPENREVIEW` | OpenReview URL or bare prefix | `https://openreview.net/forum?id=...` or `openreview:ID` |
| `TITLE` | Anything else | `"Attention Is All You Need"` → **rejected** |

`BibPipeline.resolve()` uses this to route each identifier to the correct client without manual type specification. `TITLE` inputs raise a `ResolutionError` with a hint to use `search()` instead.

### 3. Processing

- **`extract_pdf()`** — Converts PDF to Markdown + figures using PyMuPDF
- **`FileSystemStorage`** — Manages the paper folder layout (`{key}/{key}.pdf`, `paper.bib`)

### 4. API Clients

Each client wraps a single external API. All `httpx`-based clients use `request_with_retry` (see below) for resilient HTTP requests:

| Client | API | Transport |
|---|---|---|
| `SemanticScholarClient` | S2 Graph API | `request_with_retry` |
| `CrossRefClient` | CrossRef REST | `habanero` |
| `UnpaywallClient` | Unpaywall REST | `request_with_retry` |
| `DBLPClient` | DBLP REST | `request_with_retry` |
| `OpenReviewClient` | OpenReview v2 REST | `request_with_retry` |
| `ArXivClient` | arXiv REST + Atom XML | `request_with_retry` |
| `OpenAlexClient` | OpenAlex REST | `request_with_retry` |
| `HALClient` | HAL Solr REST | `request_with_retry` |

!!! note "CrossRef exception"
    `CrossRefClient` uses the `habanero` library which handles its own retry logic internally.

#### HTTP Resilience (`http.py`)

`request_with_retry()` is a shared module sitting between API clients and `httpx`. It provides:

- **Exponential backoff with jitter** — `base × 2^attempt + random(0, 0.5)`
- **`Retry-After` header support** — numeric values honoured, capped at 30 s
- **Retryable status codes** — `429`, `500`, `502`, `503`
- **Structured exceptions after exhaustion** — `RateLimitError` (429), `APIUnavailable` (502/503/504)
- **Network error handling** — `httpx.TransportError` triggers retries
- **Logging** — `WARNING` on each retry attempt

Default: 3 retries with 1 s base delay. Non-retryable errors (e.g. `404`) raise immediately. See [Exceptions](#9-exceptions) for the full hierarchy.

### 5. Models

`Paper`, `BibEntry`, `DownloadResult`, `ExtractionResult`, `FigureInfo` — Pydantic models shared across all layers.

```mermaid
classDiagram
    class SearchResult {
        +list~Paper~ papers
        +list~str~ sources_queried
        +list~str~ sources_failed
    }

    class Paper {
        +str title
        +list~str~ authors
        +int year
        +str doi
        +str arxiv_id
        +str dblp_key
        +str openreview_id
        +str openalex_id
        +str abstract
        +int citation_count
    }

    class BibEntry {
        +str key
        +str raw
        +str doi
    }

    class DownloadResult {
        +str|None doi
        +Path path
        +int size_bytes
        +Path paper_dir
        +Path bibtex_path
        +ExtractionResult extraction
    }

    class ExtractionResult {
        +Path markdown_path
        +int word_count
        +int char_count
        +int page_count
        +list~FigureInfo~ figures
    }

    class FigureInfo {
        +Path path
        +int page
        +int index
    }

    DownloadResult --> ExtractionResult : extraction
    ExtractionResult --> FigureInfo : figures
```

### 6. Storage

`StorageBackend` is an abstract base class (`ABC`) defining the interface for paper storage. `FileSystemStorage` is the default implementation.

**Default root:** `~/axm/papers/`

**Folder layout per paper:**

```
~/axm/papers/{key}/
├── {key}.pdf       # downloaded PDF
├── paper.bib       # BibTeX entry
├── content.md      # extracted Markdown
└── figures/
    ├── fig_001.png
    └── ...
```

The `{key}` is generated by the key generation module (see below).

### 7. Key Generation

`generate_key()` in `keys.py` creates deterministic BibTeX keys from paper metadata:

**Formula:** `{first_author_surname}{year}{first_significant_title_word}`

**Rules:**

1. Extract first author's surname (handles "Last, First" and "First Last" formats)
2. Strip diacritics (é → e, ü → u)
3. Append year (if known)
4. Append first significant word from title (skips articles: "a", "an", "the", etc.)
5. Lowercase everything

**Examples:**

| Authors | Year | Title | Key |
|---|---|---|---|
| Vaswani, Ashish et al. | 2017 | Attention is All you Need | `vaswani2017attention` |
| Hoare, C. A. R. | 1969 | An axiomatic basis for... | `hoare1969axiomatic` |

### 8. Hooks

`ParseBibAction` (`hooks.py`) is a plain class implementing the `HookAction` protocol from `axm.hooks.base`. It parses a `.bib` file and returns paper metadata for injection into protocol session context.

**Returned metadata keys:**

- `paper_title`, `paper_authors`, `paper_doi`, `paper_year`, `paper_bibtex_key`

Auto-discovered by `axm-engine`'s `HookRunner` via the `axm.hooks` entry point:

```toml
[project.entry-points."axm.hooks"]
parse_bib = "axm_bib.hooks:ParseBibAction"
```

**Usage in a protocol:** When a protocol task references a `.bib` file, `axm-engine` automatically runs `ParseBibAction.execute(path=...)` and merges the returned metadata into the session context. Downstream tasks can then reference `{{ paper_title }}`, `{{ paper_doi }}`, etc.

### 9. Exceptions

`exceptions.py` defines a structured hierarchy rooted at `AXMBibError`:

```
AXMBibError
├── ResolutionError
│   ├── IdentifierNotFound
│   └── AmbiguousIdentifier
├── ClientError
│   ├── RateLimitError (retry_after: float | None)
│   └── APIUnavailable
├── DownloadError
├── ExtractionError
└── MergeConflict
```

All exceptions are exported from `axm_bib` and can be caught broadly (`AXMBibError`) or precisely (`RateLimitError`). `RateLimitError` exposes the `retry_after` attribute parsed from the `Retry-After` header.

## Data Flow: PDF Pipeline

```mermaid
sequenceDiagram
    participant User
    participant CLI
    participant Pipeline
    participant Download as download.py
    participant S2
    participant ArXiv
    participant Unpaywall
    participant Storage
    participant Extract

    User->>CLI: axm-bib pdf REF
    CLI->>Pipeline: download_pdf(REF)
    Pipeline->>Pipeline: detect_identifier(REF)
    Pipeline->>Pipeline: _build_pdf_urls(REF, type, value)
    Pipeline->>S2: search_paper(REF) → open_access_pdf
    alt arXiv ID detected
        Pipeline->>ArXiv: pdf_url(id)
    end
    alt DOI available
        Pipeline->>Unpaywall: get_pdf_urls(DOI)
    end
    Pipeline->>Download: fetch_pdf_content(urls)
    Pipeline->>Pipeline: resolve(REF) → BibEntry
    Pipeline->>Storage: save(pdf_bytes, key)
    Storage-->>Pipeline: {key}/{key}.pdf
    Pipeline->>Storage: save_bibtex(key, raw)
    Storage-->>Pipeline: {key}/paper.bib
    Pipeline->>Extract: extract_pdf(pdf, dir)
    Extract-->>Pipeline: ExtractionResult
    Pipeline-->>CLI: DownloadResult
    CLI->>User: Formatted output
```

## Data Flow: Search

```mermaid
sequenceDiagram
    participant User
    participant CLI
    participant Pipeline
    participant SearchMod as search.py
    participant S2
    participant CrossRef
    participant DBLP
    participant OpenReview
    participant ArXivC as arXiv
    participant OpenAlex

    User->>CLI: axm-bib search "query" -n 10
    CLI->>Pipeline: search("query", limit=10)
    Pipeline->>Pipeline: Validate limit (1-100)
    par Fan-out to 6 sources
        Pipeline->>S2: search("query", limit=10)
        Pipeline->>CrossRef: search_by_title("query", limit=10)
        Pipeline->>DBLP: search("query", limit=10)
        Pipeline->>OpenReview: search("query", limit=10)
        Pipeline->>ArXivC: search("query", limit=10)
        Pipeline->>OpenAlex: search("query", limit=10)
    end
    Pipeline->>SearchMod: dedup_papers(all_papers)
    SearchMod-->>Pipeline: deduplicated
    Pipeline->>SearchMod: filter_papers(deduplicated, filters)
    SearchMod-->>Pipeline: filtered
    Pipeline->>SearchMod: rank_by_relevance(query, filtered)
    SearchMod-->>Pipeline: ranked
    Pipeline-->>CLI: SearchResult
    CLI->>User: Formatted output
```
