# Concepts and Architecture

RetroCast provides a standardized framework for evaluating retrosynthesis models. It addresses the fragmentation of output formats in the field by decoupling the model's internal representation from the evaluation logic.

## The Core Philosophy: Adapters as an Air Gap

Retrosynthesis models produce diverse output formats. Some output bipartite graphs (AiZynthFinder), others output precursor maps (Retro*), and some output recursive dictionaries (DirectMultiStep). Comparing these directly requires writing bespoke evaluation code for every model, which leads to bugs and inconsistent metrics.

RetroCast introduces an **adapter layer** between the model and the evaluation pipeline.

1.  **The Model** runs independently and saves its output in its native format.
2.  **The Adapter** reads this specific format and transforms it into the canonical RetroCast schema.
3.  **The Pipeline** performs scoring and analysis on the canonical objects, unaware of the original model format.

This architecture ensures that metrics like solvability and route length are calculated identically for every model.

## The Canonical Data Model

RetroCast defines a strict, recursive object model in `retrocast.models.chem`. Structurally, this model is a **directed acyclic bipartite graph** consisting of alternating molecule nodes and reaction nodes.

### A Resolved AND/OR Tree
Many retrosynthesis frameworks (e.g., Syntheseus, AiZynthFinder) utilize an AND/OR graph to represent the entire search space. In these graphs, a Molecule node (OR) may have multiple child Reaction nodes (AND), representing competing choices.

The RetroCast `Route` object represents a **resolved instance** of this graph: a single, specific pathway, with a Molecule node having at most one child Reaction node.

### Schema Definition
The schema enforces the minimal information required for rigorous evaluation while allowing for extensibility via metadata dictionaries.

```python
class Molecule(BaseModel):
    """Represents a chemical node (OR node)."""
    # Core Identity
    smiles: SmilesStr
    inchikey: InchiKeyStr  # Primary ID for hashing/equality

    # Tree Structure: 0 or 1 reaction step
    synthesis_step: ReactionStep | None 

    # Extensibility
    metadata: dict[str, Any]

    # Computed Properties
    @property
    def is_leaf(self) -> bool: ...

class ReactionStep(BaseModel):
    """Represents a reaction node (AND node)."""
    # Tree Structure: N reactant children
    reactants: list[Molecule]

    # Chemical Details (Optional)
    mapped_smiles: str | None
    template: str | None
    reagents: list[str] | None
    solvents: list[str] | None

    # Extensibility
    metadata: dict[str, Any]

    # Computed Properties
    @property
    def is_convergent(self) -> bool: ...

class Route(BaseModel):
    """The root container for a single prediction."""
    target: Molecule
    rank: int # Populated by Adapter (Model's preference order)

    # Provenance
    metadata: dict[str, Any]
    retrocast_version: str

    # Computed Properties
    @property
    def depth(self) -> int: ...
    @property
    def leaves(self) -> set[Molecule]: ...
    @property
    def content_hash(self) -> str: ...
```

### Rationale: Why Bipartite?
We deem the bipartite structure, explicitly separating `Molecule` and `ReactionStep`, natural for representing multistep routes and it allows precise attribution of data:
*   **Molecule properties** (e.g., "is purchasable", molecular weight) belong to the `Molecule` node.
*   **Reaction properties** (e.g., template scores, probability, patent IDs) belong to the `ReactionStep` node.

### Interchange Format
While we welcome adoption of the format, model developers are not required to utilize this schema for their internal logic. RetroCast treats this schema as an **interchange format**. The role of the adapter is to cast the model's native output—however complex or verbose—into this minimal structure. Any data not strictly required for topological evaluation (such as internal attention weights or raw search trees) can either be discarded or preserved in the `metadata` dictionary for downstream analysis.

## Data Organization and Lifecycle

RetroCast enforces a structured directory layout to manage the transformation of data from raw predictions to final statistics. This structure ensures reproducibility and traceability.

### 1. Benchmarks (`data/1-benchmarks`)
This directory contains the immutable definitions of the evaluation tasks.
*   `definitions/`: Gzipped JSON files defining the targets (IDs and SMILES).
*   `stocks/`: Text files containing available building blocks (one SMILES per line).

### 2. Raw Data (`data/2-raw`)
This directory holds the primary artifacts generated by the models. These files are treated as read-only sources of truth.
*   Structure: `data/2-raw/<model>/<benchmark>/<filename>`

### 3. Processed Data (`data/3-processed`)
The `retrocast ingest` command transforms raw data into standardized `Route` objects.
*   **Format:** A dictionary mapping `target_id` to a list of `Route` objects.
*   **Operations:** At this stage, routes are deduplicated (and optionally sampled to keep only first *n* routes).

### 4. Scored Data (`data/4-scored`)
The `retrocast score` command evaluates processed routes against a specific stock file.
*   **Metrics:** Routes are annotated with boolean flags (e.g., `is_solved`) and comparisons to ground truth.
*   **Independence:** Scoring is separate from ingestion, allowing the same routes to be scored against multiple stocks without re-processing.

### 5. Results (`data/5-results`)
The `retrocast analyze` command aggregates scored data into statistical distributions.
*   **Bootstrap:** Confidence intervals are calculated for metrics like Top-K accuracy.
*   **Artifacts:** This directory contains JSON statistics, Markdown reports, and HTML visualizations.

## Provenance and Verification

Reproducibility is a primary design goal. RetroCast tracks the lineage of every data artifact using **Manifests**.

A manifest is a JSON file associated with every generated dataset (e.g., `routes.json.gz` has a companion `routes.manifest.json`). It records:

1.  **Action:** The command or script that generated the file.
2.  **Inputs:** The paths and SHA256 hashes of all source files used.
3.  **Parameters:** Configuration arguments (e.g., stock name, random seed).
4.  **Outputs:** The paths and hashes of the generated files.

The `retrocast verify` command uses these manifests to audit the data pipeline. It performs a two-phase check:
1.  **Logical Consistency:** Ensures that the input hash recorded in a child manifest matches the actual output hash of the parent manifest.
2.  **Physical Integrity:** Ensures that the file currently on disk matches the hash recorded in its manifest.

This system detects data corruption, manual tampering, or out-of-order execution steps.
