# ALADIN Core Concepts — Product Requirements Document

| Field | Value |
| --- | --- |
| Status | Draft |
| Version | v0.1 |
| Author | JuneTech |
| Date | 2026-02-26 |

---

## 1. Background

ALADIN is an infrastructure for implementing and experimenting with algorithms described in academic papers.
A recurring problem when asking an LLM to implement a paper's algorithm and experiment is that the two tasks are entangled:
the LLM tends to produce a single monolithic script that mixes data feeding, algorithm logic, and result reporting.
This makes the implementation hard to verify, reuse, or extend.

To address this, ALADIN decomposes an experiment into five well-defined concepts with explicit data contracts between them.
Each concept can be implemented and tested independently, and an LLM can be given one concept at a time to implement.

---

## 2. Goals

- Enable LLMs to implement each concept independently without needing to understand the whole system.
- Ensure traceability: every experiment records all settings required to interpret its results.
- Ensure verifiability: every artifact produced by a run is attributable to a specific execution context.
- Separate concerns so that algorithm logic is never contaminated by reporting concerns, and vice versa.

---

## 3. The Five Concepts

The five concepts and their data flow:

```plaintext
Launcher
  │
  ├─── reads: Problem Instance set, Algorithm Option set
  ├─── produces: AlgSpec (per pair)
  │
  ▼
Algorithm
  │
  ├─── receives: AlgSpec
  ├─── produces: AlgRecord
  │
  ▼
Reporter
  │
  ├─── reads: AlgRecord set, Instance Metadata (user-provided)
  └─── produces: Report (tables, figures, statistics)
```

Launcher and Reporter never call each other. Algorithm never calls Reporter. Reporter never calls Algorithm.

---

## 4. Concept Definitions

### 4.1 Problem Instance

A Problem Instance is the input data for a single optimization problem instance. It is a passive data object — it has no behavior and knows nothing about algorithms or reporting.

**Contents:**

- `instance_id`: unique identifier (string)
- Problem data specific to the problem class (e.g., distance matrix, job list, graph edges)

**Invariants:**

- A Problem Instance is immutable once created.
- Two Problem Instances with the same `instance_id` must have identical data.

**What it is not:**

- It does not contain algorithm parameters.
- It does not contain reference/best-known values (those belong to Instance Metadata, provided separately to Reporter).
- It does not contain a reference solution (that is passed separately in AlgSpec).

---

### 4.2 Launcher

The Launcher is responsible for coordinating an experiment: it enumerates (Problem Instance, Algorithm Option) pairs, constructs AlgSpecs, and dispatches them to Algorithm for execution. The Launcher decides parallelism.

**Responsibilities:**

- Load or generate the set of Problem Instances for the experiment.
- Define the set of Algorithm Options to be evaluated.
- Construct one AlgSpec per (Problem Instance, Algorithm Option) pair.
- Dispatch AlgSpecs to Algorithm, in parallel if configured.
- Collect AlgRecords and persist them to the experiment directory.
- Monitor hardware resource usage (CPU %, thermal throttling, RAM, swap) throughout execution and write the resource log.
- Propagate `quiet` mode to Algorithm and Reporter.

**Inputs:** Problem Instance set, Algorithm Option set, experiment configuration (parallelism, seed policy, quiet flag).

**Outputs:** AlgRecord set written to the experiment directory; resource log (`resource_log.csv`).

**Parallelism:** The Launcher is the sole authority on parallelism. Algorithm has no awareness of whether it is running in parallel with other instances.

**Quiet mode:** If `quiet=True` is set on the Launcher, console output is suppressed for Launcher, Algorithm, and Reporter alike. File-based logs are always written regardless of quiet mode.

**Directory structure:** The Launcher instantiates ExperimentResolver with a fresh experiment ID at the start of execution and uses it to create the experiment directory structure.

---

### 4.3 Algorithm

The Algorithm receives a AlgSpec and produces a AlgRecord. It is a self-contained execution unit from AlgSpec to AlgRecord — it has no side effects beyond writing its record to `alg_root`.

**Responsibilities:**

- Execute the algorithm logic on the Problem Instance contained in the AlgSpec.
- Record all data required by Reporter in the AlgRecord: final result, timing, progress log, objective/bound time series.
- Write the AlgRecord to `alg_root` provided in AlgSpec. If `alg_root` is `None`, skip file writing.
- Know nothing about other Algorithm executions, the experiment as a whole, or Reporter.

**Inputs:** AlgSpec (contains: Problem Instance, Algorithm Option, reference solution if any, alg_root if file writing is enabled).

**Outputs:** AlgRecord written to `alg_root`.

**What Algorithm must not do:**

- Access `experiment_root` or any sibling run directory.
- Read Instance Metadata or best-known objective values.
- Communicate with Reporter or Launcher during execution.

**Timing:** Only the execution phase (between preprocessing and postprocessing) is timed. Preprocessing (data loading, initialization) and postprocessing (serialization) are excluded from the timed region.

---

### 4.4 AlgSpec

AlgSpec is the data contract passed from Launcher to Algorithm. It bundles everything Algorithm needs for a single execution.

**Contents:**

- `instance`: the Problem Instance
- `option`: the Algorithm Option (includes all algorithm parameters, solver options, termination policy, random seed, `progress_log_mode`, and `progress_log_interval_ms`)
  - `progress_log_mode`: `time_interval` (default) or `improvement_triggered`
  - `progress_log_interval_ms: float` — sampling interval in milliseconds; required when mode is `time_interval`
- `ref_solution` (optional): an externally provided initial solution; `None` for cold-start
- `alg_root` (optional): path to the directory where Algorithm writes its AlgRecord; set by Launcher via ExperimentResolver. If `None`, file writing is skipped (useful for timing measurements without I/O overhead).

**Invariants:**

- AlgSpec is immutable: Algorithm must not modify it.
- All fields required for traceability are present in AlgSpec. Algorithm must not rely on any external state not present in AlgSpec.

---

### 4.5 AlgRecord

AlgRecord is the data contract passed from Algorithm to Reporter. It is an immutable record of a single execution.

**Contents:**

- `instance_id`: identifies which Problem Instance was solved
- `algorithm_id`: identifies the algorithm implementation
- `option`: the Algorithm Option used (copied from AlgSpec for traceability)
- `result`: the best solution found and its objective value
- `timing`: wall-clock time and CPU time of the execution phase (milliseconds)
- `progress_log`: list of entries recorded during execution
  - `elapsed_ms: float` — time elapsed since execution phase start (milliseconds)
  - `ref_obj_val: float | int` — objective value of the current reference solution
  - `note: str | None` — optional context (e.g., restart count, subalgorithm step name)
- `objective_bound_series` (optional): time series of objective value and bound for solver-based algorithms
- `work_status`: outcome semantics (e.g., feasible, optimal, proven infeasible)
- `termination_reason`: why execution stopped (e.g., time limit, gap threshold, no-improvement)
- `error` (optional): full traceback if execution failed

**Invariants:**

- AlgRecord is immutable once produced.
- `work_status` must not encode termination reasons.
- `termination_reason` must not encode solution quality.

---

### 4.6 Reporter

The Reporter reads AlgRecords and user-provided Instance Metadata to produce reports. It operates asynchronously: it can begin as soon as a benchmark set's AlgRecords are available.

**Responsibilities:**

- Read AlgRecords from the experiment directory.
- Read Instance Metadata provided by the user (best-known values, theoretical bounds, paper reference values).
- Compute per-instance and per-algorithm metrics (RPD, RDI, rank, etc.).
- Produce comparison tables and aggregate statistics as CSV files.
- Produce figures if configured.

**Inputs:** AlgRecord set (from experiment directory), Instance Metadata file (user-provided CSV).

**Outputs:** Report files (CSV tables, figures) written to the experiment directory.

**What Reporter must not do:**

- Call Algorithm or re-execute any computation.
- Modify AlgRecords.

**Instance Metadata format:** A user-provided CSV file with columns `instance_id` and one column per reference metric (e.g., `best_known_value`, `theoretical_lower_bound`). Empty cells mean no reference is available for that (instance, metric) pair; the instance-best fallback applies.

---

## 5. File System Layout

All path structure is determined by ExperimentResolver. Launcher and Reporter both instantiate ExperimentResolver to resolve paths; neither hardcodes directory structure.

```
{experiment_root}/
└── {experiment_id}/                   # timestamp: YYYYMMDD_HHmmss_ffffff
    ├── experiment_meta.json           # experiment-level metadata (Launcher writes)
    ├── environment_snapshot.json      # hardware/software snapshot at start (Launcher writes)
    ├── resource_log.csv               # time-series hardware usage (Launcher writes)
    ├── algorithm_ids.json             # algorithm ID list (ExperimentResolver manages)
    ├── instance_ids.json              # instance ID list (ExperimentResolver manages)
    │
    ├── params_{hash}/                 # one directory per Algorithm Option setting
    │   ├── params.json                # Algorithm Option for this setting
    │   └── {benchmark_id}/            # one directory per benchmark set
    │       ├── {instance_id}.json     # AlgRecord per instance (Algorithm writes)
    │       └── ...
    │
    └── reports/                       # Reporter writes here
        ├── comparison_{timestamp}.csv
        ├── aggregate_{timestamp}.csv
        └── ...
```

**`experiment_root`** is the base directory passed to ExperimentResolver, known to Launcher and Reporter.
**`alg_root`** is the `{experiment_id}/params_{hash}/{benchmark_id}/` directory resolved by ExperimentResolver and passed to Algorithm via AlgSpec. Algorithm writes its AlgRecord here and knows nothing above this level.

---

## 6. ExperimentResolver

ExperimentResolver is the single authority on directory structure. Launcher and Reporter both use ExperimentResolver as their sole reference for file system navigation; neither hardcodes any path. Algorithm does not use ExperimentResolver — it only knows `alg_root` as provided in AlgSpec.

**Inputs at construction:**

- `experiment_root`: base directory for all experiments
- `experiment_id`: timestamp string (`YYYYMMDD_HHmmss_ffffff`), determined by Launcher at experiment start

**Responsibilities:**

- Determine and create the experiment directory (`{experiment_root}/{experiment_id}/`).
- Generate `params_{hash}` directory names from Algorithm Option content (hash generation logic lives here).
- Resolve `alg_root` for a given (params hash, benchmark ID) pair — passed to Algorithm via AlgSpec.
- Provide paths for all experiment-level files: `experiment_meta.json`, `environment_snapshot.json`, `resource_log.csv`.
- Provide paths for all AlgRecord files given (params hash, benchmark ID, instance ID).
- Provide the `reports/` directory path for Reporter output.
- Read and write the algorithm ID list and instance ID list as files within the experiment directory.
- Validate that expected directories and files exist; raise explicit errors when they do not.

**Usage by Launcher:**

Launcher instantiates ExperimentResolver with a fresh experiment ID at the start of each experiment. It uses ExperimentResolver to create directories, resolve `alg_root` for each AlgSpec, and write experiment-level metadata.

**Usage by Reporter:**

Reporter instantiates ExperimentResolver with an existing experiment ID to read from a completed or in-progress experiment. It uses ExperimentResolver to enumerate all AlgRecord paths and resolve the `reports/` output directory. Reporter never needs to know how directories are structured — it only asks ExperimentResolver.

---

## 7. Randomized Repetitions

Randomized repetitions are supported by executing multiple independent experiments, each with a different random seed, and grouping them under a common **Repetition Group**.

Repetition is modeled at the experiment level, not at the Algorithm or AlgRecord level.

### 7.1 Design Principle

- Each random seed corresponds to one fully independent experiment.
- Algorithm is unaware of repetition count or repetition index.
- Repetition grouping is handled entirely via experiment metadata and Reporter logic.
- Aggregation across repetitions is performed by specialized Reporters that operate on multiple experiment directories.

---

### 7.2 Repetition Group

A **Repetition Group** represents a set of experiments that share identical configuration except for the random seed.

Each experiment belonging to a repetition group must contain the following additional fields in `experiment_meta.json`:

- `repetition_group_id`: unique identifier for the repetition group
- `seed`: random seed used for this experiment
- `repetition_count`: total number of intended repetitions
- `seed_list`: list of all seeds in the group (optional but recommended)
- `base_config_hash`: hash of experiment configuration excluding the random seed

The `repetition_group_id` must be identical across all experiments in the group.

The `base_config_hash` ensures that experiments in the same group differ only by seed.

---

### 7.3 Execution Model

Given a seed list `[s1, s2, ..., sk]`:

- The Launcher executes `k` independent experiments.
- Each experiment:

  - Has its own `experiment_id` (timestamp-based).
  - Has its own directory under `{experiment_root}`.
  - Contains full experiment metadata.
  - Contains complete AlgRecords.

There is no shared directory structure across repetitions.

---

### 7.4 Aggregation Reporter

Aggregation across repetitions is performed by a specialized Reporter that:

- Accepts a list of `experiment_root` paths or a repetition group identifier.
- Loads `experiment_meta.json` from each experiment.
- Verifies:

  - All experiments share the same `repetition_group_id`.
  - `base_config_hash` values are identical.
  - Benchmark sets and parameter configurations are consistent.
- Aggregates AlgRecords across seeds.

Aggregation metrics may include:

- Mean objective value
- Standard deviation
- Median
- Best-of-k
- Worst-of-k
- Success rate
- Confidence intervals

Aggregation must tolerate partial completion:

- If fewer than `repetition_count` experiments are present,

  - Reporter must record the actual number of successful repetitions.
- Experiments containing errors are either:

  - Excluded with explicit reporting, or
  - Counted as failed runs according to Reporter policy.

---

### 7.5 Directory Independence

Repetition does not modify the core directory layout:

```plaintext
{experiment_root}/
└── {experiment_id}/
```

Each repetition remains a first-class experiment.

No additional `rep_k` directory layer is introduced.

Repetition grouping exists purely through metadata.

---

## 8. Assumptions

- Experiments are research-oriented, not production deployments. Compilation time and disk I/O overhead are acceptable.
- File-based storage is the primary persistence mechanism. No database is assumed.
- Parallelism is at the (instance, algorithm option) pair level. Intra-algorithm parallelism is the algorithm's own concern and is not managed by Launcher.
- The user is responsible for providing Instance Metadata. ALADIN does not automatically extract reference values from papers.

---

## 9. Out of Scope

- Intra-algorithm parallelism policy.
- Automatic extraction of Instance Metadata from papers.
- GUI or real-time visualization.
- Network-based or distributed execution.

---

## 10. Open Questions

None at this time.
