# Propose Next Experiment

## CLI Reference

Run all abreka commands with `uv run abreka <command>`.

**Finalizing the reserved experiment:**
- `uv run abreka exp update {{reserved_exp_id}} --hypothesis "..." --method "..." --parent <id> --tag <tag> --status proposed`  (extending prior work)
- `uv run abreka exp update {{reserved_exp_id}} --hypothesis "..." --method "..." --tag <tag> --status proposed`  (new direction, no parent)

**Querying experiments:**
- `uv run abreka exp show <id>` — full details: hypothesis, method, metrics, findings
- `uv run abreka exp list --sort metric:val:accuracy` — list sorted by metric
- `uv run abreka exp list --parent <id>` — list children of an experiment
- `uv run abreka exp search "<query>"` — keyword search over hypotheses, methods, findings
- `uv run abreka exp tree` — show the experiment DAG
- `uv run abreka exp diff <id1> <id2>` — compare two experiments side-by-side

**Backlog:**
- `uv run abreka backlog list` — list all open backlog ideas
- `uv run abreka backlog show <id>` — full details of a backlog idea
- `uv run abreka backlog resolve <id> --exp <new_exp_id>` — mark a backlog idea as resolved after creating its experiment

**Project Wiki** (durable cross-experiment knowledge — see "Project Wiki" section below):
- `uv run abreka wiki list` — list every wiki page
- `uv run abreka wiki show <slug>` — read a page in full
- `uv run abreka wiki search "<query>"` — keyword/regex search across pages

{{wiki_index}}

## Research Goal

{{goal}}

## Primary Metric

{{primary_metric}} ({{metric_direction}})

## Branch Overview

{{branch_overview}}

{{focused_branch_summary}}

## Experiment History

{{experiment_context}}

{{focus_constraint}}

## Pending Backlog Ideas

{{backlog_context}}

{{guidance}}

## Your Task

Choose the highest-value next expansion of the experiment graph. Use the wiki, branch overview, focused branch summary (if any), experiment history, backlog, and research goal to decide where the next unit of effort should go.

**Reserved experiment ID:** `{{reserved_exp_id}}`

You must populate this already-reserved experiment and set its status to `proposed`. Do **not** create another experiment.

Your job is to make a good graph-search decision, not just to invent an interesting experiment. Your next experiment should be one of:

- **Exploit**: extend a branch that is still improving and close to the global best.
- **Discriminate**: isolate an important uncertainty or confound in a promising branch — change one variable at a time from a known baseline so the outcome tells you *which* factor matters.
- **Explore**: start a fundamentally new branch because existing branches are plateauing, noisy, or the search space has major gaps.

Do NOT choose a branch just because it currently holds the best metric. Also consider whether it is plateauing, noisy, underexplored, or confounded — the branch overview's health and recommended actions are there to help.

Before creating the experiment, decide:
1. **Which branch or subtree you are expanding** (or "new branch" if exploring).
2. **Which mode applies**: exploit, discriminate, or explore.
3. **Why this choice is better than the most plausible alternatives**.

Your proposal must include:
1. **Hypothesis**: A clear, testable statement of what you expect to happen and why.
2. **Method**: A concise description of the experimental approach. If building on prior work, describe what you'll change. If starting a new direction, describe the approach from scratch.
3. **Parent experiments** (if applicable): Which prior experiment(s) this builds on (by ID). Omit parents entirely when proposing a genuinely new direction.
4. **Tags**: A few keywords for categorization.

### Decision guidance

**Extend an existing line** (include `--parent`):
- There is a promising experiment with clear room to improve.
- The best result is still improving and the next logical step is obvious.
- You want to isolate a single variable change from a known baseline.

**Start a new direction** (omit `--parent`):
- All active branches are plateauing or showing diminishing returns.
- The search space has obvious gaps — entire families of approaches remain untried.
- Prior findings suggest the current paradigm has a ceiling.
- There are no experiments yet, or very few.

General principles:
- Avoid re-testing ideas that prior findings already ruled out.
- Prefer experiments that change one variable at a time for clearer attribution.
- When starting a new direction, make your hypothesis specific enough to be falsifiable — do not just "try something different."

### How to work

- Your working directory is the **project root**. All project source files, tests, and experiment code are under `.` — use paths like `src/...`, `tests/...`, etc. Do NOT use `..` to navigate to a parent directory.
- Review the experiment history above to understand what has been tried, what worked, and what failed.
- Skim the wiki index above and read any pages relevant to the proposal. The wiki holds prior EDA, methodology lessons, and durable cross-experiment knowledge — use it before reasoning from scratch.
- If you need more detail on a specific experiment, use `uv run abreka exp show <id>` to see its full hypothesis, method, metrics, and findings.
- You may read project source files to understand the current model code.

### Optional EDA

If a relevant data question would meaningfully sharpen this proposal, do a small amount of exploratory data analysis before finalizing. Save any artifacts (notes, plots, summary stats) under `abreka/experiments/{{reserved_exp_id}}/artifacts/eda/`. Keep EDA scoped to what informs *this* proposal — anything broader belongs in a dedicated EDA experiment. The evaluate stage will later promote durable EDA findings into the wiki.

After deciding, finalize the reserved experiment. If extending prior work:
```bash
uv run abreka exp update {{reserved_exp_id}} --hypothesis "Your hypothesis here" --method "Brief method" --parent <parent_id> --tag <tag1> --tag <tag2> --status proposed
```

If starting a new direction with no parent:
```bash
uv run abreka exp update {{reserved_exp_id}} --hypothesis "Your hypothesis here" --method "Brief method" --tag <tag1> --tag <tag2> --status proposed
```
