# handoff_260202_7.md
Repo housekeeping + API sanitization plan for `envelope-decay-fit` (as a library for `wav-to-freq`)

Date: 2026-02-02  
Scope: restructure repo to (1) clearly separate unit tests vs human review runs, (2) prevent “run everything” workflows by default, (3) keep auto-segmentation work without letting it pollute the default path, and (4) provide a clean importable API for integration into `wav-to-freq`.

---

## 0) Goals (non-negotiable)

### G0.1 Library-first behavior
- Importing the package **must not**:
  - write files
  - open Matplotlib windows
  - print large logs
  - run long computations implicitly
- All side-effects move behind **explicit CLI** or **explicit API calls**.

### G0.2 Fast tests stay fast
- `pytest` default run is **fast** (seconds-ish) and deterministic.
- Anything that touches real datasets, renders plots, or loops over many files is **not** a default unit test.

### G0.3 Manual segmentation is the primary workflow
- Manual, human-in-the-loop breakpoint selection is the primary, documented workflow.
- Automatic segmentation may remain as experimental/secondary, but must be isolated and never be the default path.

### G0.4 Integration readiness for `wav-to-freq`
- Provide a small, stable API surface suitable for importing.
- `wav-to-freq` should call into `envelope-decay-fit` without needing to know repo internals or dev scripts.

---

## 1) Proposed repo structure

### 1.1 Target top-level layout
```
envelope-decay-fit/
├─ src/envelope_decay_fit/
│  ├─ __init__.py
│  ├─ api.py
│  ├─ models.py
│  ├─ fitters/
│  │  ├─ __init__.py
│  │  ├─ exp_decay.py
│  │  └─ piecewise.py
│  ├─ segmentation/
│  │  ├─ __init__.py
│  │  ├─ manual.py
│  │  └─ auto/               # quarantined “experimental”
│  │     ├─ __init__.py
│  │     ├─ window_scan.py
│  │     ├─ breakpoints.py
│  │     └─ pipeline.py
│  ├─ metrics.py
│  ├─ plotting/
│  │  ├─ __init__.py
│  │  ├─ storyboard.py
│  │  └─ overlays.py
│  └─ io/
│     ├─ __init__.py
│     └─ load_csv.py
│
├─ tests/                     # pytest “fast only”
│  ├─ test_fitters.py
│  ├─ test_metrics.py
│  ├─ test_breakpoints.py
│  ├─ test_windows.py
│  └─ test_api_smoke.py
│
├─ validation/                # human review + slow runs
│  ├─ README.md
│  ├─ review_runner.py
│  ├─ review_manifest.csv     # optional
│  ├─ bench_fitters.py        # optional
│  └─ bench_windows.py        # optional
│
├─ examples/                  # minimal examples (small inputs, no heavy loops)
│  ├─ quickstart.py
│  └─ manual_segmentation_demo.py
│
├─ scripts/                   # utilities only (no “test_*.py”)
│  ├─ verify_install.sh
│  └─ dev_helpers/...
│
├─ data/
│  ├─ fixtures/               # tiny data for unit tests
│  ├─ review_sets/            # real-ish datasets (may be LFS)
│  └─ external_outputs/       # outputs imported from other toolchains
│
├─ out/                       # generated artifacts (gitignored)
├─ pyproject.toml
└─ README.md
```

### 1.2 Rules of meaning
- `tests/` == “unit-ish”, fast, deterministic, CI-friendly.
- `validation/` == “process like a human”, batch runs, slow, produces plots/reports.
- `examples/` == “how to use the library”, small and simple.
- `scripts/` == glue/utilities; never mistaken for unit tests.
- `data/fixtures/` is the only data used by default unit tests.

---

## 2) Rename/move existing “script tests” (housekeeping actions)

### 2.1 Prohibit misleading names
- No `scripts/test_*.py` (these will be interpreted as pytest tests by humans even if not collected).
- Rename to `run_*`, `bench_*`, or `review_*`.

### 2.2 Move plan (mapping)
For each current file, choose destination based on behavior:

**A) Fast + deterministic** → `tests/`
- Example: pure math tests of fitters/metrics, small synthetic signals.

**B) Uses real datasets / loops many files / makes plots** → `validation/`
- Example: “run all datasets”, “assess quality visually”, “export many PNGs”.

**C) One-off dev experiments** → `experiments/` (optional) or delete after porting learnings.

**Deliverable for worker:** create a concrete mapping list once the worker opens the repo and inspects each script’s behavior.

---

## 3) Testing strategy

### 3.1 Pytest markers
Adopt markers to keep behavior explicit:
- `@pytest.mark.slow`
- `@pytest.mark.dataset`
- `@pytest.mark.interactive` (avoid in CI; prefer to keep interactive outside pytest)

`pytest.ini`:
- register markers
- default run is unmarked tests only

### 3.2 CI default
- Run: `pytest -q -m "not slow and not dataset and not interactive"`
- Optional nightly/manual job: run dataset/slow benches (if needed).

### 3.3 “Golden” validation (optional)
If you want regression protection for the manual workflow without huge compute:
- Store a few small “known-good” fixture files + expected fit outputs (json).
- `test_api_smoke.py` validates:
  - calling API returns stable shapes/types
  - metrics are within tolerance
  - no file outputs by default

---

## 4) Data management / outputs

### 4.1 `out/` is the only default output directory
- All generated plots/reports go to `out/...` unless user explicitly requests otherwise.
- `out/` is gitignored.
- No scripts should write into `data/` or `examples/` outputs by default.

### 4.2 Datasets classification
- `data/fixtures/`: tiny files committed normally
- `data/review_sets/`: larger datasets (consider Git LFS)
- `data/external_outputs/`: imported outputs from other pipelines (not native inputs)

---

## 5) API sanitization for `wav-to-freq` integration

### 5.1 Public API surface (stable)
Create `src/envelope_decay_fit/api.py` and re-export from `__init__.py`.

#### Proposed public functions
1) **Manual segmentation fit (primary)**
```python
fit_piecewise_manual(
    t: np.ndarray,
    y: np.ndarray,
    breakpoints_t: list[float],
    *,
    fitter: str = "exp",               # or enum
    weights: str | None = None,
    return_diagnostics: bool = True,
) -> FitResult
```

2) **Automatic segmentation fit (secondary/experimental)**
```python
fit_piecewise_auto(
    t: np.ndarray,
    y: np.ndarray,
    *,
    config: AutoSegmentationConfig,
    return_diagnostics: bool = True,
) -> FitResult
```

3) **Manual segmentation UI (explicit side effects)**
```python
launch_manual_segmentation_ui(
    t: np.ndarray,
    y: np.ndarray,
    *,
    initial_breakpoints_t: list[float] | None = None,
    ui_config: ManualUIConfig | None = None,
) -> list[float]
```
Notes:
- This function may open windows and is *not* called automatically.
- It returns breakpoints only; fitting remains a separate step so `wav-to-freq` can decide.

4) **Storyboard plotting (explicit)**
```python
plot_segmentation_storyboard(
    t: np.ndarray,
    y: np.ndarray,
    fit: FitResult,
    *,
    ax: matplotlib.axes.Axes | None = None,
    yscale: str = "linear",
) -> matplotlib.figure.Figure
```

### 5.2 Dataclasses / models
Create `models.py` for typed return values.

**FitResult (minimum)**
- `pieces: list[PieceFit]`
- `breakpoints_t: list[float]`
- `global_metrics: GlobalFitMetrics | None`
- `diagnostics: FitDiagnostics | None`

**PieceFit (minimum)**
- `t_start`, `t_end`
- `params` (A0, zeta, etc.)
- `r2`
- `n_points`
- `flags: list[str]` (e.g., “too_short”, “noise_floor”, “transient”)

### 5.3 Strict “no implicit plotting”
- No module-level `plt.show()`.
- Plotting functions return `Figure`/`Axes`.
- UI is in `segmentation/manual.py` and only invoked explicitly.

### 5.4 Logging
- Use Python `logging`, not `print`.
- Default library logging level: WARNING (or quiet).
- CLI can set verbose/DEBUG explicitly.

---

## 6) CLI (optional but recommended)

Add a console script entry point (pyproject):
- `env-decay-fit`

Subcommands:
- `segment <csv>`: launches manual UI; saves breakpoints to a sidecar file if requested.
- `fit <csv> --breakpoints <file|list>`: runs fit and writes outputs (json/png) to `out/`.
- `review <manifest.csv>`: batch review and exports to `out/review/...`.

CLI is allowed to be side-effectful; library is not.

---

## 7) Automatic algorithm retention policy

### 7.1 Keep, but quarantine
- Keep auto pipeline under `segmentation/auto/`.
- Document it as:
  - **experimental**
  - **not the default**
  - **may change without notice**
- Provide a config object to avoid magic constants.

### 7.2 Ditch conditions (only if needed)
Only remove auto if:
- it becomes a maintenance burden
- it blocks shipping manual workflow
- it causes confusion in public API

Default: **keep but isolate**.

---

## 8) Documentation updates

### 8.1 README (top-level)
Must explain:
- what the package does
- the primary workflow: manual breakpoints
- quickstart: fit from known breakpoints
- optional UI segmentation tool
- how to run validation vs unit tests

### 8.2 `validation/README.md`
- how to run batch review
- where outputs go (`out/`)
- how to add datasets to a manifest
- performance cautions

### 8.3 API reference section
- document `FitResult`, `PieceFit`
- document which functions are stable vs experimental

---

## 9) Acceptance criteria

### 9.1 Repo cleanliness
- No “script tests” under `scripts/` named like pytest tests.
- `pytest` default finishes quickly and does not open windows.
- `validation/` contains human review runners and can take long.

### 9.2 Import safety
- `import envelope_decay_fit` has no side effects (no plotting, no disk writes).
- API functions do not write outputs unless user explicitly asks.

### 9.3 Integration
- `wav-to-freq` can:
  - call `fit_piecewise_manual(...)` with breakpoints
  - optionally call `launch_manual_segmentation_ui(...)` in interactive workflows
  - generate storyboard plots using returned fit results

---

## 10) Worker task checklist (execution-ready)

1) Restructure folders per section 1; update imports accordingly.
2) Move/rename scripts per section 2; remove `__main__` from test modules.
3) Create `api.py` + `models.py` and re-export from `__init__.py`.
4) Add pytest markers + `pytest.ini`; ensure default tests are fast.
5) Redirect outputs to `out/` and gitignore it.
6) Document workflows in README + validation README.
7) Optional: add CLI entry points and basic subcommands.

End.
