# Research & Design Decisions

---
**Purpose**: Capture discovery findings and design rationale for the `gpa-parallel-njobs` feature.
---

## Summary
- **Feature**: `gpa-parallel-njobs`
- **Discovery Scope**: Extension
- **Key Findings**:
  - `n_jobs` parameter already declared but not wired; only import + `Parallel`/`delayed` wrapping needed
  - Semilandmark iteration loop has 3 per-specimen loops that can be consolidated into one per-specimen function
  - `_transform_size_and_shape` has a pre-existing bug (undefined attributes) that must be fixed as a prerequisite

## Research Log

### Existing Parallelization Pattern in ktch
- **Context**: Need to understand how `n_jobs` is implemented in other ktch classes
- **Sources Consulted**: `_elliptic_fourier_analysis.py:196-214`, `_spherical_harmonic_analysis.py:157-162`
- **Findings**:
  - Pattern: `Parallel(n_jobs=self.n_jobs, verbose=self.verbose)(delayed(self._method)(args) for i in range(n))`
  - Import: `from sklearn.utils.parallel import Parallel, delayed`
  - EFA uses `_transform_single_2d` / `_transform_single_3d` as per-specimen methods
  - SPHARM uses `_transform_single` / `_inverse_transform_single`
  - Both classes also have a `verbose` parameter
- **Implications**: GPA should follow the same import and dispatch pattern; `verbose` parameter decision needed

### GPA Iteration Structure Analysis
- **Context**: GPA is iterative (while loop), unlike EFA/SPHARM (single-pass). Need to understand parallelization granularity.
- **Sources Consulted**: `_procrustes_analysis.py:255-292` (shape), `_procrustes_analysis.py:301-315` (size-and-shape)
- **Findings**:
  - **Non-semilandmark path** (line 277): Single list comprehension `[sp.spatial.procrustes(mu, x) for x in X_]` — trivially parallelizable
  - **Semilandmark path**: 3 sequential per-specimen loops per iteration:
    1. Lines 259-263: Rotate + accumulate disparity
    2. Line 266 → `_slide_semilandmarks` (lines 540-542): Slide + reproject per specimen
    3. Lines 268-274: Center + scale per specimen
  - Each specimen's processing within all 3 loops depends only on `mu` (shared, read-only), not on other specimens
  - Therefore, the 3 loops can be consolidated into a single per-specimen function without changing semantics
- **Implications**: Consolidated approach reduces `Parallel` overhead from 3 dispatches to 1 per iteration

### `_transform_size_and_shape` Bug
- **Context**: Gap analysis identified undefined attribute references
- **Sources Consulted**: `_procrustes_analysis.py:315`
- **Findings**:
  - Line 315: `return X_.reshape(self.n_specimen_, self.n_landmarks_ * self.n_dim_)` — `self.n_specimen_` and `self.n_landmarks_` are never assigned
  - Line 303: Also uses `X` (the original input) instead of `X_` (the centered copy), so rotations are applied to uncentered data
  - This method is not covered by any test (no `scaling=False` test exists)
- **Implications**: Must fix as prerequisite for Req 2; compute `n_specimen` and `n_landmarks` as local variables

### `verbose` Parameter Decision
- **Context**: EFA and SPHARM both have `verbose` parameter passed to `Parallel`; GPA only has `debug`
- **Sources Consulted**: EFA `__init__`, SPHARM `__init__`, sklearn `Parallel` docs
- **Findings**:
  - `verbose` in sklearn `Parallel` controls progress output (message frequency)
  - GPA's `debug` serves a different purpose (prints iteration convergence info)
  - Adding `verbose` would be a public API change beyond the scope of this spec
- **Implications**: Do not add `verbose` parameter; pass `verbose=0` to `Parallel`

## Architecture Pattern Evaluation

| Option | Description | Strengths | Risks / Limitations | Notes |
|--------|-------------|-----------|---------------------|-------|
| A: Minimal wiring | Wrap each existing loop with `Parallel`/`delayed` | Minimal diff, preserves structure | 3 dispatches per semilandmark iteration | Good for non-semilandmark paths |
| B: Consolidated per-specimen | Extract per-iteration-specimen function | 1 dispatch per iteration, cleaner | More structural change | Best for semilandmark path |
| Hybrid A+B | Option A for simple paths, B for semilandmark | Best of both, minimal overhead | Slightly more design work | **Selected** |

## Design Decisions

### Decision: Hybrid parallelization approach
- **Context**: Non-semilandmark path is a trivial list comprehension; semilandmark path has 3 interleaved loops
- **Alternatives Considered**:
  1. Option A — Wrap each loop independently (3 `Parallel` calls per semilandmark iteration)
  2. Option B — Consolidate all paths into per-specimen functions
  3. Hybrid — Option A for simple paths, Option B for semilandmark path
- **Selected Approach**: Hybrid — trivial wrapping for `_transform_shape` (no semilandmarks) and `_transform_size_and_shape`; consolidated `_iterate_single_semilandmark` for the semilandmark path
- **Rationale**: Non-semilandmark paths are single list comprehensions (trivial to wrap). Semilandmark path benefits from consolidation to avoid per-iteration overhead of 3 `Parallel` dispatches. Each specimen's iteration is independent of others (depends only on shared read-only `mu`).
- **Trade-offs**: Slightly more code for the consolidated function, but significantly fewer `Parallel` dispatches
- **Follow-up**: Verify numerical equivalence across `n_jobs` values in tests

### Decision: Do not add `verbose` parameter
- **Context**: EFA/SPHARM pass `verbose=self.verbose` to `Parallel`, but GPA has no `verbose`
- **Selected Approach**: Pass `verbose=0` (silent) to `Parallel`. Do not add a `verbose` parameter.
- **Rationale**: Adding `verbose` is out-of-scope API change. The `debug` parameter already controls GPA's own logging. `Parallel` verbosity can be added in a future enhancement.

### Decision: Fix `_transform_size_and_shape` bug in-scope
- **Context**: The method references undefined `self.n_specimen_` and `self.n_landmarks_`
- **Selected Approach**: Fix by using local variables (`n_specimen = len(X_)`, `n_landmarks`) and fix the `X` vs `X_` bug
- **Rationale**: Cannot test Req 2 without this fix; it's a small, contained change

## Risks & Mitigations
- **Risk**: `Parallel` overhead per iteration degrades performance for small specimen counts → sklearn's `Parallel(n_jobs=None)` runs sequentially by default; no overhead when `n_jobs=None`
- **Risk**: Numerical non-determinism from parallel execution order → `Parallel` with `loky` backend preserves ordering; tests assert `assert_array_almost_equal`
- **Risk**: `_transform_size_and_shape` fix may have unintended side effects → Method is currently broken and untested; fix is straightforward local variable usage

## References
- [sklearn.utils.parallel.Parallel](https://scikit-learn.org/stable/modules/generated/sklearn.utils.parallel.Parallel.html) — parallelization API
- `ktch/harmonic/_elliptic_fourier_analysis.py:196-214` — reference implementation
- `ktch/harmonic/tests/test_elliptic_fourier_analysis.py:88-106` — reference test pattern
