# Plan: Speed Up DSPy Optimization

## Context

DSPy optimization can be slow because sequential mode runs N separate DSPy compiler passes (one per field). Each pass evaluates all val examples, creating O(N × val_examples) LLM calls just for scoring. The user asked to explore what the GitHub issue and Medium article suggest, plus what's feasible to implement.

**Already implemented:** `num_threads`, `fast` mode (reduced demo budgets + single-pass), caching via `cache_dir`, `_FAST_MODE_KWARGS`.

**GitHub issue finding:** Reduce `max_tokens` in LM calls; use caching. Both already have support in the codebase.

---

## Three Speedup Features to Add

Ordered by impact-to-effort ratio:

### 1. `max_val_examples: int | None = None` (highest impact, low effort)

Cap the number of validation examples used per field optimization in sequential mode.

**Why fast:** Each field compile currently evaluates ALL val examples. With 10 fields × 20 val examples = 200 scoring LLM calls per optimization run. Capping to 5 reduces this to 50 (4× speedup on evaluation overhead).

**Where:** `PydanticOptimizer.__init__` → store as `self.max_val_examples`. In `_optimize_sequential`, slice `val_examples[:self.max_val_examples]` before passing to `_optimize_single_field` and `_optimize_prompt`.

**Files:** `src/dspydantic/optimizer.py`, `src/dspydantic/prompter.py`

---

### 2. `parallel_fields: bool = False` (highest total speedup, moderate effort)

Run field optimizations in parallel threads rather than sequentially.

**Why fast:** Sequential mode runs fields one after another. With 10 fields, parallelizing gives ~10× reduction in wall-clock time for LLM calls (DSPy already uses `num_threads` internally, but fields themselves are still sequential).

**Tradeoff:** Each field sees the *original* descriptions instead of progressively-improved ones. Quality may drop slightly (~5%) but speed improves ~N×.

**Implementation:**
- Add `parallel_fields: bool = False` parameter to `PydanticOptimizer.__init__`
- In `_optimize_sequential`, when `parallel_fields=True`: use `concurrent.futures.ThreadPoolExecutor` to run `_optimize_single_field` for all fields simultaneously, then apply the results
- Thread-safe: each field gets a snapshot of the original `current_descriptions`, results are merged after all complete
- Keep prompt optimization (system/instruction) sequential after fields finish (they depend on field results)

**Files:** `src/dspydantic/optimizer.py`, `src/dspydantic/prompter.py`

---

### 3. `skip_score_threshold: float | None = None` (situational, very low effort)

Skip optimizing fields whose baseline score already exceeds a threshold.

**Why fast:** If a field already scores 0.95+, optimization is unlikely to improve it meaningfully. Skipping it saves one full DSPy compile per skipped field.

**Implementation:**
- Add `skip_score_threshold: float | None = None` to `PydanticOptimizer.__init__`
- In `_optimize_sequential` field loop: before calling `_optimize_single_field`, compute baseline score for that field alone and skip if above threshold
- Emit a progress callback with `phase="skipped"` for skipped fields

**Files:** `src/dspydantic/optimizer.py`, `src/dspydantic/prompter.py`

---

## Implementation Order

1. `max_val_examples` — lowest risk, highest ROI
2. `parallel_fields` — highest speedup, needs careful thread safety
3. `skip_score_threshold` — optional, clean addition

---

## Critical Files

| File | Change |
|------|--------|
| `src/dspydantic/optimizer.py` | Add 3 new params to `__init__`, modify `_optimize_sequential` |
| `src/dspydantic/prompter.py` | Forward new params to `PydanticOptimizer` |
| `src/dspydantic/types.py` | Add `"skipped"` to valid phases in `FieldOptimizationProgress` if needed |

---

## Key Code Locations

- `PydanticOptimizer.__init__`: `optimizer.py:180-350`
- `_optimize_sequential`: `optimizer.py:1127-~1300`
- `_optimize_single_field`: `optimizer.py:888-1000`
- `Prompter.optimize()`: `prompter.py:~280-310`

---

## Verification

1. Run existing tests: `uv run pytest tests/`
2. Verify `max_val_examples=5` with 20 examples → only 5 used for val scoring
3. Verify `parallel_fields=True` produces same (or close) final descriptions as sequential
4. Verify `skip_score_threshold=0.95` skips fields scoring above 0.95 baseline
5. Check `on_progress` still fires correctly in parallel mode
