# Performance Reference

Benchmarks collected on a Windows desktop (AMD Ryzen / Intel i7-class, 32 GB RAM)
with Python 3.12, OpenMM 8.x, Tinker 25.6, and Psi4 1.10. All times are
wall-clock. Your mileage will vary with hardware and molecule size.

---

## Single-Point Energy Evaluation

How long does one energy call take? This is the inner loop of optimization —
every objective function evaluation requires one energy call per molecule per
geometry.

| Backend | Molecule | Cold Start | Warm Avg | Throughput |
|---------|----------|------------|----------|------------|
| **OpenMM** | Water (3 atoms) | 248 ms | **5 ms** | ~190 eval/s |
| **Tinker** | Water (3 atoms) | 168 ms | **163 ms** | ~1.3 eval/s |

**Key takeaways:**

- **OpenMM is ~150× faster** than Tinker in optimizer throughput. OpenMM runs
  in-process and reuses its context across evaluations, while Tinker spawns a
  fresh subprocess, writes files, and parses output for every call.
- **Cold start** includes system/topology building (OpenMM) or first file I/O
  (Tinker). After warmup, OpenMM evaluations drop to ~5 ms each.
- For larger molecules, OpenMM's advantage grows because its in-process
  evaluation scales better than Tinker's file-based I/O.

### Cross-Backend Parity

Both backends agree within **0.003 kcal/mol** at displaced geometries for the
same MM3 functional form. At equilibrium geometry, both give exactly
0.000 kcal/mol.

---

## Optimizer Performance

Full optimization run: perturbed water FF (4 parameters: bond k, bond eq,
angle k, angle eq) optimized to recover known true parameters.

### OpenMM Backend

| Method | Time | Evaluations | Evals/s | Initial → Final Score |
|--------|------|-------------|---------|----------------------|
| **Nelder-Mead** | 2.2 s | 410 | 190.5 | 42.9 → 0.000 |
| **L-BFGS-B** | 3.1 s | 576 | 187.7 | 42.9 → 0.932 |
| **Powell** | 5.8 s | 1118 | 194.4 | 42.9 → 0.000 |

### Tinker Backend

| Method | Time | Evaluations | Evals/s | Initial → Final Score |
|--------|------|-------------|---------|----------------------|
| **Nelder-Mead** | 286.9 s | 376 | 1.3 | 42.9 → 0.000 |

!!! warning "L-BFGS-B convergence"
    L-BFGS-B does not fully converge on this problem (final score 0.93 vs
    0.000 for Nelder-Mead and Powell). Finite-difference gradients with
    `eps=1e-3` can miss shallow features in the objective landscape. For
    production use, prefer Nelder-Mead or Powell unless analytical gradients
    are available.

**Key takeaways:**

- **OpenMM is ~150× faster** per evaluation than Tinker (~190 eval/s vs ~1.3
  eval/s). OpenMM runs in-process; Tinker spawns a subprocess per call.
- **Nelder-Mead** is the most reliable derivative-free method, converging to
  score ~0 in ~400 evaluations.
- **Powell** also converges fully but uses ~2.7× more evaluations.
- **L-BFGS-B** uses finite-difference gradients. Each "iteration" costs
  `2 × n_params + 1` evaluations. With 4 params, that's 9 evaluations per
  step. It converges faster in wall-time but may not reach the global minimum.
- **For 4 parameters**, OpenMM-backed methods converge in under 10 seconds.
- **Scaling estimate**: a 20-parameter FF with 5 molecules would need
  ~5× more evaluations per objective call (5 molecules), and gradient methods
  would need ~41 evaluations per step (2×20+1). Expect ~1–3 minutes per
  optimization with OpenMM.

*Benchmarks generated by `scripts/generate_benchmarks.py` — all methods
start from identical perturbed parameters to ensure comparable initial scores.*

---

## Seminario Method

Extracting bond/angle force constants from a QM Hessian matrix.

| Molecule | Atoms | Time |
|----------|-------|------|
| Water | 3 | **0.4 ms** |
| Rh-enamide TS | ~60 | **< 50 ms** (estimated) |

The Seminario method is pure NumPy linear algebra (eigenvalue decomposition of
3×3 Hessian sub-blocks). It's effectively instant compared to everything else
in the pipeline. Even a 200-atom molecule would take < 1 second.

---

## QM Calculations (Psi4)

These are typically run once per molecule to generate reference data, not during
the optimization loop.

| Calculation | Level | Molecule | Time |
|------------|-------|----------|------|
| **Energy** | B3LYP/6-31G* | Water (3 atoms) | 1.1 s |
| **Hessian** | B3LYP/6-31G* | Water (3 atoms) | 7.8 s |

**Scaling notes:**

- QM cost scales as O(N³)–O(N⁴) with basis functions. A 30-atom organic
  molecule with 6-31G* takes ~5–30 minutes for a Hessian.
- Transition state Hessians (for TSFF work) take the same time but contain
  one negative eigenvalue along the reaction coordinate.
- Psi4 parallelizes well — set `psi4.set_num_threads(N)` for multi-core
  speedup.

---

## Test Suite Timing

| Test Category | Tests | Time | Notes |
|---------------|-------|------|-------|
| Unit tests (models, parsers) | ~45 | **< 2 s** | Pure Python, no backends |
| Integration (Seminario parity) | ~25 | **~1 s** | Fixture-based, no live QM/MM |
| Integration (scipy optimizer) | 10 | **~220 s** | Real OpenMM evaluations |
| Integration (cross-backend) | 10 | **~19 min** | OpenMM + Tinker, full pipelines |
| **Total (without cross-backend)** | **114** | **~3:47** | Default `pytest -q` |
| **Total (all)** | **124** | **~23 min** | All integration tests |

**Recommendation:** For development iteration, skip the cross-backend validation
tests:

```bash
# Fast iteration (3–4 minutes)
pytest -q --ignore=test/integration/test_optimization_validation.py

# Full validation (before PR/merge)
pytest -q
```

---

## Bottleneck Analysis

For a typical TSFF optimization workflow:

```
QM Hessian (one-time)    ████████████████  7.8 s
Seminario (one-time)     ▏                 0.0004 s
Optimization loop        ████████████████████████████████████  20–45 s
  └─ per evaluation      ██                0.1–0.16 s (energy call)
```

**The energy evaluation is the bottleneck.** Every optimization evaluation
calls the MM backend once per molecule per geometry. Strategies to speed up:

1. **Use OpenMM over Tinker** — 1.6× faster per evaluation (in-process vs
   subprocess).
2. **Reduce evaluations** — Nelder-Mead converges reliably in ~400 evaluations.
   Powell converges too but uses ~1100. L-BFGS-B may not fully converge with
   finite-difference gradients.
3. **Fewer molecules/geometries** — each additional training point adds one
   energy call per evaluation.
4. **Analytical gradients** (future) — would eliminate the `2N+1` overhead
   for gradient methods, reducing cost from O(N) to O(1) energy calls per
   optimization step.

---

## Environment Variables

| Variable | Purpose | Example |
|----------|---------|---------|
| `TINKER_DIR` | Tinker installation directory | `C:\tinker` |
| `TINKER_PRM` | Default Tinker parameter file | `C:\tinker\params\mm3.prm` |

If not set, Tinker paths are auto-detected from common install locations.

---

*Last updated: March 2026. Benchmarks on AMD/Intel desktop, Windows 11,
Python 3.12.*
