# Debugging Enhancements Review + Literature Synthesis (Axes A-G)

Date: 2026-02-16  
Scope: Evaluate 8 proposed llmdebug enhancements and map evidence from recent + classic literature across seven research axes.

## Executive Verdict

Your ideas are directionally strong. The biggest caveat from literature is not "whether these signals help", but "how much noise they introduce if unfiltered."  

Recommended rollout:

1. Build now:
   - `#2 SBFL suspiciousness (Ochiai)`
   - `#4 Violated invariant hints` (ranked hints only)
   - `#6 Call graph summary` (budget-capped, provenance-aware)
2. Pilot next:
   - `#1 Variable provenance` (shallow backward slice only)
   - `#3 First divergence detection` (only for deterministic tests at first)
   - `#8 Historical lesson capture` (curated lessons, TTL/decay)
3. Defer until infra hardening:
   - `#5 Partial execution trace` (unless aggressively filtered/summarized)
   - `#7 Branch decisions` (attach to trace pilot; do not build standalone first)

## Cross-Axis Findings (A-G)

## A) Execution Traces & Provenance
- Strong positive signal exists, but only when traces are structured and short.
- NExT and LDB report clear gains from execution information.
- A 2025 trace-for-repair study shows long/raw traces can hurt and be truncated.
- Practical implication: provenance should be selective (causal chain snippets), not raw event dumps.

## B) Fault Localization Signals
- SBFL remains highly effective as a low-cost ranking prior.
- Classic work supports Ochiai as a robust baseline.
- LLM-era systems (AgentFL, LLM4FL) still benefit from SBFL-style ranking even with richer reasoning.
- Practical implication: SBFL is a high-ROI early addition.

## C) Constraint/Invariant Extraction
- Dynamic invariants are useful for debugging context, but noisy and coverage-sensitive.
- Best use is ranked "hinting", not hard truth claims.
- Practical implication: ship simple invariants first (nullness, emptiness, bounds, type drift), with support/confidence metadata.

## D) State Diff & Regression Patterns
- Delta debugging and cause-effect chain literature strongly support pass/fail comparison and minimization.
- First divergence detection can be very high value when replay is stable.
- Practical implication: gate divergence on determinism checks.

## E) Call Graph & Dependency Context
- Inter-procedural context is often decisive in cross-file failures.
- Static call graphs alone are noisy; runtime pruning improves usefulness.
- Practical implication: hybrid static + observed runtime edges, with token budgets.

## F) Minimal Reproduction
- One of the most reliable debugging accelerators in the literature.
- ddmin/HDD/C-Reduce/Perses all show meaningful reduction benefits for root-cause finding.
- Practical implication: add minimization capability to your roadmap; it likely outperforms several heavier signal features.

## G) Structured Prompting Patterns
- Evidence-first, schema-constrained prompting improves consistency and lowers reasoning drift.
- ReAct/Self-Debugging/Reflexion/NExT-style staged reasoning patterns are relevant.
- Practical implication: enforce evidence schema and citation IDs in prompts/retries.

## Assessment of Your 8 Enhancements

| # | Enhancement | Will it likely help? | Why | Primary risks | Verdict |
|---|---|---|---|---|---|
| 1 | Variable provenance | Yes, materially | Improves causal reasoning and hidden-state debugging | Bytecode/source mismatch in dynamic Python; noisy chains | Pilot |
| 2 | SBFL scores | Yes, strongly | Cheap, robust suspiciousness prior; complements LLMs | Non-causal correlations/coincidental correctness | Build now |
| 3 | First divergence point | Yes, high upside | Directly narrows cause in regression-style failures | Requires deterministic pass/fail replay | Pilot |
| 4 | Violated invariant hints | Yes, if framed as hints | Quickly surfaces broken assumptions | False positives if treated as truths | Build now |
| 5 | Partial execution trace | Mixed to strong (conditional) | Useful for value/branch history | Overhead + prompt bloat + noise if not filtered | Defer/pilot behind flag |
| 6 | Call graph summary | Yes, medium-high | Helps cross-file and dependency reasoning | Dynamic language imprecision | Build now (scoped) |
| 7 | Branch decisions | Useful but secondary | Helpful for logic/path bugs | Path explosion and overhead | Bundle with trace pilot |
| 8 | Historical lesson capture | Yes, medium | Prevents repeated dead-end patch strategies | Stale/over-generalized lessons | Pilot with curation |

## Revised Implementation Order (Evidence-Weighted)

1. `#2 SBFL`  
2. `#4 Invariant hints`  
3. `#6 Call graph summary`  
4. `#1 Variable provenance (shallow)`  
5. `#3 First divergence`  
6. `#8 Historical lessons`  
7. `#5 Partial trace`  
8. `#7 Branch decisions` (within trace pipeline)

## Additions Not in Original List (High Value)

1. Minimal reproduction pipeline (`ddmin` first, syntax-aware reducer later).
2. Prompt contract hardening: strict schema, evidence citation IDs, retry rules ("new evidence or changed hypothesis").
3. Determinism/flake guardrail before divergence/minimization features.

## Notes on Source Quality / Verification

- Verified and used heavily: NExT, LDB, AgentFL, LLM4FL, Daikon, DySy, delta debugging, HDD/C-Reduce/Perses, ReAct/Self-Debugging/Reflexion.
- Several names in the original proposal were not reliably verifiable in this pass as canonical papers/venues (examples: `iFix (OOPSLA 2025)`, `DDMIN-LOC (2026)`, `FuzzSight (2025)`, `Quokka (2025)`, `TraceCoder (2026)`, `HLLM historical lessons`).
- Recommendation: keep those as hypotheses unless pinned to stable bibliographic entries.

## Key Sources

### Axis A / B
- NExT (arXiv:2404.14662): https://arxiv.org/abs/2404.14662
- LDB (arXiv:2402.16906): https://arxiv.org/abs/2402.16906
- AgentFL (arXiv:2403.16362): https://arxiv.org/abs/2403.16362
- LLM4FL (arXiv:2409.13642): https://arxiv.org/abs/2409.13642
- Toward Effectively Leveraging Execution Traces for APR (arXiv:2505.04441): https://arxiv.org/abs/2505.04441
- SliceMate (arXiv:2507.18957): https://arxiv.org/abs/2507.18957
- Tarantula (ICSE; DOI): https://doi.org/10.1145/581339.581397
- Ochiai evaluation (PRDC; DOI): https://doi.org/10.1109/PRDC.2006.18

### Axis C / D
- Daikon (TSE 2001): https://homes.cs.washington.edu/~mernst/pubs/invariants-tse2001.pdf
- DySy (ICSE 2008): https://yanniss.github.io/dysy-icse08.pdf
- Are My Invariants Valid? (arXiv): https://arxiv.org/abs/1903.06089
- Delta Debugging overview: https://www.st.cs.uni-saarland.de/dd/
- ESEC/FSE 1999 delta debugging details: https://www.st.cs.uni-saarland.de/publications/details/zeller-esec-1999/
- Simplifying and Isolating Failure-Inducing Input (TSE 2002): https://www.st.cs.uni-saarland.de/publications/details/zeller-tse-2002/

### Axis E / F / G
- CHA (ECOOP 1995): https://projectsweb.cs.washington.edu/research/projects/cecil/www/Papers/hierarchy.html
- RTA context (IBM/OOPSLA 1996 page): https://research.ibm.com/publications/fast-static-analysis-of-c-virtual-function-calls
- HDD (ICSE 2006): https://www.cs.purdue.edu/homes/suresh/502-Fall2008/papers/icse06-hdd.pdf
- C-Reduce (PLDI 2012 page): https://fsl.cs.illinois.edu/publications/regehr-chen-cuoq-eide-ellison-yang-pldi-2012.html
- Perses (ICSE 2018 page): https://2018.icse-conferences.org/details/icse-2018-Technical-Papers/112/Perses-Syntax-Guided-Program-Reduction
- ReAct (arXiv): https://arxiv.org/abs/2210.03629
- Reflexion (arXiv): https://arxiv.org/abs/2303.11366
- Self-Debugging (arXiv): https://arxiv.org/abs/2304.05128

