# Snapshot Retrieval Optimization: Reducing Over-Retrieval & Increasing Success Rate

Date: 2026-02-25
Status: Research complete, pending implementation.
Predecessor: `docs/adaptive_context_retrieval_findings_2026-02-25.md`

## Problem Statement

Eval runs show two correlated issues:

1. **Over-retrieval**: The LLM requests snapshot context in ~90% of adaptive cases.
2. **No lift from context**: Success rate with snapshot ≈ success rate with traceback only.

This document synthesizes findings from five parallel research tracks (40+ papers,
10+ open-source tools) and proposes a prioritized intervention plan.

## Why This Happens: Three Converging Research Threads

### Thread 1 — Context length alone hurts

Even with perfect retrieval, performance degrades 14–85% as input length increases.
Raw snapshot data extends the prompt with semantically-similar-but-irrelevant locals —
the worst kind of distractor.

- "Context Length Alone Hurts LLM Performance Despite Perfect Retrieval"
  ([arXiv:2510.05381](https://arxiv.org/abs/2510.05381), EMNLP 2025 Findings)
- "Context Rot: How Increasing Input Tokens Impacts LLM Performance"
  ([Chroma Research](https://research.trychroma.com/context-rot), 2025)
- "How Is LLM Reasoning Distracted by Irrelevant Context?"
  ([arXiv:2505.18761](https://arxiv.org/html/2505.18761v2), GSM-DC 2025)

### Thread 2 — Execution traces have empirically limited value

Simply incorporating execution traces into prompts works in only 2 of 6 dataset×model
configurations. Effectiveness *diminishes* as trace complexity increases.

- "Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs"
  ([arXiv:2505.04441](https://arxiv.org/abs/2505.04441), KnowledgeNLP@ACL 2025)
- "Do Code Semantics Help?" ([arXiv:2509.11686](https://arxiv.org/abs/2509.11686))

However: LLM-optimized prompts (where traces are reformatted for the model's reasoning
patterns) outperform raw traces consistently.

### Thread 3 — LLMs over-request tools 30%+ of the time

Without calibration, models default to "more information is better." SMART reduced tool
use by 24% while *improving* performance by 37%.

- "SMART: Self-Aware Agent for Tool Overuse Mitigation"
  ([arXiv:2502.11435](https://arxiv.org/abs/2502.11435), ACL 2025 Findings)
- "SMARTCAL: Self-Aware Tool-Use Evaluation and Calibration"
  ([arXiv:2412.12151](https://arxiv.org/abs/2412.12151), EMNLP 2024 Industry Track)

## Current Baseline (Already Implemented)

These changes are in the current diff on `main`:

- `--adaptive-max-helper-calls` default changed to `1` (was unlimited).
- Adaptive retries stay `full` by default (no implicit delta mode).
- Stage-A prompt rewritten to be cost-aware.
- Stage-A uses a *snapshot index* (exception + frame list, no locals) for adaptive conditions.
- Stage-A can request full snapshot via `need_snapshot=true` + `snapshot_justification`.
- Attempt-1 snapshot inclusion denied when justification is generic.
- Progress-based adaptive stop logic added.
- Evidence blocks use stable short IDs (`e_` prefix) and an explicit Stage-A evidence index.

## Proposed Interventions

### TIER 0 — Prompt-level changes (zero architecture change)

#### P0.1 — Restructure Stage-A as "Diagnose-then-Decide"

The current Stage-A asks "do you need more context?" — LLMs default to "yes."
Research (SMART, Self-RAG, Pre-Act) shows that forcing diagnosis before the retrieval
decision is the single highest-impact intervention.

**Change `render_stage_a_request_prompt()` instructions to:**

```
STEP 1 — DIAGNOSE from traceback alone:
- Exception type and message?
- Crash file and line?
- ONE SENTENCE root-cause hypothesis.
- What SPECIFIC fact (variable value, function signature, config key) would
  CHANGE your diagnosis? Name it, or write "none".

STEP 2 — DECIDE:
- If you wrote "none" above → need_more_context=false (you have enough).
- If you named a fact → request ONLY that item (1 request, max 2).
- DEFAULT is need_more_context=false. Historical data shows snapshot-assisted
  patches succeed at the same rate as traceback-only.
```

Add `diagnosis` and `missing_fact` fields to the JSON schema:

```json
{
  "diagnosis": "one sentence hypothesis",
  "missing_fact": "specific named item, or null",
  "need_more_context": false,
  "need_snapshot": false,
  "requests": [],
  "snapshot_justification": "",
  "sufficiency_rationale": ""
}
```

**Why**: VibeRepair (2025) showed that forcing articulation of intended behavior →
root cause → repair improved correct repairs by 19–23%. The "name a missing fact or
decline" pattern from Adaptive-RAG is the gating mechanism. If the LLM cannot name
what is missing, it should not request more context.

**Sources**:
- VibeRepair ([arXiv:2602.08263](https://arxiv.org/abs/2602.08263))
- Adaptive-RAG ([arXiv:2403.14403](https://arxiv.org/abs/2403.14403), NAACL 2024)
- Pre-Act ([arXiv:2505.09970](https://arxiv.org/html/2505.09970v2))

**Expected effect**: Retrieval rate drops from ~90% to ~30–50%.

#### P0.2 — Add calibration few-shot examples to Stage-A

Without examples of correct "no retrieval" decisions, the LLM has no reference point
for when traceback alone suffices. Research shows 2–3 examples is optimal for
tool-calling calibration.

**Add 3 compact examples after the schema:**

```
=== Example: Traceback IS sufficient ===
Pytest: IndexError: list index out of range at items.py:23 in get_item()
Diagnosis: Off-by-one — accesses items[idx] where idx == len(items)
Missing fact: none
Decision: {"need_more_context":false, "diagnosis":"off-by-one in index access"}

=== Example: Traceback IS sufficient (informative message) ===
Pytest: KeyError: 'timeout' at config.py:42 in load_settings()
Diagnosis: Config dict missing key 'timeout'
Missing fact: none — the KeyError names the exact missing key
Decision: {"need_more_context":false}

=== Example: Traceback NOT sufficient ===
Pytest: AssertionError at test_parser.py:45
Diagnosis: Test asserts parse result but traceback has no implementation detail
Missing fact: parser.py source code
Decision: {"need_more_context":true, "requests":[{"type":"file","path":"parser.py"}]}
```

**Sources**:
- Few-shot Dilemma ([arXiv:2509.13196](https://arxiv.org/html/2509.13196v1))
- LangChain few-shot tool-calling ([blog](https://blog.langchain.com/few-shot-prompting-to-improve-tool-calling-performance/))

**Expected effect**: Additional 10–15% reduction in retrieval rate.

#### P0.3 — Add distractor-mitigation instruction to Stage-B

Even when snapshot IS included, irrelevant locals are harmful distractors.

**Add to Stage-B instructions:**

```
If a snapshot is included below: focus ONLY on variables mentioned in the
exception message, the crash line, or the test assertion. Other locals are
likely irrelevant distractors — ignore them.
```

**Sources**:
- GSM-DC ([arXiv:2505.18761](https://arxiv.org/html/2505.18761v2))
- Context Rot ([Chroma Research](https://research.trychroma.com/context-rot))
- Large Language Models Can Be Easily Distracted (ICLR 2024, [OpenReview](https://openreview.net/pdf?id=JSZmoN03Op))

**Expected effect**: +5–10% success rate when snapshot is used.

#### P0.4 — Strengthen the closing section (recency zone)

Due to "Lost in the Middle" (TACL 2024), the LLM attends most to the beginning and
end of the prompt. The current closing is weak: "Goal: return the minimal code edit
that resolves the failing behavior."

**Replace with:**

```
CRITICAL — YOUR PATCH MUST:
- Target the ROOT CAUSE visible in the traceback, not the symptom.
- If the traceback alone identifies the bug, ignore supplementary context.
- Be the smallest correct edit (no refactoring, no added features).
- Not repeat a previously failed equivalent patch.
```

**Source**: Lost in the Middle ([arXiv:2307.03172](https://arxiv.org/abs/2307.03172), TACL 2024)

**Expected effect**: +3–5% success rate.

#### P0.5 — Make budget visible in Stage-A prompt

Agents without budget awareness hit a performance ceiling (BATS, 2025).

**Add to Stage-A:**

```
Budget: You have 1 helper call for this case.
If you use it on low-value context, no further calls are available.
```

**Source**: BATS ([arXiv:2511.17006](https://arxiv.org/abs/2511.17006))

**Expected effect**: 5–10% reduction in retrieval rate.

### TIER 1 — Architectural improvements (moderate code changes)

#### P1.1 — Rule-based traceback informativeness pre-gate

Not all bugs need the same amount of context. ChatDBG showed runtime state gives ~2x
improvement for *semantic* bugs but minimal benefit for crashing bugs where the
exception message is diagnostic.

**Informativeness score from traceback alone:**

| Signal                          | Weight | Logic                                                     |
|---------------------------------|--------|-----------------------------------------------------------|
| Exception type specificity      | 0.30   | SyntaxError/NameError/ImportError = high; AssertionError = low |
| Message contains values         | 0.25   | "expected X got Y", "index N out of range"                |
| Crash frame in user code        | 0.15   | Not in site-packages or stdlib                            |
| Assertion message quality       | 0.15   | Bare `assert` = 0; `assert x == y, f"got {x}"` = high    |
| Frame count / depth             | 0.15   | Short + specific exception = high                         |

**Routing:**

| Score   | Action                                      |
|---------|---------------------------------------------|
| > 0.7   | Skip Stage-A entirely → traceback-only path |
| 0.3–0.7 | Normal Stage-A with diagnose-first prompt   |
| < 0.3   | Stage-A with snapshot index (current design)|

**Category-level default routing (from converging evidence):**

| Category            | P(traceback sufficient) | Default               |
|---------------------|-------------------------|-----------------------|
| syntax_error        | 0.95                    | Skip helper           |
| reference_error     | 0.85                    | Skip helper           |
| path_import_env     | 0.90                    | Skip helper           |
| type_coercion       | 0.50                    | Normal Stage-A        |
| off_by_one          | 0.40                    | Normal Stage-A        |
| wrong_operator      | 0.35                    | Normal Stage-A        |
| missing_condition   | 0.30                    | Normal Stage-A        |
| logic_error         | 0.20                    | Snapshot-indexed      |
| cross_file_contract | 0.30                    | Normal Stage-A        |
| data_shape_runtime  | 0.10                    | Full snapshot         |
| hidden_state        | 0.10                    | Full snapshot         |
| async_temporal      | 0.05                    | Full snapshot         |

**Sources**:
- ChatDBG ([arXiv:2403.16354](https://arxiv.org/abs/2403.16354), FSE 2025)
- DebugBench ([arXiv:2401.04621](https://arxiv.org/abs/2401.04621), ACL 2024)
- Layered Knowledge Injection ([arXiv:2506.24015](https://arxiv.org/abs/2506.24015))
- PyTER ([ACM](https://dl.acm.org/doi/10.1145/3540250.3549130), ESEC/FSE 2022)

**Expected effect**: 20–30% of cases skip retrieval entirely. Remaining cases get
more targeted context.

#### P1.2 — Extractive snapshot compression

When snapshot IS included, dump all locals for up to 3 frames. SWE-Pruner (2026)
showed that task-aware code compression improves success while reducing tokens 23–54%.

**Replace `_format_snapshot_section()` with relevance-filtered version:**

1. Always include: exception info, crash-site frame.
2. For locals, prioritize by:
   - Priority 1: Variables mentioned in exception message
   - Priority 2: Variables on the crash line (from code context)
   - Priority 3: Function parameters
   - Priority 4: Variables with anomalous values (None, empty, shape mismatch)
3. Omit all other locals with a summary: `... N more locals omitted`.
4. Place crash-site frame at both START and END of the section (primacy + recency).

**Sources**:
- SWE-Pruner ([arXiv:2601.16746](https://arxiv.org/abs/2601.16746))
- RECOMP ([arXiv:2310.04408](https://arxiv.org/abs/2310.04408), ICLR 2024)
- LongLLMLingua ([arXiv:2310.06839](https://arxiv.org/abs/2310.06839), ACL 2024)

**Expected effect**: +5–15% success rate when snapshot is used.

#### P1.3 — Post-retrieval relevance check (CRAG-inspired)

Currently, once Stage-A retrieves context, it is always included. CRAG's pattern of
evaluating and potentially discarding irrelevant retrieval is applicable.

**After `execute_context_requests()` but before Stage-B prompt, check:**

```python
def _context_adds_novelty(stage_a_data, traceback_text, snapshot):
    """Returns False if retrieved context is redundant with traceback."""
    # If retrieved files are already visible in traceback frames → no novelty
    # If snapshot locals are all primitives matching expected types → no novelty
    # If total retrieved chars < 100 → too little to be useful
    return novelty_found
```

If no novelty, downgrade to monolithic (traceback-only) path.

**Source**: CRAG ([arXiv:2401.15884](https://arxiv.org/abs/2401.15884))

**Expected effect**: 15–20% of retrievals suppressed (redundant context).

#### P1.4 — Multi-sample + majority vote for patch generation

The pipeline does single-shot patch generation. Agentless showed that generating
multiple samples and selecting by majority vote produces 1.5–2x upper bound
improvement over greedy.

**When `k=1`, generate 3–5 patches at `temperature=0.8` and select by:**

1. Filter: syntax-valid patches only (AST parse check).
2. Normalize: strip whitespace, canonicalize formatting.
3. Vote: select most frequent patch family.

This is orthogonal to the retrieval problem — it improves success rate regardless.

**Source**: Agentless ([arXiv:2407.01489](https://arxiv.org/abs/2407.01489), ICLR 2025)

**Expected effect**: +10–20% success rate.

### TIER 2 — Experimental (A/B test before committing)

#### P2.1 — "Attempt-first, retrieve-on-failure" (FLARE-inspired inversion)

Instead of Stage-A → Stage-B, try: attempt patch with traceback only first. Retrieve
snapshot only if the first attempt fails and the failure signature is uninformative.

This would eliminate ~40–50% of snapshot requests (those where the first attempt would
have succeeded anyway).

**Source**: FLARE ([arXiv:2305.06983](https://arxiv.org/abs/2305.06983), EMNLP 2023)

#### P2.2 — Train retrieval-utility predictor from telemetry

Use existing paired A/B eval data to train a lightweight classifier:

- Features: exception type, traceback depth, message informativeness score, bug
  category, attempt number.
- Target: `P(solve | with_context) - P(solve | traceback_only) > 0`.
- Gate: Only invoke helper when predicted marginal utility > cost threshold.

**Source**: TALE ([arXiv:2412.18547](https://arxiv.org/abs/2412.18547), ACL 2025 Findings)

#### P2.3 — Consistency-based retrieval gating

Generate 2–3 patch proposals at low temperature from traceback-only. If proposals
converge (same file, same fix location), skip retrieval. If they diverge, retrieve
targeted context for the point of disagreement.

**Source**: Rowen ([arXiv:2402.10612](https://arxiv.org/abs/2402.10612))

#### P2.4 — Context-collapsing summaries for retry attempts

For attempt N+1, instead of carrying forward raw evidence from attempt N, carry
forward a 2–3 line structured summary:
`{hypothesis_tested, evidence_used, outcome, new_constraint}`.

This prevents context accumulation that degrades reasoning.

**Sources**:
- Context Rot ([Chroma](https://research.trychroma.com/context-rot))
- Observation masking ([Complexity Trap](https://github.com/JetBrains-Research/the-complexity-trap), NeurIPS 2025 Workshop)

#### P2.5 — Counterfactual controller

On every helper request, run both paths (with and without context) and record
outcomes. Build a growing dataset for the retrieval-utility predictor (P2.2).

## Priority Matrix

| ID     | Intervention                         | Effort | Retrieval reduction | Success lift   | Risk     |
|--------|--------------------------------------|--------|---------------------|----------------|----------|
| P0.1   | Diagnose-then-decide Stage-A         | Low    | **40–60%**          | Neutral to +5% | Low      |
| P0.2   | Calibration examples                 | Low    | **10–15%**          | Neutral        | Very low |
| P0.3   | Distractor mitigation instruction    | Low    | 0%                  | **+5–10%**     | Very low |
| P0.4   | Stronger closing section             | Low    | 0%                  | **+3–5%**      | Very low |
| P0.5   | Visible budget                       | Low    | **5–10%**           | Neutral        | Very low |
| P1.1   | Traceback informativeness pre-gate   | Medium | **20–30%** (skip)   | **+5–10%**     | Medium   |
| P1.2   | Extractive snapshot compression      | Medium | 0%                  | **+5–15%**     | Low      |
| P1.3   | Post-retrieval novelty check         | Medium | **15–20%** (suppress) | Neutral to +5% | Low    |
| P1.4   | Multi-sample majority vote           | Medium | 0%                  | **+10–20%**    | Low      |
| P2.1   | Attempt-first inversion              | High   | **40–50%**          | 0–10%          | Medium   |
| P2.2   | Learned retrieval predictor          | High   | **30–40%**          | +5–10%         | Medium   |
| P2.3   | Consistency-based gating             | Medium | **20–30%**          | +5%            | Medium   |
| P2.4   | Context-collapsing for retries       | Medium | 0%                  | +5–10%         | Low      |
| P2.5   | Counterfactual controller            | High   | (data collection)   | (future)       | Low      |

## Recommended Evaluation Plan

Run an intervention matrix on the same case set and seeds:

1. **Baseline** (current implementation).
2. **P0.1 + P0.2 + P0.3 + P0.4 + P0.5** (all prompt changes, Tier 0 bundle).
3. **#2 + P1.1** (add pre-gate classifier).
4. **#2 + P1.2** (add snapshot compression).
5. **#2 + P1.4** (add multi-sample vote).
6. **#2 + P1.1 + P1.2 + P1.3** (full Tier 0+1 stack).

Track at least:

- `solve_rate`
- `solve_rate_delta_vs_traceback_only`
- `helper_invocation_rate` (target: < 40%, down from 90%)
- `helper_rescue_rate` (target: when helper IS used, solve > traceback-only by ≥ 5pp)
- `mean_tokens_total`
- `mean_tokens_per_attempt`
- `no_progress_stop_rate`
- Solve-on-attempt distribution

**Success criteria for rollout:**

- Non-inferior or improved solve rate vs current baseline.
- Helper invocation rate < 40%.
- When helper IS used, solve rate > traceback-only by ≥ 5pp (targeted value).
- Materially lower token cost per case.

## Codebase Analysis: Current Design Observations

### Strengths

1. **Index-before-locals** in Stage-A for adaptive conditions: showing only frame
   locations (not values) is a principled way to let the LLM decide without biasing.
2. **Evidence ID deduplication** across attempts: `e_` hash IDs prevent re-sending
   identical file content.
3. **Budget hierarchy with named priorities**: drop order ensures LLM-requested
   evidence is the last to be shed.
4. **Attempt-1 snapshot denial** for generic justifications.

### Bias Signals and Gaps

1. **Stage-A prompt structure defaults to "yes"**: The current prompt asks "do you
   need more context?" without requiring diagnosis first. The model's prior is to
   request context.

2. **`need_snapshot` overlaps with saturation guard**: Snapshot-only invocations
   (no file requests) erode the saturation guard even though they returned valid
   reasoning. Consider not incrementing `helper_calls_without_context` for
   snapshot-only invocations.

3. **Fallback context is silent**: When all Stage-A requests fail (bad paths),
   `build_fallback_context_bundle()` sends crash file + test file without indicating
   that the original requests failed. The LLM may think its requests succeeded.

4. **Snapshot position in drop order**: The snapshot is dropped before
   `stage_a_context` under budget pressure. This is correct (explicitly requested
   evidence > snapshot) but means snapshot silently disappears under token pressure.

5. **Generic justification detection is attempt-1 only**: On attempt 2+, any
   `need_snapshot=true` with any justification succeeds. This is intentional but
   means the protection does not persist.

6. **The RCA checklist is marked "internal only"**: Research shows requiring the
   LLM to *output* its reasoning (not just think internally) significantly improves
   accuracy (Self-Debug, ICLR 2024; ThinkRepair, ISSTA 2024).

## SWE-Bench Agent Patterns — Lessons for llmdebug

### Agentless (ICLR 2025): Fixed pipeline eliminates retrieval waste

Three-stage localization: tree → skeleton → full code. Each stage filters
drastically. Total cost: $0.34/instance. 40.7% on SWE-bench Lite with Claude.

Lesson: A fixed pipeline with progressive detail is cheaper and often more effective
than an agent loop where the LLM decides retrieval.

### SWE-agent → Observation Masking (NeurIPS 2025 Workshop)

Simple masking of tool outputs older than 10 turns matches or beats LLM
summarization. 52% cost reduction with +2.6% solve rate improvement.

Lesson: Hiding old verbose context is cheap and effective.

### SWE-Pruner (arXiv 2026): Task-aware neural compression

0.6B parameter skimmer achieves 23–54% token reduction while improving success.
Generic compression (LLMLingua) actually *degrades* code tasks. Code-aware,
task-aware compression is required.

Lesson: Snapshot compression must understand Python types, shapes, and values —
not just prune tokens.

### Aider: Repo map as proactive context

Graph-ranked 1K-token skeleton of the repo prevents exploratory searches entirely.
PageRank with personalization toward active files.

Lesson: Proactive lightweight context reduces the need for expensive retrieval.

### Moatless / SWE-Search: Value function for context evaluation

Separate LLM call scores state utility (0–100). State-specific prompts are
essential — generic value prompts fail.

Lesson: If we add a post-retrieval evaluator (P1.3), it must be context-aware,
not a generic "is this useful?" prompt.

## Error Classification Research

### Traceback-sufficient categories (from ChatDBG, DebugBench, Layered Injection)

Exception types where the message itself identifies the fix:

- `SyntaxError` — near-perfect LLM fix rates
- `NameError` / `ImportError` — undefined symbol named in message
- `FileNotFoundError` — path in message
- Simple `TypeError` — "unsupported operand type(s)" with types named

### Context-beneficial categories

- `ValueError` — depends on message informativeness
- `KeyError` — key named, but may need to see available keys
- Complex `TypeError` — upstream type transformation

### Context-required categories

- `AssertionError` with bare/uninformative message
- Logic errors (wrong output computed)
- Data shape bugs (need actual shapes)
- State management / async bugs (need runtime state)

Key evidence: ChatDBG ablation showed runtime state gives ~2x improvement for
semantic bugs, but enriched stacks alone helped more with crashing errors.

## References

### Core explanatory papers

- Context Length Alone Hurts: [arXiv:2510.05381](https://arxiv.org/abs/2510.05381)
- Execution Traces Limited Value: [arXiv:2505.04441](https://arxiv.org/abs/2505.04441)
- SMART — Tool Overuse Mitigation: [arXiv:2502.11435](https://arxiv.org/abs/2502.11435)
- Lost in the Middle: [arXiv:2307.03172](https://arxiv.org/abs/2307.03172)
- Context Rot: [Chroma Research](https://research.trychroma.com/context-rot)
- GSM-DC Distractor Effects: [arXiv:2505.18761](https://arxiv.org/html/2505.18761v2)
- Do Code Semantics Help: [arXiv:2509.11686](https://arxiv.org/abs/2509.11686)

### Retrieval control patterns

- Self-RAG: [arXiv:2310.11511](https://arxiv.org/abs/2310.11511)
- Adaptive-RAG: [arXiv:2403.14403](https://arxiv.org/abs/2403.14403)
- CRAG: [arXiv:2401.15884](https://arxiv.org/abs/2401.15884)
- FLARE: [arXiv:2305.06983](https://arxiv.org/abs/2305.06983)
- Rowen: [arXiv:2402.10612](https://arxiv.org/abs/2402.10612)
- BATS — Budget-Aware Tool Scaling: [arXiv:2511.17006](https://arxiv.org/abs/2511.17006)
- TALE — Token-Budget-Aware Reasoning: [arXiv:2412.18547](https://arxiv.org/abs/2412.18547)
- SMARTCAL: [arXiv:2412.12151](https://arxiv.org/abs/2412.12151)

### Code repair architectures

- Agentless: [arXiv:2407.01489](https://arxiv.org/abs/2407.01489)
- SWE-Pruner: [arXiv:2601.16746](https://arxiv.org/abs/2601.16746)
- ChatDBG: [arXiv:2403.16354](https://arxiv.org/abs/2403.16354)
- VibeRepair: [arXiv:2602.08263](https://arxiv.org/abs/2602.08263)
- Layered Knowledge Injection: [arXiv:2506.24015](https://arxiv.org/abs/2506.24015)
- ThinkRepair: [arXiv:2407.20898](https://arxiv.org/html/2407.20898)
- Self-Debug: [arXiv:2304.05128](https://arxiv.org/abs/2304.05128)
- RepairAgent: [arXiv:2403.17134](https://arxiv.org/abs/2403.17134)
- PyTER: [ACM DL](https://dl.acm.org/doi/10.1145/3540250.3549130)
- DebugBench: [arXiv:2401.04621](https://arxiv.org/abs/2401.04621)
- LDB: [arXiv:2402.16906](https://arxiv.org/abs/2402.16906)

### Context management

- Complexity Trap / Observation Masking: [GitHub](https://github.com/JetBrains-Research/the-complexity-trap)
- SWE-ContextBench: [arXiv:2602.08316](https://arxiv.org/abs/2602.08316)
- Anthropic Context Engineering: [Blog](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)
- SWE-Pruner: [GitHub](https://github.com/Ayanami1314/swe-pruner)
- RECOMP: [arXiv:2310.04408](https://arxiv.org/abs/2310.04408)
- LongLLMLingua: [arXiv:2310.06839](https://arxiv.org/abs/2310.06839)

### SWE-bench agent architectures

- SWE-agent: [arXiv:2405.15793](https://arxiv.org/abs/2405.15793)
- Mini-SWE-agent: [GitHub](https://github.com/SWE-agent/mini-swe-agent)
- AutoCodeRover: [arXiv:2404.05427](https://arxiv.org/abs/2404.05427)
- Aider repo map: [Docs](https://aider.chat/docs/repomap.html)
- OpenHands: [arXiv:2407.16741](https://arxiv.org/abs/2407.16741)
- Moatless / SWE-Search: [arXiv:2410.20285](https://arxiv.org/html/2410.20285v1)
- Cognition SWE-grep: [Blog](https://cognition.ai/blog/swe-grep)

### Error classification and fault localization

- Defects4J Anatomy: [arXiv:2502.02299](https://arxiv.org/abs/2502.02299)
- AgentFL: [arXiv:2403.16362](https://arxiv.org/abs/2403.16362)
- LLM4FL: [OpenReview](https://openreview.net/forum?id=z91EvZbSI1)
- SBEST: [arXiv:2405.00565](https://arxiv.org/abs/2405.00565)
- BugsInPy: [ACM DL](https://dl.acm.org/doi/abs/10.1145/3368089.3417943)
- Sentry Autofix: [Blog](https://blog.sentry.io/ai-powered-updates-issue-grouping-autofix-anomaly-detection-and-more/)
- Context-aware prompting for repair: [Springer](https://link.springer.com/article/10.1007/s10515-025-00512-w)
