# Adaptive Online Policy: Priority Proposals (2026-02-26)

> **Date:** 2026-02-26 | **Status:** Complete

## Scope
This document captures the approved proposal set before implementation:
- (1) Failure observability hardening
- (2) Force-helper escape hatch
- (3) Dense intermediate reward
- (4) Adaptive exploration schedule
- (6) Patch sanitation guard

Deferred for now:
- (5) Async-specific retry critic
- (7) Paired A/B plus seeded evaluation protocol

## 2026-02-26 Addendum (Run-specific follow-up)
For the latest run triage, we approved implementation of:
- subprocess-aware source-scope inference
- disabling forced marker patches on transport/context failures
- patch sanitation guard before syntax preflight

## Implementation Status (2026-02-26)
- `Implemented`: subprocess-aware source-scope inference
- `Implemented`: disabling forced marker patches on transport/context failures
- `Implemented`: patch sanitation guard before syntax preflight
- `Deferred`: adaptive prompt compaction for large ConDefects prompts
  - Rationale: likely partially addressable with a larger model context window; revisit if failures persist.

## 1) Failure Observability Hardening (P0)

### Problem
Current runs can log only hashes for prompt and evidence (`record_policy: hash`). This blocks root-cause analysis when we need to inspect exact prompt, helper output, and patch repair loop behavior.

### Proposal
Add a failure-first observability mode that persists full artifacts for failed attempts:
- Full prompt text (`prompt.txt`) and evidence text (`evidence.txt`) for failed attempts only.
- Helper decision payload and helper response payload.
- Initial patch and all repair-cycle patches (`patch.diff`, `patch_cycle_*.diff`).
- Patch verifier diagnostics and syntax preflight diagnostics in structured JSON per attempt.
- Optional redact pass for API keys and obvious secrets before writing files.

### CLI and config sketch
- `--record-policy failed_full` (new)
- `--record-policy success_hash_failed_full` (new default candidate)
- `--record-redact-secrets` (default `on`)

### Acceptance criteria
- For any failed attempt, a single folder contains all fields needed to reproduce triage.
- No behavior change in patch generation or scoring.
- Storage growth remains bounded by writing full content only for failures.

## 2) Force-Helper Escape Hatch (P0)

### Problem
Policy can over-skip helper usage on hard or repeated failures, especially after no-progress streaks.

### Proposal
Add deterministic override rules that force helper invocation before the online action is applied:
- Trigger A: same failure signature repeats for `N` attempts.
- Trigger B: async + hard case at attempt >= 2.
- Trigger C: previous attempt ended with `syntax_preflight_failed`.

When any trigger fires:
- Override selected action to `invoke_helper`.
- Log override reason in metadata and policy event stream.

### CLI and config sketch
- `--adaptive-online-escape-hatch-enabled` (default `on`)
- `--adaptive-online-escape-repeat-threshold 2`
- `--adaptive-online-escape-async-hard-attempt 2`
- `--adaptive-online-escape-on-syntax-fail` (default `on`)

### Acceptance criteria
- Overrides are transparent in event logs (`adaptive_online_override_reason`).
- Helper calls increase only on targeted failure patterns, not globally.

## 3) Dense Intermediate Reward (P1)

### Problem
Sparse terminal reward slows learning and does not credit partial progress.

### Proposal
Keep the existing terminal reward and add a dense progress term:

`reward = success`
`       + w_progress * normalized_test_delta`
`       - lambda_tokens * token_cost`
`       - mu_latency * latency_cost`

Where:
- `normalized_test_delta = clamp((failing_before - failing_after) / max(1, failing_before), -1, 1)`
- If tests did not run, progress term is 0.

Optional shaping penalties:
- Small penalty for `patch_verifier_rejected`.
- Small penalty for `syntax_preflight_failed`.

### CLI and config sketch
- `--adaptive-reward-w-progress 0.35`
- `--adaptive-reward-penalty-patch-verifier 0.10`
- `--adaptive-reward-penalty-syntax-preflight 0.10`

### Acceptance criteria
- Reward decomposition fields are logged (terminal, progress, token, latency, penalties).
- Learning converges faster on mixed-difficulty batches in shadow and on modes.

## 4) Adaptive Exploration Schedule (P1)

### Problem
Fixed epsilon is brittle with heterogeneous case distributions.

### Proposal
Move from global fixed epsilon to bucket-aware epsilon:
- Define case buckets from available metadata (category, difficulty, snapshot_dependency, bug_source).
- Maintain per-bucket decision counts.
- Use higher epsilon for low-visit buckets and decay toward floor for high-visit buckets.

Suggested formula:
- `epsilon(bucket) = max(epsilon_min, epsilon_base / sqrt(1 + visits(bucket)))`

### CLI and config sketch
- `--adaptive-online-epsilon-mode bucket_decay` (default candidate)
- `--adaptive-online-epsilon-base 0.12`
- `--adaptive-online-epsilon-min 0.03`
- `--adaptive-online-epsilon-bucket-keys category,difficulty`

### Acceptance criteria
- Event log includes both configured and effective epsilon.
- Rare bucket coverage increases without causing global helper-call inflation.

## 6) Patch Sanitation Guard (P1)

### Problem
A meaningful slice of failures comes from malformed diff text (literal `\\n`, broken quoting, indentation drift), not true reasoning errors.

### Proposal
Insert a sanitation pass before patch apply:
- Detect suspicious patterns in added/replaced lines:
  - Literal escaped newlines and tabs in code blocks (`\\n`, `\\t`)
  - Unterminated quote artifacts from escaped text conversion
  - Obvious indentation corruption in Python blocks
- Attempt safe rewrites for narrow, high-confidence cases.
- If unsafe to auto-repair, reject with explicit sanitation reason and request corrected diff.

### CLI and config sketch
- `--patch-sanitize-enabled` (default `on`)
- `--patch-sanitize-autofix` (default `on`)
- `--patch-sanitize-strict` (default `off`)

### Acceptance criteria
- Reduction in `syntax_preflight_failed` with no increase in wrong-but-compiling patches.
- Sanitation actions are recorded (`patch_sanitize_applied`, `patch_sanitize_rules`).

## Rollout plan
1. Implement observability and sanitation first.
2. Add escape hatch and dense reward in shadow mode.
3. Enable adaptive epsilon after validating event telemetry fields.
4. Promote to `mode=on` once metrics remain stable across at least 3 short runs.
