# Session Intelligence Findings — What & Why Summary

**Source:** 8 documents from `18-session-intelligence/findings/` + [Anthropic's "Measuring Agent Autonomy" research (2026)](https://www.anthropic.com/research/measuring-agent-autonomy)  
**Date:** 2026-02-18

---

## From Anthropic's Agent Autonomy Research

### A. Autonomy Progression (Turn Duration Growth)

**What:** Track how long the AI agent works autonomously per turn before stopping. Anthropic found the 99.9th percentile turn duration nearly doubled from ~25 min to ~45 min over 3 months, while the median stayed stable at ~45 seconds.

**Why:**
- Reveals the **deployment overhang** — the gap between what the AI is capable of handling and what it actually exercises in practice. If your developers' sessions are consistently short, they may be under-utilizing agent capabilities.
- Tracking tail percentiles (p99, p99.9) separately from median exposes power-user behavior vs. typical usage — two very different populations with different needs.
- A steadily growing tail without model upgrades suggests growing developer trust, not just better models.

---

### B. Auto-Approve Rate by Developer Experience

**What:** Track what % of sessions a developer runs with full auto-approve (no manual action approval). Anthropic found this rises from ~20% for new users (<50 sessions) to >40% by 750 sessions.

**Why:**
- Directly measures **trust accumulation** — a developer granting more autonomy over time means the tool is proving itself.
- Enables segmentation: new users behaving like experienced users (high auto-approve early) may indicate either overconfidence or high skill — correlate with session success to find out.
- If auto-approve rate plateaus or drops for a team, it signals trust erosion — something is going wrong with AI output quality.

---

### C. Human Interrupt Rate

**What:** Track how often developers interrupt the AI mid-execution (per-turn basis). Anthropic found this *increases* with experience — from ~5% of turns for new users to ~9% for experienced users.

**Why:**
- Counterintuitive but critical: more interrupts + more auto-approve = **shift from approval-based oversight to monitoring-based oversight**. Experienced developers let the AI run but actively watch and redirect.
- A developer who auto-approves but never interrupts may not be monitoring at all — that's a risk signal.
- Interrupt rate by session complexity reveals whether developers are appropriately engaged on hard tasks vs. coasting on easy ones.

---

### D. Agent-Initiated Stops (Clarification Requests)

**What:** Track how often the AI itself pauses to ask the developer a question instead of proceeding. Anthropic found Claude asks for clarification >2× as often on complex tasks vs. simple ones, and more frequently than humans interrupt it.

**Why:**
- Measures **AI uncertainty calibration** — a well-calibrated agent asks when it should and proceeds when it can.
- If the AI never asks questions on complex tasks, it's likely hallucinating or guessing. If it asks too much on simple tasks, it's wasting developer time.
- The ratio of agent-stops to human-interrupts reveals who is driving oversight — the AI (good calibration) or the human (compensating for poor calibration).

**Top reasons the AI stops itself (from Anthropic's data):**
| Reason | Frequency |
|--------|-----------|
| Present choice between approaches | 35% |
| Gather diagnostic info / test results | 21% |
| Clarify vague/incomplete requests | 13% |
| Request missing credentials/access | 12% |
| Get approval before taking action | 11% |

**Top reasons humans interrupt:**
| Reason | Frequency |
|--------|-----------|
| Provide missing technical context | 32% |
| AI was slow, hanging, or excessive | 17% |
| Got enough help, proceeding independently | 7% |
| Want to take next step themselves | 7% |
| Change requirements mid-task | 5% |

---

### E. Human Involvement vs. Task Complexity

**What:** Measure human involvement per tool call, segmented by task complexity. Anthropic found 87% of tool calls on minimal-complexity tasks have human involvement vs. only 67% on high-complexity tasks.

**Why:**
- Reveals that step-by-step approval becomes structurally impractical as tasks grow — developers *must* shift to supervisory oversight for complex work.
- If your data shows the opposite (more involvement on complex tasks), it may indicate developers don't trust the AI on hard problems — an intervention opportunity.
- Helps calibrate when CLAUDE.md guidance matters most: complex tasks with low human involvement are where good repo context prevents the most waste.

---

### F. Deployment Overhang

**What:** Measure the gap between what the AI can handle autonomously and what developers actually let it do. Anthropic found success rates doubled on hard tasks while interventions dropped from 5.4 to 3.3 per session — the AI could have been trusted sooner.

**Why:**
- Quantifies unrealized productivity — if developers are over-supervising capable sessions, they're spending time on approvals that aren't adding value.
- Tracking this over time shows whether your org is capturing AI capability gains or leaving them on the table.
- A shrinking overhang means your developers are calibrating well; a persistent overhang means trust-building interventions (like CLAUDE.md with proven patterns) are needed.

---

### G. Safeguard & Reversibility Classification

**What:** Classify each tool call by: (1) whether safeguards exist (restricted permissions, human approval), (2) whether a human is in the loop, (3) whether the action is irreversible. Anthropic found 80% have safeguards, 73% have a human in the loop, only 0.8% are irreversible.

**Why:**
- Maps the actual risk surface of your AI sessions — most actions are low-stakes and reversible, which justifies granting more autonomy for those categories.
- The 0.8% irreversible actions (deployments, external API calls, emails) are where oversight effort should concentrate — not spread equally across all actions.
- Enables risk-appropriate autonomy policies: auto-approve file reads and edits, require approval for deployments and external calls.

---

### H. Risk × Autonomy Quadrant Mapping

**What:** Plot every type of agent action on a 2D grid of risk (how consequential) vs. autonomy (how independently the AI acts). Anthropic found the high-risk + high-autonomy quadrant is sparsely populated but not empty.

**Why:**
- Identifies the dangerous corner — actions that are both high-stakes and unsupervised. Even if rare, a single error there can be significant.
- Enables targeted policy: you don't need blanket rules, you need rules for the specific action types that land in the upper-right quadrant.
- Tracking this over time reveals whether autonomy is creeping into high-risk areas as developers get comfortable — an early warning system.

---

### I. User-to-Assistant Message Ratio & Interrupt Timing

**What:** Measure the ratio of user messages to assistant messages per session, and *when* in the session the user messages cluster (early, mid, late).

- A low ratio (e.g., 1:5) means the AI ran long autonomous stretches — the developer set it loose.
- A high ratio (closer to 1:1) means tight conversational back-and-forth — the developer steered heavily.

**Why:**
- **Oversight style proxy** — maps directly to Anthropic's approval-vs-monitoring finding. Experienced developers tend toward lower ratios (more autonomy), newer developers toward higher ratios (more steering).
- **Session quality signal** — cross with trouble score to answer: "Do sessions where developers intervene more often turn out better?" (Anthropic's data says yes for complex tasks).
- **Task complexity indicator** — simple tasks naturally produce fewer user messages. A high ratio on a supposedly simple task suggests the AI is struggling.
- **Developer experience proxy** — session count per developer + their typical ratio reveals trust calibration without needing account tenure metadata.

**The real power is in interrupt timing — not just the ratio, but *where* user messages cluster:**

| Pattern | Likely Interpretation |
|---------|----------------------|
| Late intervention + error cascade right before | AI got stuck, developer rescued a failing session |
| Late intervention + many explore calls, no edits | Scope creep — context got bloated with irrelevant files |
| Late intervention + high token count + repetitive tool calls | Context corruption — AI looping on stale/wrong information |
| Late intervention + clean session up to that point | Developer changed requirements mid-task |
| Early intervention + successful session | Good course-correction upfront, developer set the AI on the right track |

No single signal pinpoints the root cause, but ratio + timing + the 5 messages before the interrupt creates a **pattern fingerprint**. You can't say "this session had context corruption" with certainty, but you can say "this matches the context corruption pattern seen in 80% of similar cases" — and that's actionable enough to surface in a digest or post-session report.

---

### J. Session Topology — Multi-Session & Cross-Repo Attention Patterns

**What:** Track the temporal clustering of sessions per developer — how sessions relate to each other in time and across repos. Detect three distinct topologies:

**1. Same repo, rapid sequential sessions** (gap < N minutes between session end → next session start)

**What:** Developer finishes (or abandons) a session on a repo and starts a new one on the same repo shortly after.

**Why:**
- If the previous session ended with a high trouble score or was abandoned → **context recovery**. The developer learned the context got corrupted or bloated and is restarting fresh with lessons from the failed attempt.
- If the previous session ended cleanly → **deliberate decomposition**. The developer is breaking work into scoped chunks ("first session for the fix, second for the tests") — this is actually a sophisticated strategy.
- A developer who *consistently* runs short sequential sessions rather than long ones may have discovered that short-context sessions outperform long ones for their codebase — that's a pattern worth detecting and potentially recommending to others.
- Repeated failed sequential sessions on the same repo → the problem is genuinely hard and may need human-only thinking time or a different approach entirely.

**2. Same repo, overlapping/parallel sessions**

**What:** Developer has two or more active sessions on the same repo running at the same time.

**Why:**
- Signals the developer is attacking the same problem from two angles — trying approach A in one session while exploring approach B in another.
- Or: one session is running autonomously (refactoring, running tests) while the developer uses the other to explore/read code. This is healthy parallelization.
- If both sessions succeed → efficient parallel exploration. If both fail → the problem may need to be escalated or re-scoped.
- Rare pattern worth flagging simply because it reveals high-stakes problems the developer is investing heavily in.

**3. Cross-repo, overlapping sessions**

**What:** Developer has active sessions on two or more different repos simultaneously.

**Why — the key discriminator is interactivity level in each session:**

| Both sessions interactive (high user msg ratio) | **Attention fragmentation** — developer is context-switching between two unrelated problems, neither gets full focus. Correlate with success rate: if both sessions have elevated trouble scores, the split attention is likely hurting outcomes. |
|---|---|
| One autonomous, one interactive | **Healthy delegation** — developer kicked off a long-running autonomous task on repo A and is actively working repo B while it runs. This is the ideal power-user pattern. |
| Both autonomous | **Orchestration mode** — developer is managing multiple agents like a manager dispatching workers. Correlate with success: if outcomes are good, this developer's workflow is worth studying. |

**Composite signal:** Regardless of the specific topology, multiple concurrent sessions = developer attention is divided across problems. Whether that's productive or destructive depends entirely on the autonomy level of each session. The L1 signal is simply: "this developer's attention is on more than one problem right now." Cross it with per-session interactivity and success rate to determine if it's smart delegation or harmful fragmentation.

---

## From Internal Research

## 1. Prompt Classification & Risk Scoring

**What:** Classify every developer prompt into buckets (error_paste_and_fix, code_review, detailed_spec, general, exploration, multi_task, vague_directive, continuation) and assign a risk score based on historical trouble scores.

**Why:**
- Predicts session outcome *before* the AI even runs — multi_task prompts average 8.67 trouble score vs 0.25 for error_paste_and_fix.
- Enables pre-flight warnings: a developer sending a vague directive (17.0 avg trouble) can be nudged toward a specific prompt pattern.
- Provides a leading indicator — prompt quality predicts session success, so tracking bucket distribution over time shows whether teams are learning better prompting habits.

---

## 2. Trouble Score

**What:** Composite metric per session: `error_cascades×3 + interrupts×2 + undo_requests×3`. Higher = worse session.

**Why:**
- Single number that captures session health without subjective judgment.
- Enables before/after comparison when interventions (e.g., CLAUDE.md) are introduced — target is −20% reduction in mean trouble score.
- Segments sessions into risk tiers: score >10 flags sessions that wasted significant developer time.

---

## 3. Session Success Rate

**What:** % of AI sessions that complete successfully (has edits + low trouble score + not abandoned). The recommended **north star metric**.

**Why:**
- Directly measures the outcome developers care about — did the AI actually help ship something?
- Comparable across repos, teams, and time periods.
- Makes ROI tangible: improving from 61% → 74% translates to ~65 fewer failed sessions/week for a 50-developer org, saving ~$3,250/week.

---

## 4. Error Cascade Detection

**What:** Detect 3+ consecutive shell errors within a session.

**Why:**
- Signals the AI is stuck in a retry loop — it keeps trying the same failing approach with minor adjustments instead of rethinking.
- These cascades waste tokens, time, and developer patience. Early detection enables rescue interventions (suggest resetting context or providing correct paths).
- Quantifies a specific failure mode that CLAUDE.md guidelines can reduce.

---

## 5. Null Spiral Detection

**What:** Detect 3+ consecutive null/empty tool results from MCP or tool integrations.

**Why:**
- Indicates broken tool integration (crashed MCP server, dead DB connection) — not a prompting problem but an infrastructure problem.
- Without detection, developers waste an entire session before realizing the tooling is down.
- High-confidence signal with near-zero false positives — always worth alerting on.

---

## 6. Scope Creep Detection

**What:** Detect >10 explore/read calls with zero edit calls in a session.

**Why:**
- Reveals sessions where the AI is reading the entire codebase without making progress — the developer's prompt was too broad.
- Actionable: nudging toward specific file references ("Update User model in src/models/user.ts" instead of "Fix the user system") directly resolves the pattern.
- Tracks whether CLAUDE.md repo-specific context is reducing aimless exploration.

---

## 7. Hallucination Detection

**What:** Detect tool calls referencing files/paths that don't exist ("No such file", "does not exist").

**Why:**
- AI hallucinating file paths wastes tool calls and developer trust.
- Quantifies how well the AI "knows" the codebase — repos with good CLAUDE.md context should have fewer path hallucinations.
- Trackable over time as a proxy for context quality improvement.

---

## 8. Abandonment Rate

**What:** % of sessions abandoned (no edits produced, session >30 min or explicitly stopped).

**Why:**
- Directly measures wasted developer time — an abandoned session is pure loss.
- Complementary to success rate: a session can "not succeed" quietly, but abandonment means the developer gave up.
- ROI-friendly metric for managers: 145 sessions/month × 13% abandonment reduction = 9.5 hours/month saved.

---

## 9. Time to First Success

**What:** Minutes from session start to first successful edit.

**Why:**
- Measures how quickly the AI becomes productive — shorter = less idle waiting.
- Directly translatable to dollar savings: 50 sessions/week × 6 min saved = 5 hours/week = ~$25K/year.
- Surfaces whether CLAUDE.md context helps the AI "get going" faster by reducing initial exploration.

---

## 10. Pattern Adoption Tracking

**What:** Track shift in prompt bucket distribution over time (e.g., % of error_paste_and_fix prompts rising, % of multi_task prompts falling).

**Why:**
- Leading indicator of behavior change — if developers are shifting toward proven patterns, outcomes will follow.
- Measures whether CLAUDE.md guidelines are actually influencing developer behavior.
- Targets: +15pp good patterns, −10pp bad patterns.

---

## 11. Developer Satisfaction (NPS)

**What:** Survey-based: "How satisfied with AI sessions?" (1-10), calculated as NPS.

**Why:**
- Adoption depends on satisfaction — developers who don't like the tool will stop using it or use it poorly.
- Guards against the "technically improved but hated" failure mode — research shows AI usage is at 84% but favorability dropped from 70% to 60%.
- Target: NPS >50, >70% satisfaction.

---

## 12. Causal Measurement (RCT / Diff-in-Diff / CausalImpact)

**What:** Three statistical methods to prove whether an intervention (e.g., CLAUDE.md) *caused* improvement, not just correlated with it.

| Method | Use When |
|--------|----------|
| **RCT** | 20+ repos, need bulletproof proof — randomly assign treatment/control |
| **Diff-in-Diff** | Can't withhold treatment — compare natural adopters vs non-adopters, before and after |
| **CausalImpact** | Single repo pilot — use pre-intervention data to predict counterfactual |

**Why:**
- Simple before/after comparison is misleading — better developers adopt first (selection bias), teams improve over time regardless (maturation), tasks may differ in difficulty.
- Without causal evidence, leadership won't fund scaling and you can't distinguish real impact from noise.
- The METR study (July 2025) proved this approach works for measuring AI tool effectiveness.

---

## 13. CLAUDE.md as Delivery Mechanism

**What:** Auto-generated repo file containing data-driven AI collaboration guidelines (effective patterns, anti-patterns, troubleshooting, repo context). AI reads it automatically at session start.

**Why:**
- Zero friction — developer commits once, every session benefits automatically, no ongoing action.
- Solves the "insights exist but nobody uses them" problem — dashboards go unread, emails get buried, real-time alerts feel like surveillance.
- Measurable: repos with CLAUDE.md show 74% success rate vs 61% without (from 318-session analysis).
- Fits developer psychology: invisible by default, respects autonomy, doesn't feel like monitoring.

---

## 14. Developer Psychology Guardrails

**What:** Research-backed constraints on how interventions should be designed — derived from 50+ papers and 49K-developer surveys.

**Key principles:**
| Principle | Insight |
|-----------|---------|
| **Flow state is sacred** | Interrupting flow destroys 15-30 min of productivity; only alert for critical failures |
| **Autonomy is non-negotiable** | Suggestions OK, mandates rejected; developers must be able to ignore/dismiss |
| **Surveillance kills adoption** | Per-developer dashboards, rankings, manager reports → instant rejection |
| **Privacy first** | All insights private by default; developer controls what to share |
| **Rescue framing** | "Help when stuck" accepted; "monitor and correct" rejected |

**Why:**
- 84% of developers use AI but only 60% view it favorably (down from 70%) — the trust gap is real.
- Tools that feel like surveillance get deleted; tools that feel like help get championed.
- Every metric and intervention above must pass through this filter or it will fail at adoption regardless of technical quality.

---

## Signal Availability by Source

Not all metrics work across all tools. This matrix shows what each source actually gives us — **don't report a metric for a source that can't reliably produce it**.

### Why this matters

**Update (Feb 20, 2026):** Cursor's `.txt` agent transcripts still strip tool results and lack timestamps, but the `cursorDiskKV` table in `state.vscdb` contains both — with caveats on coverage. See [schema docs](../../../schemas/cursor/README.md) for full details.

The signal availability depends on **which storage mode** the session uses and how new it is:

| Storage mode | Sessions (sample) | Has timestamps | Has tool results | Has full message text |
|---|---|---|---|---|
| **Inline conversation** | ~1,691 | ✅ `timingInfo` on assistant msgs | ❌ | ✅ |
| **Agent hash chain** (`conversationState`) + v3 bubbleIds | ~17 | ✅ `createdAt` on all msgs | ✅ via `agentKv` `role:tool` | ✅ **Full signal** |
| **Agent hash chain** (`conversationState`) older | ~91 | ❌ | ✅ via `agentKv` `role:tool` | ✅ |
| **v3 bubbleIds only** (no conversationState) | ~13 | ✅ `createdAt` on all msgs | ❌ | ✅ |
| **Headers-only** | ~282 | ❌ | ❌ | Via `bubbleId:*` lookup |
| **Plain text transcripts** (fallback / remote SSH) | all | ❌ | ❌ | ✅ (but no tool result content) |

**~17 sessions currently have both timestamps AND tool results** — these are the newest agent sessions (v3 `bubbleId` entries with `createdAt` + `conversationState` hash chain to `agentKv`). This overlap will likely grow as Cursor updates. For the remaining sessions, timestamps and tool results come from different subsets.

Cross-source comparisons remain risky. If you compare "error cascade rate: 15% on Claude Code vs 2% on Cursor," the difference may still reflect data gaps rather than healthier sessions — just a narrower gap than before.

### The matrix

| Metric | Claude Code | Codex CLI | Cursor (transcripts only) | Cursor (+ cursorDiskKV) | Notes |
|--------|:-----------:|:---------:|:-------------------------:|:-----------------------:|-------|
| **Prompt classification** | ✅ | ✅ | ✅ | ✅ | User messages captured in all modes |
| **Trouble score** | ✅ | ✅ | ⚠️ Partial | ⚠️ Better | Error cascades detectable on ~108 agent hash chain sessions via `agentKv` tool results; still blind on ~1,691 inline sessions |
| **Session success rate** | ✅ | ✅ | ⚠️ Partial | ⚠️ Better | Tool result analysis possible for agent sessions; `totalLinesAdded/Removed` still best proxy for inline sessions |
| **Error cascade detection** | ✅ | ✅ | ❌ | ⚠️ ~108 sessions | `agentKv` entries with `role:tool` contain full stdout, exit codes, `isError` — but only for sessions using `conversationState` hash chain |
| **Null spiral detection** | ✅ | ✅ | ❌ | ⚠️ ~108 sessions | Same — can inspect tool result content for null/empty via `agentKv` |
| **Scope creep detection** | ✅ | ✅ | ✅ | ✅ | Only needs tool call names, captured in all modes |
| **Hallucination detection** | ✅ | ✅ | ❌ | ⚠️ ~108 sessions | Can check "No such file" in `agentKv` tool results |
| **Abandonment rate** | ✅ | ✅ | ⚠️ Partial | ✅ | `timingInfo` gives precise timing on ~1,691 inline sessions; `createdAt`/`lastUpdatedAt` on all |
| **Time to first success** | ✅ | ✅ | ❌ | ⚠️ ~1,691 sessions | `timingInfo.clientStartTime` on assistant messages enables this for inline conversations |
| **User-to-assistant msg ratio** | ✅ | ✅ | ✅ | ✅ | Message sequence preserved in all modes |
| **Interrupt timing** | ✅ | ✅ | ❌ | ⚠️ ~1,691 sessions | `timingInfo` gives wall-clock times; user message time inferred from gap between assistant responses |
| **Turn duration** | ✅ | ✅ | ❌ | ⚠️ ~1,691 sessions | `clientEndTime - clientStartTime` per assistant turn |
| **Session topology (J)** | ✅ | ✅ | ⚠️ Partial | ✅ | `timingInfo` + `createdAt`/`lastUpdatedAt` gives precise overlap and gap detection |
| **Auto-approve rate** | ✅ | ✅ | ❌ | ❌ | Still not exposed in Cursor's data model |
| **Agent-initiated stops** | ✅ | ✅ | ✅ | ✅ | Visible in transcripts and conversation data |
| **Pattern adoption tracking** | ✅ | ✅ | ✅ | ✅ | Derived from prompt classification |

### What Cursor *does* give you that others don't

| Signal | Source | Value |
|--------|--------|-------|
| `totalLinesAdded` / `totalLinesRemoved` | `composer.composerData` in SQLite | Direct code output volume per session |
| `contextUsagePercent` | `composer.composerData` | How much of the context window was consumed — proxy for session complexity |
| `composerSuggestedLines` / `composerAcceptedLines` | `aiCodeTracking.dailyStats` | Acceptance rate of AI suggestions — a different angle on trust/quality |
| `unifiedMode` (agent / chat / edit) | `composer.composerData` | Which interaction mode the developer chose — segments sessions by intent |
| Code attribution via `ai_code_hashes` | `ai-code-tracking.db` | Links specific code to specific sessions — can trace which AI session produced which commit |
| Per-message token counts | `cursorDiskKV:bubbleId:*` → `tokenCount` | `inputTokens` / `outputTokens` per message — enables cost tracking and context window analysis |
| Per-file token breakdown | `cursorDiskKV:composerData:*` → `tokenDetailsUpUntilHere` | Token count per attached file — reveals which files dominate context |
| Response latency | `cursorDiskKV:composerData:*` → `timingInfo` | `clientStartTime` → `clientEndTime` per response — ms precision |
| Full tool results | `cursorDiskKV:agentKv:blob:*` → `role:tool` | stdout, exit codes, `isError`, structured output (for ~108 agent sessions) |
| File state checkpoints | `cursorDiskKV:checkpointId:*` | File snapshots between agent steps — enables step-by-step replay |
| Line-level diffs | `cursorDiskKV:codeBlockDiff:*` | Exact line ranges modified per code block |

### Bottom line

**Claude Code and Codex still give the most complete signal across all metrics.** Cursor's gap has narrowed significantly: `cursorDiskKV` provides timestamps (~1,691+ sessions), tool results (~108 sessions), token counts, and per-file context breakdowns. **~17 newer agent sessions provide full signal** (both timestamps and tool results), and this number will grow with Cursor updates. The `.txt` transcripts remain the only source for remote SSH sessions (no cursorDiskKV access on remote targets).

**Practical guidance:** Extract from `cursorDiskKV` first (richest), fall back to `.txt` transcripts for remote sessions. When aggregating cross-source, note which Cursor sessions contributed to each metric and at what coverage level. For the ~17 "full signal" sessions, Cursor data quality approaches Claude Code/Codex parity.

---

## Quick Reference: Metric Tiers

```
Tier 1 — Primary (Session Outcomes)
├── Session success rate (north star)
├── Trouble score distribution
└── Pattern adoption rate (leading indicator)

Tier 2 — Manager-Friendly (ROI & Waste)
├── Time to first success
├── Abandonment rate
├── Developer satisfaction (NPS)
└── Dollar ROI

Tier 3 — Diagnostic (Root Cause)
├── Error cascade frequency
├── Null spiral frequency
├── Scope creep frequency
├── Hallucination frequency
└── Prompt bucket distribution
```
