# Consolidated Strategy: Session Intelligence System

**Date:** 2026-02-18  
**Purpose:** Actionable roadmap combining technical feasibility with developer psychology research  
**Status:** Ready for implementation

---

## TL;DR — What We Learned

### From Codebase Analysis:
- ✅ **Infrastructure ready:** 5-10s latency from session event → DB
- ✅ **Algorithms proven:** Error cascade detection, prompt classification already work
- ✅ **Data available:** 318 sessions analyzed, patterns validated
- ✅ **Architecture exists:** Daemon→Server→DB pipeline can support real-time

### From Deep Research (50+ papers, 49k developer survey):
- ⚠️ **Trust crisis:** 84% use AI, but only 60% trust it (down from 70%)
- ⚠️ **Surveillance sensitivity:** Developers reject tools that feel like monitoring
- ⚠️ **Flow state is sacred:** Interrupting flow destroys productivity for 30+ min
- ✅ **Help wanted:** Developers want support with context, debugging, getting unstuck
- ✅ **Autonomy is non-negotiable:** Suggestions OK, mandates rejected

### The Winning Formula:
**"Rescue when stuck, invisible when flowing, private always."**

---

## Part 1: The Three Intervention Layers (Recommended Architecture)

### Layer 1: Pre-Flight (Before AI Runs)
**Goal:** Prevent waste  
**Timing:** Developer submits prompt → AI processes it  
**Mechanism:** MCP `check_prompt()` tool

**What to Build:**
```python
# MCP Tool: check_prompt
# Called automatically before AI processes prompt (optional)
# Returns: risk_score + suggestion

def check_prompt(prompt_text: str) -> dict:
    bucket = classify_prompt(prompt_text)  # Use existing classifier
    score = RISK_SCORES[bucket]            # error_paste_and_fix=0.25, multi_task=8.67
    
    if score > 5:
        return {
            "risk_score": score,
            "bucket": bucket,
            "suggestion": get_suggestion(bucket),
            "example": get_example(bucket)
        }
    return {"risk_score": score, "ok": True}
```

**Why This Works (Psychology):**
- Developer stays in control (can ignore)
- Value is immediate (saves time on first use)
- No surveillance feeling (optional, private)
- Builds trust (helps without judging)

**Success Criteria:**
- [ ] Developer can ignore without penalty
- [ ] Suggests in <500ms
- [ ] Explains WHY (data-driven: "multi_task patterns score 8.67")
- [ ] Provides concrete alternative

---

### Layer 2: In-Flight (During Session)
**Goal:** Rescue failing sessions  
**Timing:** Real-time (every 10s check)  
**Mechanism:** MCP notifications + Slack for critical

**Critical Patterns to Detect (High Confidence):**

| Pattern | Detection | Action | Message |
|---------|-----------|--------|---------|
| **Null Spiral** | 3+ null tool results | Immediate alert | "Tool integration broken — check MCP config" |
| **Error Cascade** | 3+ consecutive shell errors | Soft notification | "AI seems stuck in error loop — consider resetting context" |
| **Scope Creep** | >10 explore, 0 edits | Inline suggestion | "AI exploring without acting — try narrowing scope" |
| **Hallucination** | "No such file" after file ref | Flag | "AI referenced non-existent file — verify path" |

**Implementation:**
```python
# SessionMonitor polls DB every 10s
class SessionMonitor:
    def check_session(self, session_id: str) -> list[Alert]:
        recent_messages = get_last_n_messages(session_id, n=20)
        
        alerts = []
        if self.detectors.null_spiral(recent_messages):
            alerts.append(Alert(
                severity="CRITICAL",
                message="Tool integration may be broken",
                action="Check MCP server status"
            ))
        
        if self.detectors.error_cascade(recent_messages):
            alerts.append(Alert(
                severity="HIGH",
                message="AI appears stuck in error loop",
                action="Consider providing correct paths or resetting"
            ))
        
        return alerts
```

**Why This Works (Psychology):**
- "Rescue" framing (help when stuck, not surveillance)
- Only interrupts when session is failing (not during flow)
- Gives developer an action to take (not just criticism)
- Escalation: MCP first (private), Slack only for critical

**Delivery Strategy:**
```
CRITICAL (null spiral): MCP notification + Slack alert
HIGH (error cascade): MCP inline suggestion (dismissible)
MEDIUM (scope creep): Available via MCP session_health() tool
LOW (optimization): Weekly digest only
```

---

### Layer 3: Post-Flight (After Session)
**Goal:** Learning and improvement  
**Timing:** Weekly  
**Mechanism:** Private digest (email/Slack DM)

**What to Include:**
```
📊 Your AI Sessions This Week (Private)

Overview:
• 12 sessions • 8 successful • 2 rescued by alerts
• Avg trouble score: 4.2 (down 15% from last week ✅)

What Worked Well:
• Your "error paste + fix" prompts: 95% success rate
• Good context management: avg 5 files/session

Patterns to Consider:
• 2 sessions had scope creep (AI explored >10 files)
  → Tip: Provide specific file paths upfront

• 1 session hit null spiral (MCP connection issue)
  → Check: database connection settings

This Week's Recommendation:
Try breaking multi-part tasks into separate prompts.
Example: Instead of "fix auth and add tests", do auth first, then tests.
```

**Why This Works (Psychology):**
- Private (not shared with team/managers)
- Reflective, not evaluative (focus on patterns, not blame)
- Celebrates wins (not just problems)
- Actionable (specific tip with example)

---

## Part 2: The Implementation Roadmap

### Phase 0: Foundation (Week 1)
**Goal:** Extract existing algorithms to reusable modules

**Tasks:**
1. **Extract prompt classifier**
   - Source: `analysis-feb2026/qualitative/qualitative_analysis.py:329-353`
   - Target: `qc_trace/intelligence/prompt_scorer.py`
   - Add risk scoring logic

2. **Extract detection algorithms**
   - Source: `analysis-feb2026/analyzers/helpers.py`
   - Target: `qc_trace/intelligence/detectors.py`
   - Functions to extract:
     - `is_shell_error()` (line 287)
     - `has_error_cascade()` (line 243)
     - `categorize_tool()` (line 31)

3. **Set up opt-in flow**
   - Create `developer_preferences` table
   - Fields: `user_email`, `opted_in`, `risk_tolerance`, `notification_prefs`
   - Default: opted_out (trust-building)

**Deliverable:** Reusable intelligence module + consent system

---

### Phase 1: Pre-Flight MVP (Week 2-3)
**Goal:** MCP `check_prompt()` tool

**Build:**
```python
# qc_trace/intelligence/mcp_server.py
from mcp.server.fastmcp import FastMCP

mcp = FastMCP("session-intelligence")

@mcp.tool()
def check_prompt(prompt_text: str) -> dict:
    """Score a prompt before sending to AI."""
    scorer = PromptScorer()
    result = scorer.score(prompt_text)
    
    return {
        "risk_score": result.score,
        "risk_level": result.level,  # low/medium/high
        "bucket": result.bucket,
        "suggestion": result.suggestion if result.score > 5 else None,
        "example": result.example if result.score > 5 else None
    }
```

**Developer Setup:**
```json
// ~/.claude/claude.json
{
  "mcpServers": {
    "session-intelligence": {
      "command": "python",
      "args": ["-m", "qc_trace.intelligence.mcp_server"],
      "env": {
        "QC_TRACE_DSN": "postgresql://..."
      }
    }
  }
}
```

**Testing:**
- Test with 3-5 friendly developers
- Measure: Do they use it? Do they find it helpful?
- Iterate on message framing

**Success Criteria:**
- [ ] Response time <500ms
- [ ] Developer can disable easily
- [ ] Zero false positives in test week
- [ ] Developers report finding it helpful

---

### Phase 2: In-Flight Detection (Week 4-5)
**Goal:** Real-time pattern detection

**Build:**
1. **Session Monitor Service**
   ```python
   # qc_trace/intelligence/monitor.py
   class SessionMonitor:
       def __init__(self):
           self.detectors = Detectors()
           self.db = DBConnection()
       
       async def poll(self):
           """Run every 10 seconds."""
           active_sessions = self.db.get_active_sessions()
           for session in active_sessions:
               alerts = self.check_session(session.id)
               if alerts:
                   await self.deliver_alerts(session.user_email, alerts)
   ```

2. **MCP `session_health()` tool**
   ```python
   @mcp.tool()
   def session_health(session_id: str) -> dict:
       """Get current session health."""
       monitor = SessionMonitor()
       return monitor.check_session(session_id)
   ```

3. **Alert Delivery**
   - MCP notifications (primary)
   - Slack webhooks (critical only, opt-in)

**Testing:**
- Run on 5-10 sessions
- Measure false positive rate
- Tune thresholds based on feedback

**Success Criteria:**
- [ ] Detects null spirals within 30 seconds
- [ ] <10% false positive rate
- [ ] Developers rescue sessions using alerts

---

### Phase 3: Weekly Digest (Week 6-7)
**Goal:** Automated weekly summaries

**Build:**
```python
# qc_trace/intelligence/digest.py
class WeeklyDigest:
    def generate(self, user_email: str, week_start: date) -> dict:
        sessions = self.get_sessions(user_email, week_start)
        
        return {
            "summary": self.generate_summary(sessions),
            "wins": self.identify_wins(sessions),
            "patterns": self.identify_patterns(sessions),
            "recommendation": self.generate_recommendation(sessions)
        }
    
    def send(self, user_email: str, digest: dict):
        # Send via email or Slack DM (developer preference)
        pass
```

**Testing:**
- Generate digests for 3 developers
- Get feedback on tone, content, actionability
- Refine before broader rollout

**Success Criteria:**
- [ ] Open rate >50%
- [ ] Positive sentiment in feedback
- [ ] Developers report taking action on recommendations

---

### Phase 4: Calibration & Scale (Week 8+)
**Goal:** Personalization and A/B testing

**Build:**
1. **Personalization Layer**
   - Track developer risk tolerance from their choices
   - Adjust thresholds per developer
   - Risk-averse: More alerts, gentler suggestions
   - Risk-tolerant: Fewer alerts, only critical

2. **Feedback Loop**
   - After each alert: "Was this helpful? 👍 👎"
   - Use to tune detectors
   - Track which suggestions lead to better outcomes

3. **A/B Testing**
   - Control group: No intervention
   - Treatment group: Full system
   - Measure: trouble_score reduction, satisfaction

**Success Criteria:**
- [ ] 20% reduction in avg trouble_score
- [ ] Developer satisfaction >7/10
- [ ] Voluntary usage (not just mandated)

---

## Part 3: The Psychology-First Messaging Guide

### The Core Principle: **"Help, Don't Judge"**

**Always Frame As:**
- "This pattern tends to..." (data-driven)
- "Consider..." (suggestion)
- "Want to see an example?" (offer help)
- "Your sessions..." (ownership)

**Never Frame As:**
- "You should..." (paternalistic)
- "You're doing it wrong" (judgmental)
- "Best practice is..." (prescriptive)
- "Other developers..." (comparative)

### Message Templates by Pattern

**Multi-Task Prompt Detected:**
```
💡 Pattern Notice

This looks like a multi-task prompt (risk score: 8.7/10).
Multi-task prompts often produce partial solutions.

Consider breaking into separate prompts:
❌ "Fix auth and add tests and update docs"
✅ "Fix auth issue in login.ts"
✅ "Add tests for auth fix" (after first completes)

[Show me how] [Dismiss] [Don't show again]
```

**Error Cascade Detected:**
```
🔍 Session Health Alert

The AI has hit 3 consecutive errors:
• File not found: src/utils/helpers.ts
• Command failed: npm test  
• Permission denied: ./deploy.sh

It might be stuck. Consider:
• Providing the correct file paths
• Resetting context with /reset
• Breaking the task into smaller steps

[View session details] [Reset context] [Dismiss]
```

**Null Spiral Detected:**
```
⚠️ Tool Integration Issue

Your MCP server has returned empty results 3 times.
This usually means the tool is misconfigured.

Quick checks:
• Is the database connection alive?
• Did the MCP server crash?
• Check logs: mcp-server.log

[Check tool status] [View logs] [Continue anyway]
```

**Weekly Digest:**
```
📊 Your Week in AI Sessions

Hi [Name],

Here's what we noticed about your AI sessions this week:

🎯 Wins:
• 12 sessions completed
• 85% success rate (up 10% from last week!)
• Great use of specific file references

💡 One thing to try:
You had 2 sessions where the AI explored many files
without making edits. This often means the scope
was too broad.

Next time, try: "Update the User model in src/models/user.ts"
Instead of: "Fix the user system"

[View full details] [Change digest settings]
```

---

## Part 4: The Data Model Additions

### New Tables Needed

```sql
-- Developer preferences and consent
CREATE TABLE developer_preferences (
    user_email VARCHAR(255) PRIMARY KEY,
    opted_in BOOLEAN DEFAULT FALSE,
    risk_tolerance VARCHAR(20) DEFAULT 'medium', -- low/medium/high
    notification_prefs JSONB DEFAULT '{"mcp": true, "slack": false, "email": false}',
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW()
);

-- Real-time alerts log
CREATE TABLE session_alerts (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    session_id UUID REFERENCES sessions(id),
    user_email VARCHAR(255),
    pattern_type VARCHAR(50), -- null_spiral, error_cascade, etc.
    severity VARCHAR(20), -- critical/high/medium/low
    message TEXT,
    sent_at TIMESTAMP DEFAULT NOW(),
    acknowledged_at TIMESTAMP,
    was_helpful BOOLEAN -- developer feedback
);

-- Weekly digest tracking
CREATE TABLE weekly_digests (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_email VARCHAR(255),
    week_start DATE,
    digest_data JSONB,
    sent_at TIMESTAMP,
    opened_at TIMESTAMP,
    clicked_at TIMESTAMP
);

-- Intervention effectiveness
CREATE TABLE intervention_outcomes (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    alert_id UUID REFERENCES session_alerts(id),
    session_id UUID REFERENCES sessions(id),
    intervention_type VARCHAR(50),
    session_rescued BOOLEAN, -- Did the session complete successfully after alert?
    developer_feedback TEXT,
    created_at TIMESTAMP DEFAULT NOW()
);
```

---

## Part 5: Risk Mitigation & Anti-Patterns

### The Surveillance Trap (AVOID)

**What NOT to do:**
- ❌ Team dashboards showing per-developer metrics
- ❌ "You ranked #3 of 10 this week"
- ❌ Reports to managers without developer consent
- ❌ Comparisons between developers
- ❌ Tracking "time spent coding"

**What TO do:**
- ✅ Private insights only
- ✅ Developer controls what to share
- ✅ Focus on self-improvement, not competition
- ✅ Opt-in by default

### The Interruption Trap (AVOID)

**What NOT to do:**
- ❌ Popups during coding
- ❌ Real-time suggestions on every prompt
- ❌ Alerts for minor optimization opportunities
- ❌ Cannot be disabled

**What TO do:**
- ✅ Only interrupt for critical issues
- ✅ Inline suggestions (not modal dialogs)
- ✅ Easy to dismiss
- ✅ Respects flow state

### The Paternalism Trap (AVOID)

**What NOT to do:**
- ❌ "You must do it this way"
- ❌ Prevent "risky" prompts
- ❌ Score developers on "prompt quality"
- ❌ Mandatory training on "AI best practices"

**What TO do:**
- ✅ "This pattern often works well..."
- ✅ Allow all prompts, offer suggestions
- ✅ Measure outcomes, not compliance
- ✅ Learning through doing, not lectures

---

## Part 6: Success Metrics & KPIs

### Phase 1 Metrics (Pre-Flight)
- Tool response time: <500ms target
- Developer opt-in rate: >30% target
- Helpfulness rating: >4/5 target
- False positive rate: <5% target

### Phase 2 Metrics (In-Flight)
- Alert accuracy: >90% true positives
- Session rescue rate: >50% of alerted sessions
- Developer satisfaction: >7/10
- No measurable increase in interruptions

### Phase 3 Metrics (Digest)
- Open rate: >50%
- Recommendation follow-through: >20%
- Sentiment: Positive

### Phase 4 Metrics (Overall)
- Trouble score reduction: >20% vs control
- Developer retention: Using tool after 4 weeks
- Voluntary usage: >50% of sessions (not just mandated)
- Net Promoter Score: >50

---

## Part 7: The First 7 Days (Action Plan)

### Day 1: Extract & Setup
- [ ] Extract prompt classifier to `qc_trace/intelligence/`
- [ ] Extract detection algorithms
- [ ] Create developer_preferences table
- [ ] Set up opt-in flow

### Day 2-3: MCP Skeleton
- [ ] Build basic MCP server
- [ ] Implement `check_prompt()` tool
- [ ] Test locally with Claude Code
- [ ] Refine response format

### Day 4-5: Developer Testing
- [ ] Recruit 3-5 developers
- [ ] Install MCP server
- [ ] Collect feedback on usefulness
- [ ] Iterate on message framing

### Day 6-7: Polish & Document
- [ ] Fix issues from feedback
- [ ] Write setup documentation
- [ ] Create developer onboarding guide
- [ ] Plan Phase 2

---

## Conclusion: Why This Approach Wins

**Technical Feasibility:** ✅ High  
You have the data, algorithms, and infrastructure. 10-second latency is acceptable for the patterns you detect.

**Developer Acceptance:** ✅ High  
Research shows developers WANT help with context management and debugging. They REJECT surveillance. This approach gives them the former without the latter.

**Business Value:** ✅ High  
If you reduce trouble scores by 20%, that's:
- Fewer abandoned sessions
- Less wasted AI spend
- Faster feature delivery
- Happier developers

**The Secret Sauce:**
It's not the detection algorithms (those exist). It's not the data pipeline (that exists too).

**It's the psychology:** Treating developers as partners, not pupils. Offering help, not mandates. Being invisible when they're flowing, present when they're stuck.

**Your advantage:** You have 318 sessions of data proving what works. Now you just need to deliver that knowledge in a way developers want to receive it.

---

**Ready to start?** Begin with Phase 0 (extraction) this week. Build the MCP server next. Test with 3 developers. Iterate. Scale.

**Questions or feedback?** This is a living document — update as you learn.
