# Phase 10: Pipeline Stabilization — UA Testing

## Overview

This phase stabilizes the LLM pipeline for GLM's rate limits and token constraints. Testing requires real LLM calls to verify retry behavior.

## Prerequisites

- GLM API key configured (`GLM_API_KEY` environment variable)
- Target repository available (default: `~/Documents/source/lifeLevel`)

## Test Cases

### TC-1: Full Pipeline with GLM

**Purpose:** Verify pipeline completes all categories without 429 errors.

**Steps:**
```bash
cd /project
GLM_API_KEY=your-key UNFAMILIAR_PROVIDER=glm python _planning/testing_script.py
```

**Expected:**
- [x] Phase 3 (Static Analysis) completes
- [x] Phase 6 (Briefing Generation) completes
- [x] 60-second cooldown before missions
- [x] Phase 7 (Mission Generation) completes for all test categories
- [x] Output file written to `_planning/output.md`

**Status:** ✅ PASS (verified 2026-03-14, ran successfully twice)

---

### TC-2: Rate Limit Recovery

**Purpose:** Verify 30-second retry delays are used for GLM.

**Steps:**
1. Run testing script with GLM
2. Observe console output for retry messages

**Expected:**
- [x] Retry delays show 30s+ intervals (not 1s)
- [x] 429 errors trigger retries, not immediate failure
- [x] Pipeline eventually succeeds after rate limit cooldown

**Status:** ✅ PASS (verified 2026-03-14)

---

### TC-3: Schema Validation Retry

**Purpose:** Verify schema validation errors are retried.

**Steps:**
1. Run testing script with GLM
2. If LLM returns incomplete JSON, observe retry behavior

**Expected:**
- [x] "did not match expected schema" errors trigger retry
- [x] Pipeline continues after successful retry
- [x] No immediate failure on schema errors

**Status:** ✅ PASS (verified 2026-03-14, Output 7 showed this was needed and fixed)

---

### TC-4: Mission Quality Review

**Purpose:** Verify generated missions are meaningful.

**Steps:**
1. Open `_planning/output.md`
2. Review generated missions for:
   - Meaningful titles
   - Complete descriptions
   - Valid file references
   - Appropriate difficulty levels

**Expected:**
- [x] At least 3 missions per category
- [ ] Missions have narrative intro/outro
- [ ] File references point to actual files
- [ ] Prerequisites link to earlier missions

**Status:** ⬜ PENDING (user review)

---

## Environment Notes

- Testing script uses `glm-4.7-Flash` by default
- 60-second cooldown between briefing and mission phases
- Provider-specific retry delays: GLM = 30s, Anthropic = 1s

## Issues Found During Testing

| Issue | Output | Resolution |
|-------|--------|------------|
| `provider_name` not passed | #6 | Added param to `generate_briefing()`, updated testing script |
| Schema errors not retried | #7 | Added validation patterns to transient errors in `retry.py` |

## Sign-Off

- **Tester:** User
- **Date:** 2026-03-14
- **Result:** ✅ PASS (2 successful runs)
