# Phase 1.7: Refactor Artifacts for Grep Synergy - Context

**Gathered:** 2026-01-27 (updated with grep-first discussion)
**Status:** Ready for planning

<domain>
## Phase Boundary

Restructure all GSD-lite artifacts (WORK.md, STATE.md, protocols, workflows) for two goals:

1. **Grep synergy** — Key information discoverable via single-line grep patterns. Enable non-linear retrieval — agents grep headers/tags first, then surgical read of relevant lines. WORK.md becomes perpetual (not ephemeral), with user-controlled housekeeping to archive old entries.

2. **Pair programming philosophy** — Revamp coaching philosophy from "User = visionary, Agent = builder" to "User + Agent = thinking partners exploring together." Agent as navigator (proposes hypotheses, challenges assumptions, teaches with analogies), user as driver.

**This phase is about:**
- Organizing content for grep/search workflows with `uvx fs-mcp` tools
- Revamping agent behavior to be collaborative explorer, not task executor
- Ensuring all templates and workflows are coherent after these changes

</domain>

<decisions>
## Implementation Decisions

### Coaching Philosophy Revamp (CRITICAL)

**Old framing (remove):** "User = visionary. Agent = builder."
**New framing:** "User + Agent = thinking partners exploring together."

Agent operates as **navigator**, user remains **driver**.

**Core behaviors to encode in all workflows:**

1. **Propose hypotheses** — Agent says "What if we tried X?" for user to react to, not just receives instructions
2. **Challenge assumptions** — Agent asks "Why do you think that?" and "Have you considered Y?" — pushes user to think
3. **Teach with analogies** — Agent explains concepts with relatable mental models (like "Road Map vs Road Trip")
4. **Celebrate discoveries** — Agent highlights "aha" moments: "Exactly! 🌟 You nailed it"
5. **Transparent reasoning** — Agent explains WHY it's asking a question or suggesting a path
6. **Treat errors as learning** — Failures become learning moments, not just bugs to fix
7. **Validate first** — Acknowledge correct logic before giving feedback or corrections
8. **Mandatory confirmation loops** — End responses with "[YOUR TURN]" or explicit handoff to user
9. **Handle frustration** — Switch from Socratic to direct explanation when user is stuck, then resume guided inquiry

**What to REMOVE from workflows:**
- "User owns outcome. Agent executes." (too hierarchical)
- "Never expand scope mid-phase" (treats exploration as failure)
- "Scope creep → Capture to INBOX" (parks interesting threads instead of exploring them)
- Any framing of agent as task-doer rather than thinking partner

**Reference:** prompt_guided_learning.md (full guided learning prompt)

### First Turn Protocol (Bug Fix)

**Problem:** Agent reads artifacts on first turn and writes moodboard content INTO INBOX.md instead of talking to user.

**Fix:** Agent ALWAYS talks to user first before writing to any artifact.

**First turn sequence:**
1. Read PROTOCOL.md (silently)
2. Read WORK.md Current Understanding (silently)
3. **TALK to user:** "Here's what I understand from the artifacts... What would you like to explore today?"
4. Only write to artifacts AFTER conversing with user

**Never on first turn:**
- Write to INBOX.md
- Propose a plan without discussing
- Start executing without understanding context

### Log Entry Format
- Use `[LOG-NNN]` prefix for all log entries (sequential, zero-padded)
- Each entry includes: timestamp, type tag, task tag, summary, details with code
- **Always include code snippets** in EXEC/DISCOVERY logs — maximum context for PR extraction

### Log Type Taxonomy (6 types)
- `[VISION]` — Goals, what success looks like
- `[DECISION]` — Choices made with rationale
- `[DISCOVERY]` — Evidence, findings, data (with code snippets)
- `[PLAN]` — Strategy, approach chosen
- `[BLOCKER]` — Issues, pivots, what didn't work
- `[EXEC]` — Concrete actions taken (with code snippets)

### File Structure Changes
- **Merge STATE.md into WORK.md** — fewer files = less confusion for weak agents
- WORK.md structure:
  1. Current Understanding (handoff header)
  2. Key Events Index (grep accelerator table)
  3. Session Log (chronological, atomic entries)
- Deprecate STATE.md — remove from templates

### Multi-Task Support in WORK.md
- Each log entry tagged with Task: `[LOG-017] - [timestamp] - [VISION] - Task: MODEL-C`
- Index table includes Task column for filtering
- Current Understanding shows: Active Task + Parked tasks

### Housekeeping Workflow (Replaces Promotion)
- Single workflow handles both PR extraction AND archiving
- User triggers with natural language: "write PR for MODEL-A" or "clean up WORK.md"
- Archived entries move to HISTORY.md
- WORK.md is now **perpetual** — not deleted after promotion

### Grep Pattern Design
- All patterns are **single-line only** (simpler, more portable)
- No multiline grep required — use "grep line number → read section" workflow
- Key patterns:
  - Header discovery: `^## `
  - Log ID lookup: `^\[LOG-028\]`
  - Type filtering: `\[DECISION\]`
  - Task filtering: `Task: MODEL-A`

### Grep-First Behavior Prompting (NEW - 2026-01-27 Discussion)

**Problem identified:** Plans assumed agents would use grep_content/Grep tools, but agents often default to reading entire files. Nothing in workflows explicitly prompted "grep first, read second" behavior.

**Gap addressed:** Agents need explicit instruction to grep before reading, and guidance on calculating line ranges.

**Location of instructions:** PROTOCOL.md (both locations)
1. **Fresh Agent Resume Protocol** — Brief instruction in the resume sequence
2. **File Reading Strategy section** — Detailed reference with patterns and fallbacks

**Why both:** Resume protocol is the sequence agents follow (need to know HOW before reading). Reference section provides detailed patterns for mid-session lookup.

**Workflow files:** Do NOT repeat grep-first instruction (DRY principle). PROTOCOL.md is the authority.

### Line Range Calculation (NEW - 2026-01-27 Discussion)

**The gap:** grep_content returns line numbers, but agents must calculate end_line from "next match - 1". Weaker agents skip this and either read to EOF or fixed chunks.

**Solution: Two-pronged approach**

1. **Workflow-side (PROTOCOL.md):**
   - Teach pattern: grep all headers → calculate ranges → surgical read
   - Graceful degradation: If no grep tool, read first 50 lines (Current Understanding always at top)

2. **Server-side (fs-mcp enhancement - separate spec):**
   - Add `read_to_next_pattern` parameter to `read_files`
   - Agent specifies regex pattern, server finds where section ends
   - Example: `read_files([{"path": "WORK.md", "start_line": 120, "read_to_next_pattern": r"^\[LOG-"}])`
   - Works with open tags (no closing tag needed) — stops at START of next item

**Boundary patterns supported:**
- Headers: `^## ` (level 2), `^#+ ` (any level), `^# ` (level 1)
- Log entries: `^\[LOG-`
- Configurable via regex — not hardcoded

**fs-mcp enhancement spec:** Created separately (see discussion). Includes:
- `read_to_next_pattern` parameter implementation
- Edge cases (pattern not found → read to EOF)
- Updated docstrings showing the workflow
- Optional `Section-End-Hint` output from grep_content

### README Grep Documentation (NEW - 2026-01-27 Discussion)

**Decision:** Include grep patterns in README's Artifact Overview table (brief "How to Read" column)

**Why this location:**
- README audience is humans (users, maintainers), not agents
- Brief mention in table is sufficient — detailed patterns live in PROTOCOL.md
- Maintainer's Guide already has testing guidance for grep patterns

### Template Coherence Updates Required
- PROTOCOL.md — update file references, remove ephemeral WORK.md language
- WORK.md template — new 3-part structure
- All workflow files — update for perpetual WORK.md, merged housekeeping
- STATE.md — deprecate (merged into WORK.md)

### Testing with fs-mcp
- Create golden sample WORK.md (50-100 lines) with all log types
- Spin up `uvx fs-mcp` server, send real grep queries via stdio
- Test cases:
  1. Header discovery: `^## ` finds all section headers
  2. Log ID lookup: `[LOG-NNN]` finds specific entry
  3. Type filtering: `[DECISION]` finds all decisions
  4. Task filtering: `Task: MODEL-A` finds task-specific entries

### README Revamp (Human-Focused Documentation)

**Problem:** The hyper-condensed artifacts are optimized for agents, but hard for human maintainers/users to understand the philosophy, design, and flow.

**Goal:** README helps humans understand:
1. What this workflow is about (philosophy)
2. How all artifacts synergize (the system)
3. Typical workflow with examples (the experience)

**README Structure:**
- Philosophy section — Why pair programming approach? Why perpetual WORK.md?
- Artifact overview — What each file does, how they connect
- Typical session walkthrough — Example of moodboard → exploration → execution → housekeeping
- Workflow diagram — Mermaid showing artifact lifecycle and workflow routing

**Maintainer's Guide (Semantic CICD):**

Add a "For Maintainers" section directly in README with:
1. **Testing new workflows** — How to verify changes work (eval framework, golden samples)
2. **Updating README** — When to update, what tone to use, what examples to include
3. **Preserving the vision** — Core principles that must not regress (pair programming, exploration > tasks, etc.)

This acts as "semantic CICD" — guiding future agents/humans on how to iterate while preserving the vision.

**Tone for README:**
- Conversational, not corporate
- Show don't tell (examples > descriptions)
- Explain the "why" not just the "what"
- Match the pair programming philosophy (README itself should feel like a thinking partner explaining things)

### Final Coherence Verification (Regression Prevention)
- **Cross-reference check:** All workflow files must reference the same artifact names and structures
- **No orphan references:** Remove all references to deprecated STATE.md
- **No conflicting instructions:** Ensure no workflow says "delete WORK.md" or "ephemeral"
- **Consistent log format:** All examples in all files use [LOG-NNN] format with 6 types
- **Human walkthrough:** User reads through all updated templates to verify coherence

### Claude's Discretion
- Exact formatting of the Key Events Index table
- How to handle edge cases in log ID sequencing
- Structure of HISTORY.md archive entries
- Error handling in test harness

</decisions>

<specifics>
## Specific Ideas

- "I like the verbose logging in WORK.prod.md — include code snippets, full context"
- "The goal is context condensing — fresh agent can read logs and one-shot write PR"
- "Agent should grep headers first (^## ) to discover structure, then surgical read"
- "Think of WORK.md as a simple database — grep to find, read to retrieve"
- "I treat agents like pair programmers — throwing ideas back and forth, creating hypotheses, verifying them, having fun learning, documenting the journey"
- "The last thing I want is an agent that jumps straight to proposing a 'fix' without interviewing me, without pushing ME to think"

**References:**
- WORK.prod.md (production session artifact)
- suggestions.md (V2 proposal for grep-friendly structure)
- prod_session.md (detailed session flow showing pair programming in action)
- prompt_guided_learning.md (guided learning prompt with Socratic method, analogies, confirmation loops)
- fs-mcp server.py (reference for grep_content and read_files tools)

**fs-mcp Enhancement Spec (created 2026-01-27):**
Full spec for MCP server enhancement lives in this discussion transcript. Key points:
- Add `read_to_next_pattern` parameter to `read_files` for section-aware reading
- Pattern is regex, works with open tags (`^\[LOG-`) and headers (`^## `)
- Edge case: pattern not found → read to EOF
- P0: Update docstrings to show grep→read workflow
- P1: Implement `read_to_next_pattern`
- P2: Add `Section-End-Hint` to grep_content output

</specifics>

<deferred>
## Deferred Ideas

None — discussion stayed within phase scope.

</deferred>

---

*Phase: 01.7-refactor-artifacts-and-protocols-and-workflows-to-synergy-with-grep*
*Context gathered: 2026-01-27*
