# PLAN: Skill Template Refactor

## Related Plans (Must Stay In Sync)

- `docs/PLAN-quality-regression-fix.md` (quality restoration + Output Contract enforcement + framework/data footgun)

## Problem Statement

We have skills for 3 integrations (Amp, Claude, Codex) that are substantively identical but differ in:
- Frontmatter format (Amp vs Claude vs Codex)
- Invocation syntax (`/check` vs `$check` vs natural language)
- Tool-specific notes (hooks, version control, permissions)

Current state: **8 Claude skills + 2 Codex skills + 5 Amp skills = 15 SKILL.md files** with significant content overlap. This creates:

1. **Maintenance burden** - updating methodology requires editing multiple files
2. **Drift risk** - skills can diverge unintentionally
3. **Inconsistency** - some skills are more complete than others

## Goals

1. **Single source of truth** for core methodology content
2. **DRY assembly** - generate integration-specific skills from templates
3. **Easy updates** - change once, regenerate all
4. **Validation** - ensure generated skills are well-formed
5. **Minimal complexity** - simple enough to maintain without specialized knowledge

## Key Decisions

| Decision | Choice | Rationale |
|----------|--------|-----------|
| Template structure | **Partials** (not monolithic) | Evidence hierarchy, claim types, etc. appear in 5+ skills - one edit updates all |
| Source of truth | **Templates are canonical** | Avoid drift. Human-readable docs (e.g., `methodology/workflows/check-core.md`) should be **generated from the same templates** and treated as read-only reference |
| Integration coverage | **Generate all skills for all integrations** | Consistency over minimalism; no `enabled: false` flags |
| Quality restoration | **Restore full templates from POC** | See `PLAN-quality-regression-fix.md` - we lost ~675 lines of critical structure |
| Validation | **Add formatter + validator** | Machine-checkable output contract (see Option E in quality fix) |

## Quality Regression Integration

**Critical**: This refactor must restore the quality lost when we over-trimmed the methodology.

**Source**: `postsingularity-economic-theories/AGENTS.md` (934 lines) → our templates

### Missing Templates to Restore

These were identified in `PLAN-quality-regression-fix.md` and must be added as partials:

| Template | Stage | Purpose |
|----------|-------|---------|
| Key Claims table (full) | 1 | With `Verified?` and `Falsifiable By` columns |
| Argument Structure diagram | 1 | Logical flow visualization |
| Theoretical Lineage | 1 | What traditions this builds on |
| Key Factual Claims Verified | 2 | With `Crux?` column |
| Disconfirming Evidence Search | 2 | Actively search for contradictions |
| Internal Tensions | 2 | Self-contradictions table |
| Persuasion Techniques | 2 | Rhetorical devices used |
| Unstated Assumptions | 2 | With `Critical?` and `Problematic?` |
| Supporting/Contradicting Theories | 3 | With source IDs |
| Confidence in Analysis | End | 0.0-1.0 score with reasoning |
| Top-of-file Legends | Top | Claim types + evidence hierarchy quick ref |

### Output Contract Enforcement (Option E)

This refactor must not only restore templates, it must make the Output Contract **hard to accidentally violate**:

- **Preprocessor / skeleton**: generate a standard analysis markdown skeleton (headings + legends + empty tables) before the model writes content.
- **Post-processor / formatter**: idempotently insert missing boilerplate (especially legends) and normalize tables/headings for auditability.
- **Validator / lint**: strict checker that can fail the run (or block auto-commit) if required elements are missing.

Important nuance from `PLAN-quality-regression-fix.md`:
- Support **profiles**: “full analysis” vs “quick extraction/cross-reference” (explicitly labeled), so quick sources don’t force the full Stage 2 table set.

### Framework/Data Footgun Guardrails

The templating system should bake in the separation guardrails from `PLAN-quality-regression-fix.md`:

- Prominently require `REALITYCHECK_DATA` and deriving `PROJECT_ROOT` from it.
- Include “red flags you’re in the framework repo” (presence of `scripts/`, `integrations/`, etc).
- Ensure any autogenerated commands (examples) never default to writing into `./data/realitycheck.lance` in the framework repo.
- Track the corresponding **code-level** mitigations (CLI hard guardrail + “require explicit project marker” resolution) per `PLAN-quality-regression-fix.md` so the docs/templates are not the only protection.

### Additional Methodology from POC

From `postsingularity-economic-theories/AGENTS.md`, also capture:

- Claim Registry System (ID format, record structure, relationship types)
- Argument Chain Notation (chain format, weakest link analysis)
- Scenario Matrix Template
- Theory Comparison Matrix Template
- Indicator Dashboard Template
- Confidence Calibration guidelines
- Research Protocols (step-by-step procedures)

## Current Skill Inventory

### Skills to Generate

We want consistent core content, but **surface area differs per tool** (naming + UX):

- **Claude**: individual skills (`check`, `analyze`, `extract`, `search`, `validate`, `export`, `stats`) plus a `realitycheck` alias to `check` (current behavior).
- **Codex**: keep `$check` for analysis; keep `$realitycheck` as the **utilities** entry point (current behavior). We can still generate the full skill set for parity, but docs should prefer `$realitycheck …` for utilities so we don’t fragment the UX.
- **Amp**: individual skills prefixed `realitycheck-*`.

Logical “capabilities” to cover (may map differently per integration):

| Skill | Description |
|-------|-------------|
| `check` | Full analysis workflow (fetch → analyze → extract → register → validate) |
| `realitycheck` | Claude: alias for `check`; Codex: utilities wrapper (data/stats/search/validate/export/embed) |
| `analyze` | Manual 3-stage analysis without registration |
| `extract` | Quick claim extraction |
| `search` | Semantic search across claims/sources |
| `validate` | Database integrity validation |
| `export` | YAML/Markdown export |
| `stats` | Database statistics |

### Content Layers

```
┌─────────────────────────────────────────────────────┐
│  Integration-Specific Frontmatter                   │  <- varies by tool
│  (name, description, argument-hint, allowed-tools)  │
├─────────────────────────────────────────────────────┤
│  Tool-Specific Header                               │  <- invocation, prerequisites
│  (/check vs $check vs natural language)             │
├─────────────────────────────────────────────────────┤
│  SHARED: Core Methodology                           │  <- same everywhere
│  (workflow steps, evidence hierarchy, claim types,  │
│   output format, database commands)                 │
├─────────────────────────────────────────────────────┤
│  Tool-Specific Footer                               │  <- hooks, version control
│  (Claude hooks vs Codex manual vs Amp notes)        │
└─────────────────────────────────────────────────────┘
```

## Proposed Architecture

### Directory Structure

```
integrations/
├── _templates/                    # Jinja2 templates (SOURCE OF TRUTH)
│   ├── partials/                  # Reusable content snippets
│   │   ├── data-repo-guardrails.md.j2  # DATA vs framework repo + red flags
│   │   ├── legends.md.j2          # Top-of-file claim types + evidence quick ref
│   │   ├── evidence-hierarchy.md.j2
│   │   ├── claim-types.md.j2
│   │   ├── domain-codes.md.j2
│   │   ├── claim-relationships.md.j2
│   │   ├── db-commands.md.j2
│   │   ├── prerequisites.md.j2
│   │   └── confidence-calibration.md.j2
│   ├── tables/                    # Analysis table templates (restored from POC)
│   │   ├── key-claims.md.j2       # Full table with Verified?/Falsifiable By
│   │   ├── claim-summary.md.j2
│   │   ├── factual-claims-verified.md.j2  # With Crux? column
│   │   ├── disconfirming-evidence.md.j2
│   │   ├── internal-tensions.md.j2
│   │   ├── persuasion-techniques.md.j2
│   │   ├── unstated-assumptions.md.j2
│   │   └── supporting-contradicting.md.j2
│   ├── sections/                  # Analysis section templates
│   │   ├── argument-structure.md.j2
│   │   ├── theoretical-lineage.md.j2
│   │   └── confidence-assessment.md.j2
│   ├── skills/                    # Skill-specific content
│   │   ├── check.md.j2            # Check workflow (includes partials + tables)
│   │   ├── realitycheck.md.j2     # Claude: alias UX; Codex: utilities wrapper
│   │   ├── analyze.md.j2
│   │   ├── extract.md.j2
│   │   ├── search.md.j2
│   │   ├── validate.md.j2
│   │   ├── export.md.j2
│   │   └── stats.md.j2
│   ├── analysis/                  # Analysis preprocessor skeleton(s) (Option E)
│   │   ├── source-analysis-full.md.j2
│   │   └── source-analysis-quick.md.j2
│   └── wrappers/                  # Integration-specific wrappers
│       ├── amp.md.j2              # Amp frontmatter + natural language triggers
│       ├── claude.md.j2           # Claude frontmatter + /command syntax
│       └── codex.md.j2            # Codex frontmatter + $command syntax
├── _config/
│   └── skills.yaml                # Skill definitions + per-integration metadata
├── amp/skills/                    # Generated (DO NOT EDIT DIRECTLY)
├── claude/skills/                 # Generated (DO NOT EDIT DIRECTLY)
├── codex/skills/                  # Generated (DO NOT EDIT DIRECTLY)
└── assemble.py                    # Build script
```

### Configuration Schema (skills.yaml)

```yaml
# integrations/_config/skills.yaml

# Global defaults per integration
defaults:
  amp:
    skill_dir: "~/.config/agents/skills"
    name_prefix: "realitycheck-"      # e.g., realitycheck-check
  claude:
    skill_dir: "~/.claude/skills"
    name_prefix: ""                   # e.g., check
  codex:
    skill_dir: "$CODEX_HOME/skills"
    name_prefix: ""                   # e.g., check

# Skill definitions (all generated for all integrations)
skills:
  check:
    title: "Full Analysis Workflow"
    description: "Full Reality Check analysis - fetch, analyze, extract, register, validate"
    template: "check.md.j2"
    related: ["search", "validate", "stats"]
    amp:
      triggers:
        - "Analyze this article for claims"
        - "Reality check this URL"
        - "Run a check on"
    claude:
      argument_hint: "<url> [--domain DOMAIN] [--quick] [--no-register] [--continue]"
      allowed_tools:
        - "WebFetch"
        - "Read"
        - "Write"
        - "Bash(rc-db *)"
        - "Bash(rc-validate *)"
    codex:
      invocation: "$check <url>"
    
  realitycheck:
    description: "Claude: alias of check. Codex: Reality Check utilities wrapper (data/stats/search/validate/export/embed)."
    template: "realitycheck.md.j2"
    
  analyze:
    title: "Manual 3-Stage Analysis"
    description: "Perform 3-stage analysis without automatic database registration"
    template: "analyze.md.j2"
    related: ["check", "extract"]
    amp:
      triggers:
        - "Analyze this source"
        - "Do a 3-stage analysis"
    claude:
      argument_hint: "<url_or_source_id>"
      allowed_tools: ["WebFetch", "Read", "Write"]
    codex:
      invocation: "$analyze <url>"

  extract:
    title: "Quick Claim Extraction"
    description: "Extract claims from a source without full analysis"
    template: "extract.md.j2"
    related: ["check", "analyze"]
    amp:
      triggers:
        - "Extract claims from"
        - "Quick extract"
    claude:
      argument_hint: "<source>"
      allowed_tools: ["WebFetch", "Read", "Write"]
    codex:
      invocation: "$extract <source>"

  search:
    title: "Semantic Search"
    description: "Search claims and sources using natural language queries"
    template: "search.md.j2"
    related: ["stats", "validate"]
    amp:
      triggers:
        - "Search for claims about"
        - "Find related claims"
    claude:
      argument_hint: "QUERY [--domain DOMAIN] [--limit N]"
      allowed_tools: ["Bash(rc-db search *)"]
    codex:
      invocation: "$search <query>"

  validate:
    title: "Data Validation"
    description: "Check database integrity and referential consistency"
    template: "validate.md.j2"
    related: ["check", "stats"]
    amp:
      triggers:
        - "Validate the database"
        - "Check data integrity"
    claude:
      argument_hint: "[--strict] [--json]"
      allowed_tools: ["Bash(rc-validate *)"]
    codex:
      invocation: "$validate"

  export:
    title: "Data Export"
    description: "Export data to YAML or Markdown formats"
    template: "export.md.j2"
    related: ["stats", "validate"]
    amp:
      triggers:
        - "Export claims to YAML"
        - "Generate a summary report"
    claude:
      argument_hint: "<format> <type> [--id ID] [-o OUTPUT]"
      allowed_tools: ["Bash(rc-export *)"]
    codex:
      invocation: "$export <format> <type>"

  stats:
    title: "Database Statistics"
    description: "Show counts for claims, sources, chains, and predictions"
    template: "stats.md.j2"
    related: ["search", "validate"]
    amp:
      triggers:
        - "Show database stats"
        - "How many claims"
    claude:
      argument_hint: ""
      allowed_tools: ["Bash(rc-db stats)"]
    codex:
      invocation: "$stats"
```

### Template Examples

**Partial (partials/evidence-hierarchy.md.j2)**:
```jinja2
## Evidence Hierarchy

| Level | Description | Credence Range |
|-------|-------------|----------------|
| E1 | Systematic review, meta-analysis | 0.9-1.0 |
| E2 | Peer-reviewed study, official stats | 0.6-0.8 |
| E3 | Expert consensus, preprints | 0.5-0.7 |
| E4 | Industry reports, journalism | 0.3-0.5 |
| E5 | Opinion, anecdote | 0.2-0.4 |
| E6 | Speculation, unfalsifiable | 0.0-0.2 |
```

**Skill template (skills/check.md.j2)**:
```jinja2
{# Check workflow - includes partials for shared content #}

## Workflow

1. **Fetch** - Retrieve source content
2. **Metadata** - Extract title, author, date, type
3. **Stage 1: Descriptive** - Neutral summary, key claims
4. **Stage 2: Evaluative** - Evidence quality, coherence
5. **Stage 3: Dialectical** - Steelman, counterarguments
6. **Extract** - Format claims with IDs, credence
7. **Register** - Add to database
8. **Validate** - Run integrity checks
9. **Report** - Generate summary

{% include "partials/evidence-hierarchy.md.j2" %}

{% include "partials/claim-types.md.j2" %}

{% include "partials/db-commands.md.j2" %}
```

**Amp wrapper (wrappers/amp.md.j2)**:
```jinja2
---
name: {{ name }}
description: {{ description }}
---

# {{ title }}

{{ description }}

## When This Skill Activates

{% for trigger in triggers %}
- "{{ trigger }}"
{% endfor %}

{% include "partials/prerequisites.md.j2" %}

{% include "skills/" ~ template %}

## Related Skills

{% for r in related %}
- `realitycheck-{{ r }}`
{% endfor %}
```

**Claude wrapper (wrappers/claude.md.j2)**:
```jinja2
---
name: {{ name }}
description: {{ description }}
argument-hint: "{{ argument_hint }}"
allowed-tools: {{ allowed_tools | tojson }}
---

# /{{ name }} - {{ title }}

{{ description }}

## Usage

```
/{{ name }} {{ argument_hint }}
```

{% include "partials/prerequisites.md.j2" %}

{% include "skills/" ~ template %}

## Related Commands

{% for r in related %}
- `/{{ r }}`
{% endfor %}
```

**Codex wrapper (wrappers/codex.md.j2)**:
```jinja2
---
name: {{ name }}
description: "{{ description }}"
---

# {{ title }} (Codex)

{{ description }}

## Invocation

```
{{ invocation }}
```

Note: Codex reserves `/...` for built-in commands. Use `${{ name }}` instead.

{% include "partials/prerequisites.md.j2" %}

{% include "skills/" ~ template %}

## Related Skills

{% for r in related %}
- `${{ r }}`
{% endfor %}
```

### Build Script (assemble.py)

```python
#!/usr/bin/env python3
"""Assemble integration skills from templates."""

import argparse
from pathlib import Path
import yaml
from jinja2 import Environment, FileSystemLoader

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--integration", choices=["amp", "claude", "codex", "all"])
    parser.add_argument("--skill", help="Build specific skill only")
    parser.add_argument("--dry-run", action="store_true")
    parser.add_argument("--diff", action="store_true", help="Show diff only")
    args = parser.parse_args()
    
    config = load_config()
    env = setup_jinja_env()
    
    for integration in get_integrations(args.integration):
        for skill_name, skill_config in config["skills"].items():
            if args.skill and skill_name != args.skill:
                continue
            if not skill_config.get("integrations", {}).get(integration, {}).get("enabled", True):
                continue
            
            output = render_skill(env, integration, skill_name, skill_config)
            write_skill(integration, skill_name, output, args.dry_run, args.diff)

if __name__ == "__main__":
    main()
```

## Migration Plan

### Phase 1: Create Template Structure + Restore from POC
- [ ] Create `_templates/{partials,tables,sections,skills,wrappers}/` directories
- [ ] Extract from `postsingularity-economic-theories/AGENTS.md`:
  - [ ] `partials/data-repo-guardrails.md.j2` (framework vs data repo + red flags)
  - [ ] `partials/legends.md.j2` (claim types + evidence quick ref)
  - [ ] `partials/evidence-hierarchy.md.j2` (full table with weights)
  - [ ] `partials/claim-types.md.j2` (with definitions)
  - [ ] `partials/domain-codes.md.j2`
  - [ ] `partials/claim-relationships.md.j2` (→+, →✗, →?, etc.)
  - [ ] `partials/confidence-calibration.md.j2`
  - [ ] `partials/db-commands.md.j2`
  - [ ] `partials/prerequisites.md.j2`

### Phase 2: Create Table Templates (Quality Restoration)
- [ ] `tables/key-claims.md.j2` - Full table with Verified?/Falsifiable By columns
- [ ] `tables/claim-summary.md.j2`
- [ ] `tables/factual-claims-verified.md.j2` - With Crux? column
- [ ] `tables/disconfirming-evidence.md.j2` - Counterevidence/Alternative/Search Notes
- [ ] `tables/internal-tensions.md.j2` - Tension/Parts in Conflict/Implication
- [ ] `tables/persuasion-techniques.md.j2` - Technique/Example/Effect
- [ ] `tables/unstated-assumptions.md.j2` - With Critical?/Problematic?
- [ ] `tables/supporting-contradicting.md.j2` - With source IDs

### Phase 3: Create Section Templates
- [ ] `sections/argument-structure.md.j2` - Logical flow diagram
- [ ] `sections/theoretical-lineage.md.j2` - What traditions this builds on
- [ ] `sections/confidence-assessment.md.j2` - End-of-analysis score

### Phase 4: Create Skill Templates
- [ ] `skills/check.md.j2` - Full workflow with all tables/sections
- [ ] `skills/realitycheck.md.j2` - Claude alias UX / Codex utilities wrapper
- [ ] `skills/analyze.md.j2` - Manual 3-stage (subset of check)
- [ ] `skills/extract.md.j2` - Quick extraction
- [ ] `skills/search.md.j2`
- [ ] `skills/validate.md.j2`
- [ ] `skills/export.md.j2`
- [ ] `skills/stats.md.j2`

### Phase 5: Create Integration Wrappers
- [ ] `wrappers/amp.md.j2` - Natural language triggers
- [ ] `wrappers/claude.md.j2` - /command syntax + allowed-tools
- [ ] `wrappers/codex.md.j2` - $command syntax

### Phase 6: Configuration
- [ ] Create `_config/skills.yaml` with all skill definitions
- [ ] Include per-integration metadata (triggers, argument hints, etc.)

### Phase 7: Build Script
- [ ] Implement `assemble.py`
- [ ] Support `--integration` filter (amp/claude/codex/all)
- [ ] Support `--skill` filter (single skill)
- [ ] Add `--dry-run` mode (show what would be generated)
- [ ] Add `--diff` mode (show changes vs existing)
- [ ] Add `--check` mode (exit non-zero if files would change)
- [ ] Add frontmatter validation (name ≤64 chars, required fields)

### Phase 8: Output Contract Enforcement (Option E)

- [ ] Add analysis skeleton templates (`_templates/analysis/source-analysis-{full,quick}.md.j2`)
- [ ] Implement an analysis formatter (idempotent) that:
  - [ ] Inserts legends if missing
  - [ ] Ensures required headings/tables exist
  - [ ] Normalizes the “Key Claims” and “Claim Summary” table headers/columns
- [ ] Implement an analysis validator with “full” and “quick” profiles (per `PLAN-quality-regression-fix.md`)
- [ ] Validator must also fail fast if `PROJECT_ROOT` appears to be the framework repo (red-flag dirs like `scripts/`, `integrations/`, `methodology/`)
- [ ] Add tests for formatter/validator (fixtures for “missing legends”, “missing tables”, “quick profile allowed”)
- [ ] Wire Claude plugin hooks to run formatter before auto-commit and validator at end-of-run (or block commit on failure)

### Phase 9: Makefile & CI Integration
- [ ] Add `make assemble-skills` target
- [ ] Add `make check-skills` target (uses `--check` mode)
- [ ] Update install scripts to use generated paths
- [ ] Add generated file header markers

### Phase 10: Cleanup & Documentation
- [ ] Remove old manual skill files (replaced by generated)
- [ ] Update `integrations/README.md`
- [ ] Update `CONTRIBUTING.md` with template workflow
- [ ] Generate `methodology/workflows/check-core.md` from templates (read-only reference; do not edit by hand)

## Considerations

### Generated File Markers

Add header to generated files so editors know not to modify directly:
```markdown
<!-- GENERATED FILE - DO NOT EDIT DIRECTLY -->
<!-- Source: integrations/_templates/ + _config/skills.yaml -->
<!-- Regenerate: make assemble-skills -->
```

### Validation (in assemble.py)

- Frontmatter schema: name ≤64 chars, description ≤1024 chars
- Required fields per integration (Claude needs `allowed-tools`, etc.)
- Template syntax validation (Jinja2 errors caught early)
- All skills generate for all integrations (no gaps)

### Diff-Friendly Output

- Consistent YAML key ordering in frontmatter
- Stable partial include order
- No trailing whitespace
- Deterministic output (same input = same output)

## Success Criteria

- [ ] Single edit to evidence hierarchy updates all 24 generated skills
- [ ] `make assemble-skills` regenerates all skills in <5s
- [ ] `make check-skills` fails if generated files are stale
- [ ] Adding new integration requires only: new wrapper template + defaults in skills.yaml
- [ ] Adding new skill requires only: new skill template + entry in skills.yaml
- [ ] Generated skills pass validation (frontmatter, required fields)
- [ ] Quality regression is resolved on a known-bad case: `stross-2025-the-pivot-1` reprocessed and passes the analysis validator (full profile)

## Dependencies

- Python 3.10+
- Jinja2 (already in dev dependencies or add to pyproject.toml)
- PyYAML (already in dependencies)

## Estimated Effort

| Phase | Effort |
|-------|--------|
| Phase 1: Template structure + partials (from POC) | 2-3 hours |
| Phase 2: Table templates (quality restoration) | 2-3 hours |
| Phase 3: Section templates | 1 hour |
| Phase 4: Skill templates | 2-3 hours |
| Phase 5: Integration wrappers | 1 hour |
| Phase 6: skills.yaml config | 1 hour |
| Phase 7: assemble.py script | 2-3 hours |
| Phase 8: Output Contract enforcement (Option E) | 3-4 hours |
| Phase 9: Makefile + CI | 0.5 hours |
| Phase 10: Cleanup + docs | 0.5 hours |
| **Total** | **15-20 hours** |

## Output Summary

After implementation:

```
integrations/
├── _templates/
│   ├── partials/           # 8 shared partials
│   ├── tables/             # 8 table templates (restored from POC)
│   ├── sections/           # 3 section templates
│   ├── skills/             # 8 skill templates (tool-specific `realitycheck` behavior)
│   ├── analysis/           # 2 analysis skeleton templates (full + quick)
│   └── wrappers/           # 3 integration wrappers
├── _config/
│   └── skills.yaml         # Single config file
├── assemble.py             # Build script
├── amp/skills/             # 8 generated skills
├── claude/skills/          # 8 generated skills
└── codex/skills/           # 8 generated skills

scripts/
├── analysis_validator.py   # Machine-checkable output contract
└── analysis_formatter.py   # Insert missing legends/tables
```

Total: **24 generated SKILL.md files** from **~30 template files** + **1 config file**

Quality restoration: Full methodology from 934-line POC AGENTS.md captured in templates

## Related Documents

- **[PLAN-quality-regression-fix.md](PLAN-quality-regression-fix.md)** - Details the quality loss and specific templates to restore
- **Source**: `postsingularity-economic-theories/AGENTS.md` (934 lines) - Original full methodology

---

*Created: 2026-01-22*
*Updated: 2026-01-22 - Integrated quality regression fix requirements*
