# Winkers Update: Convention Rules with Observable MCP Delivery

## Context

Winkers already has:
- `graph.json` — structural graph (computed from code)
- `semantic.json` — intent layer (zone intents, business constraints, conventions — AI-generated)
- SessionEnd hooks — `winkers record --hook` + autocommit (the only hooks that exist)
- MCP tools — `map`, `functions_graph`, `hotspots`, `scope`, `routes`
- `winkers init`, `winkers serve`, `winkers record`, `winkers analyze`, `winkers insights`, `winkers improve`, `winkers dashboard`

No PreToolUse/PostToolUse hooks exist. No rules engine exists (rules.py was deleted — it was unused).

## What We're Adding

A **convention rules** subsystem: structured project conventions delivered to agents through MCP with full observability. Think of it as the "eslint for AI agents" layer inside Winkers.

Key insight: if conventions are injected into prompt, we can't know which ones the agent used. If conventions are available *only* through MCP tool calls, the tool call trace *is* the attribution log.

### New user workflow

```bash
# First time setup (extend existing winkers init)
winkers init
# → existing: builds graph.json, semantic.json
# → NEW: semantic agent proposes rules → [Accept/Skip] per rule
#         tree-sitter detectors propose rules → [Accept/Skip] per rule
#         writes .winkers/rules/rules.json + overview.md

# After agent sessions
winkers conventions analyze
# → reads traces, reports which rules are effective

winkers conventions suggest
# → proposes rule changes based on trace data

# Manual rule management
winkers conventions add "All models inherit from BaseModel in app/models/base.py"
# → interactive: asks for topic, related topics, creates structured rule

winkers conventions list
# → shows rules with effectiveness stats
```

---

## Part 1: Data Model — `.winkers/rules/`

### rules.json (source of truth)

Lives alongside `graph.json` and `semantic.json`. Committed to git.

```json
{
  "version": 1,
  "project": "auto-detected or from winkers init",
  "config": {
    "mode": "hybrid",
    "overview_max_tokens": 300,
    "auto_analyze": true,
    "suggest_model": "haiku"
  },
  "rules": [
    {
      "id": 1,
      "topic": "models",
      "title": "Model inheritance",
      "content": "All models MUST inherit from BaseModel in app/models/base.py...",
      "related": ["validation"],
      "source": "semantic-agent",
      "created": "2026-03-28",
      "stats": {
        "times_requested": 0,
        "times_applied": 0,
        "times_confident_wrong": 0
      }
    }
  ]
}
```

Valid `source` values:
- `"semantic-agent"` — proposed by SemanticEnricher (Claude Sonnet), accepted by user
- `"auto-detected"` — proposed by tree-sitter detector, accepted by user
- `"manual"` — created via `winkers conventions add`
- `"migrated-from-semantic"` — imported from existing semantic.json (existing users only)

Key design decisions:
- `stats` updated by `winkers conventions analyze`, not by MCP server at runtime
- `related` enables discovery chains (agent reads rule #1, sees related "validation", fetches #2)
- Rule IDs are stable integers, never reused after deletion
- Only `source != "manual"` rules are candidates for auto-deletion proposals

### overview.md (auto-generated)

Generated by `winkers conventions compile`. Compact view injected in hybrid mode.
Budget: `config.overview_max_tokens`. Regenerated after any rule change.

```markdown
# Project conventions (use convention tools for details)
- models: inherit from BaseModel
- validation: use @validate_payload decorator
- errors: use api_error() helper
```

### traces/ (session logs, gitignored)

```
.winkers/rules/traces/
├── 2026-03-28_session-a1b2c3.jsonl
└── ...
```

Each line is a JSON event:
```json
{"timestamp":"...","event":"orient","topics_returned":["models","validation"]}
{"timestamp":"...","event":"rule_read","topic":"models","rule_id":1}
{"timestamp":"...","event":"implicit_commit","rule_id":1,"file":"app/models/user.py"}
{"timestamp":"...","event":"confident_wrong","rule_id":1,"file":"app/models/user.py"}
```

Retention: configurable, default 30 days. Added to `.gitignore` by `winkers init`.

### insights.md (auto-generated)

Updated by `winkers conventions analyze`. Read by agent via `orient()` next session.

---

## Part 2: MCP Tools — 2 New Tools

Add to existing MCP server in `src/winkers/mcp/tools.py` alongside `map`, `functions_graph`, `hotspots`, `scope`, `routes`.

```python
@mcp.tool()
def orient() -> str:
    """Get project orientation — convention topics, overview, and recent insights.
    Call this first. Use rule_read(topic) for details on any topic."""

@mcp.tool()
def rule_read(topic: str) -> str:
    """Read the full convention rule for a topic.
    Returns rule content, related topics, and architecture context."""
```

Both tools append events to the current session's `.jsonl` trace file.
Session ID sourced from the MCP server context (already tracked in `mcp.log`).

No `rule_commit`, `rule_assess`, or `rule_browse` — agent navigates naturally, we observe.

---

## Part 3: Rule Sources — `winkers init` Extension

Three sources, all go through `[Accept / Skip]` (or `[Delete / Keep]` for removals).
Never auto-write rules without user confirmation.

### Source 1: SemanticEnricher

`SemanticEnricher.enrich()` is extended: it still writes `semantic.json` (data_flow, domain_context,
zone_intents, monster_files, new_feature_checklist), but no longer writes `conventions[]` or
`constraints[]` — those fields are removed from `SemanticLayer`.

Instead the API response schema is extended with a `proposed_rules` section:

```json
{
  "data_flow": "...",
  "domain_context": "...",
  "zone_intents": { ... },
  "monster_files": { ... },
  "new_feature_checklist": [ ... ],
  "proposed_rules": [
    {
      "topic": "models",
      "title": "Model inheritance",
      "content": "All models MUST inherit from BaseModel in app/models/base.py",
      "wrong_approach": "Inheriting from plain dict or object — loses validation"
    }
  ]
}
```

`proposed_rules` is not saved to `semantic.json`. It drives the reconcile step:

```python
proposed = semantic_layer.proposed_rules
existing = rules_store.load()

new_rules   = [r for r in proposed if not exists_in(r, existing)]
stale_rules = [r for r in existing
               if r.source == "semantic-agent"
               and not exists_in(r, proposed)]

present_new(new_rules)      # [Accept / Skip] per rule
present_stale(stale_rules)  # [Delete / Keep] per rule
```

### Source 2: tree-sitter Detectors

Use existing tree-sitter infrastructure from `src/winkers/parser.py`.
Same reconcile logic: propose new, propose deleting stale.

Detectors:
- `base_class_detector` — abstract base classes inherited by 3+ models
- `error_handler_detector` — centralized error functions (jsonify + status_code pattern)
- `decorator_detector` — validation/auth decorators on 3+ route functions
- `test_fixture_detector` — pytest fixtures in conftest.py
- `import_pattern_detector` — stdlib wrappers imported by 3+ files

### Source 3: Manual

`winkers conventions add "..."` — interactive, always `source: "manual"`.
Manual rules are never proposed for auto-deletion.

### Reconcile logic (shared)

```python
def reconcile_rules(existing: list[Rule], proposed: list[Rule], source: str):
    new     = [r for r in proposed if not exists_in(r, existing)]
    stale   = [r for r in existing
               if r.source == source and not exists_in(r, proposed)]
    # present new   → [Accept / Skip]
    # present stale → [Delete / Keep]
    # everything else → untouched
```

Rules edited by user after acceptance keep their original `source` — they are not
affected by reconciliation of that source.

---

## Part 4: CLI Commands — `winkers conventions`

New command group in `src/winkers/cli/main.py`, alongside existing commands.

### `winkers conventions list [--stats]`

```
 ID  Topic       Title                Requests  Applied  Miscal.  Health
  1  models      Model inheritance     10/12     10/10    0.33    EFFECTIVE
  2  validation  Input validation       8/12      8/8     0.00    EFFECTIVE
 10  responses   Response wrapper       5/12      1/5     —       HARMFUL
```

### `winkers conventions add "description"`

Interactive: asks for topic, related topics, optional file reference.
Writes to `rules.json` with `source: "manual"`. Runs `compile` after.

### `winkers conventions compile`

Regenerates `overview.md` from `rules.json`. Respects `overview_max_tokens`.
Priority order: EFFECTIVE → UNDERUSED with high miscalibration → skip HARMFUL.

---

## Part 5: Trace Analyzer

### `winkers conventions analyze [--period 7d] [--session SESSION_ID]`

Reads `.jsonl` traces, computes per-rule metrics, writes `insights.md`, updates `stats` in `rules.json`.

```
Per-rule metrics:
  request_rate          — sessions where rule was requested / total sessions
  apply_rate            — times applied / times requested
  confident_wrong_rate  — "used own knowledge" but code violates rule

Rule health:
  EFFECTIVE   — high request_rate + high apply_rate
  UNDERUSED   — low request_rate (discovery problem)
  IGNORED     — high request_rate + low apply_rate (wording problem)
  HARMFUL     — agents apply then revert, or conflicts with codebase
```

Stats in `rules.json` updated by CLI only — never by MCP server at runtime.

---

## Part 6: Hook Integration

### PostToolUse hook (new — implicit commit detection)

Currently Winkers only has SessionEnd hooks. A PostToolUse hook must be added.
`winkers init` installs it in `.claude/settings.json` alongside existing SessionEnd hooks.

```python
# Fires after agent Write/Edit tool calls
def post_tool_use_conventions(file_path, file_content, session_id):
    rules_read = read_session_trace(session_id)  # which rules were read via rule_read()

    for rule in load_rules():
        if not applies_to_file(rule, file_path):
            continue
        was_read = rule.id in rules_read
        follows  = check_positive(rule, file_content)
        violates = check_negative(rule, file_content)

        if violates and not was_read:
            log_event("confident_wrong", rule.id, file_path)
        elif violates and was_read:
            log_event("implicit_violation", rule.id, file_path)
        elif follows and was_read:
            log_event("implicit_commit", rule.id, file_path)
```

### SessionEnd hook (extend existing)

Existing SessionEnd hook runs `winkers record --hook`.
Extend to also run `winkers conventions analyze --session <session_id>` after recording.

### Three cycles, non-blocking

```
Fast (seconds):   agent → orient/rule_read/write → PostToolUse hook observes
                  ↓ (SessionEnd)
Medium (minutes): analyze updates insights.md + stats in rules.json
                  ↓ (ready before next session)
Slow (weekly):    winkers conventions suggest → review → apply
                  ↓
Next session:     agent sees updated orient() response
```

---

## Part 7: `winkers conventions suggest`

LLM-based (Haiku by default). Needs trace data — implement last.

```
Input:  rules.json + insights.md + last N session traces (summarized)
Output: suggestions with [Accept / Edit / Skip] per suggestion
        - Rules to REMOVE (harmful or consistently ignored)
        - Rules to REWRITE (unclear wording)
        - Rules to ADD (recurring agent mistakes not covered)
        - Rules to PROMOTE to overview.md
Cost: ~$0.01 per run with Haiku
```

---

## Part 8: Migration (existing users only)

`winkers conventions migrate-from-semantic`

For users who already have `semantic.json` with conventions/constraints before this feature existed.
Scans `semantic.json`, proposes converting entries to `rules.json` with `source: "migrated-from-semantic"`.

New projects: no migration needed — SemanticEnricher writes to `rules.json` directly from first `winkers init`.

Fields `conventions[]` and `constraints[]` are removed from `SemanticLayer` and from the
`map` and `scope` MCP tool outputs. Rules are now the only channel for this information.

---

## Implementation Order (strict)

1. **Data model** — `rules.json` schema, `RulesStore`, `overview.md` generator, trace JSONL format
2. **MCP tools** — `orient()` + `rule_read()`, trace logging, wire into existing server
3. **Rule sources** — SemanticEnricher extension + tree-sitter detectors + reconcile logic, wire into `winkers init`
4. **CLI CRUD** — `conventions list`, `conventions add`, `conventions compile`
5. **Trace analyzer** — `conventions analyze`, per-rule metrics, `insights.md`, stats update
6. **Hook integration** — PostToolUse hook (new), extend SessionEnd hook
7. **Suggest** — `conventions suggest`, LLM-based proposals
8. **Migration** — `conventions migrate-from-semantic` (existing users only)

## Key Constraints

- Convention tools MUST NOT inject rule content into prompt in gated mode — MCP is the only channel
- In hybrid mode, `overview.md` is injected but rule details still require MCP calls
- `traces/` is gitignored, `rules.json` and `overview.md` are committed
- Auto-detection and semantic agent propose — human accepts. Never auto-write without confirmation
- Only `source != "manual"` rules are candidates for auto-deletion proposals
- Stats updated by CLI analysis only, not by MCP server (avoids write conflicts)
- `overview.md` respects token budget — prioritized by rule health
- All CLI commands follow existing `winkers` patterns and support `--help`
