---
phase: 15-event-model-hardening
plan: 03
type: execute
wave: 3
depends_on: ["15-01", "15-02"]
files_modified:
  - src/ztlctl/services/check.py
  - src/ztlctl/plugins/event_bus.py
  - src/ztlctl/infrastructure/vault.py
  - src/ztlctl/actions/_register_core.py
  - src/ztlctl/controllers/check.py
  - tests/plugins/test_event_bus.py
  - tests/services/test_check.py
autonomous: true
requirements: [DEBT-03]

must_haves:
  truths:
    - "Dead-letter events are reported in ztlctl check output under CAT_STRUCTURAL at info severity"
    - "Startup drain auto-purges dead-letter events older than dead_letter_retention_days"
    - "event_purge action is registered in ActionRegistry and clears dead-letter events"
  artifacts:
    - path: "src/ztlctl/services/check.py"
      provides: "Dead-letter count in structural validation"
      contains: "dead_letter"
    - path: "src/ztlctl/plugins/event_bus.py"
      provides: "purge_dead_letters method"
      contains: "purge_dead_letters"
    - path: "src/ztlctl/actions/_register_core.py"
      provides: "event_purge ActionDefinition"
      contains: "event_purge"
  key_links:
    - from: "src/ztlctl/services/check.py"
      to: "src/ztlctl/infrastructure/database/schema.py"
      via: "query event_wal for dead_letter rows"
      pattern: "event_wal.*dead_letter"
    - from: "src/ztlctl/infrastructure/vault.py"
      to: "src/ztlctl/plugins/event_bus.py"
      via: "startup drain calls purge_dead_letters"
      pattern: "purge_dead_letters"
---

<objective>
Implement dead-letter event observability: reporting in `ztlctl check`, auto-purge during startup, and manual `event_purge` action. Resolves DEBT-03 (dead-letter accumulation).

Purpose: Dead-letter events currently accumulate silently in the WAL with no visibility or cleanup. This plan makes them observable (check output), auto-managed (startup purge), and manually clearable (event_purge action).

Output: Dead-letter reporting in CheckService, purge_dead_letters on EventBus, auto-purge in startup drain, event_purge action registered, all tests passing.
</objective>

<execution_context>
@~/.claude/get-shit-done/workflows/execute-plan.md
@~/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/15-event-model-hardening/15-CONTEXT.md
@.planning/phases/15-event-model-hardening/15-RESEARCH.md
@.planning/phases/15-event-model-hardening/15-01-SUMMARY.md

<interfaces>
<!-- From Plan 01 outputs -->

From src/ztlctl/config/models.py (Plan 01):
```python
class EventBusConfig(BaseModel):
    model_config = {"frozen": True}
    shutdown_timeout_seconds: float = 5.0
    max_retries: int = 3
    dead_letter_retention_days: int = 30
    per_future_timeout_seconds: float = 30.0
```

From src/ztlctl/plugins/event_bus.py (Plan 01 refactored):
```python
# self._dead_letter_retention_days stored from config
```

From src/ztlctl/services/check.py (existing):
```python
CAT_STRUCTURAL = "structural_validation"
SEVERITY_ERROR = "error"
SEVERITY_WARNING = "warning"
# No SEVERITY_INFO constant yet
def _check_structural_validation(self, conn: Connection) -> list[dict[str, Any]]:
```

From src/ztlctl/infrastructure/database/schema.py (existing):
```python
event_wal = Table("event_wal", metadata,
    Column("id", Integer, primary_key=True),
    Column("hook_name", Text, nullable=False),
    Column("payload", Text, nullable=False),
    Column("status", Text, nullable=False),  # pending, completed, failed, dead_letter
    Column("retries", Integer, default=0),
    Column("error", Text),
    Column("session_id", Text),
    Column("created", Text),
    Column("completed", Text),
)
```

From src/ztlctl/actions/_register_core.py (pattern):
```python
from ztlctl.actions.definitions import ActionDefinition, ActionParam
from ztlctl.actions.registry import get_action_registry
registry = get_action_registry()
registry.register(ActionDefinition(
    name="...", category="...", description="...",
    handler=ControllerClass.method_name,
    params=[ActionParam(name="...", type="...", required=..., description="...")],
))
```
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Dead-letter reporting in CheckService and auto-purge</name>
  <files>
    src/ztlctl/services/check.py,
    src/ztlctl/plugins/event_bus.py,
    src/ztlctl/infrastructure/vault.py,
    tests/plugins/test_event_bus.py,
    tests/services/test_check.py
  </files>
  <read_first>
    src/ztlctl/services/check.py,
    src/ztlctl/plugins/event_bus.py,
    src/ztlctl/infrastructure/vault.py,
    src/ztlctl/infrastructure/database/schema.py,
    tests/plugins/test_event_bus.py,
    tests/services/test_check.py
  </read_first>
  <behavior>
    - Test: When event_wal has 3 dead_letter rows, _check_structural_validation returns an issue with category=CAT_STRUCTURAL, severity="info", message containing "3 dead-letter"
    - Test: When event_wal has 0 dead_letter rows, no dead-letter issue is returned
    - Test: purge_dead_letters(older_than_days=30) deletes dead_letter rows with created date > 30 days ago, returns count deleted
    - Test: purge_dead_letters(older_than_days=30) does NOT delete dead_letter rows from today
    - Test: Startup drain calls purge_dead_letters before draining pending events
  </behavior>
  <action>
    **Part A — SEVERITY_INFO constant (per D-19):**

    1. Add `SEVERITY_INFO = "info"` constant to `src/ztlctl/services/check.py` alongside existing `SEVERITY_ERROR` and `SEVERITY_WARNING`.

    **Part B — Dead-letter reporting in CheckService (per D-19):**

    2. At the end of `_check_structural_validation()` method in `src/ztlctl/services/check.py`, add a dead-letter count query:
       ```python
       from ztlctl.infrastructure.database.schema import event_wal
       from sqlalchemy import func

       dead_letter_count = conn.execute(
           select(func.count()).select_from(event_wal).where(
               event_wal.c.status == "dead_letter"
           )
       ).scalar_one()

       if dead_letter_count > 0:
           issues.append({
               "category": CAT_STRUCTURAL,
               "severity": SEVERITY_INFO,
               "message": (
                   f"{dead_letter_count} dead-letter event(s) in WAL. "
                   "Run 'ztlctl event purge' to clear."
               ),
           })
       ```

    **Part C — purge_dead_letters on EventBus (per D-20):**

    3. Add `purge_dead_letters()` method to `EventBus` in `src/ztlctl/plugins/event_bus.py`:
       ```python
       def purge_dead_letters(self, *, older_than_days: int | None = None) -> int:
           """Delete dead-letter WAL rows older than N days. Returns count deleted.

           If older_than_days is None, uses self._dead_letter_retention_days from config.
           """
           from datetime import datetime, timedelta, timezone

           retention = older_than_days if older_than_days is not None else self._dead_letter_retention_days
           cutoff = (datetime.now(timezone.utc) - timedelta(days=retention)).isoformat()

           with self._engine.begin() as conn:
               # Count before delete for return value
               count = conn.execute(
                   select(func.count()).select_from(event_wal).where(
                       event_wal.c.status == "dead_letter",
                       event_wal.c.created < cutoff,
                   )
               ).scalar_one()

               if count > 0:
                   conn.execute(
                       event_wal.delete().where(
                           event_wal.c.status == "dead_letter",
                           event_wal.c.created < cutoff,
                       )
                   )

           return count
       ```
       Add necessary imports at the top of the file: `from sqlalchemy import delete, func, insert, select, update` (some already exist).

    **Part D — Auto-purge during startup drain (per D-20):**

    4. In `Vault.init_event_bus()` in `src/ztlctl/infrastructure/vault.py`, add auto-purge before the existing startup drain (added by Plan 02):
       ```python
       # Auto-purge old dead-letter events (D-20)
       try:
           purged = self._event_bus.purge_dead_letters()
           if purged > 0:
               logger.info("Purged %d old dead-letter event(s) from WAL", purged)
       except Exception:
           logger.warning("Dead-letter purge failed; continuing", exc_info=True)

       # Startup recovery: drain pending/failed events from prior runs (D-05)
       # (already added by Plan 02)
       ```
       Note: If Plan 02 hasn't been executed yet (parallel wave), the startup drain line may not exist yet. Add both the purge and the drain call — the drain call placement is the same as what Plan 02 specifies.

    **Part E — Tests:**

    5. Add dead-letter tests to `tests/plugins/test_event_bus.py`:
       - `test_purge_dead_letters_removes_old_entries`: Insert dead_letter WAL rows with old dates, call purge_dead_letters, verify they're gone.
       - `test_purge_dead_letters_keeps_recent_entries`: Insert dead_letter WAL row with today's date, call purge_dead_letters(older_than_days=30), verify it remains.

    6. Add dead-letter check test to `tests/services/test_check.py`:
       - `test_check_reports_dead_letter_events`: Insert dead_letter WAL rows, run check, verify output contains dead-letter info issue.
  </action>
  <verify>
    <automated>cd /Users/shparki/Documents/Workspace/thatdev/ztlctl && uv run pytest tests/plugins/test_event_bus.py -k dead_letter -x -q --timeout=30 && uv run pytest tests/services/test_check.py -k dead_letter -x -q --timeout=30</automated>
  </verify>
  <acceptance_criteria>
    - src/ztlctl/services/check.py contains `SEVERITY_INFO = "info"`
    - src/ztlctl/services/check.py contains `dead_letter` in _check_structural_validation method
    - src/ztlctl/services/check.py contains `event_wal` import
    - src/ztlctl/plugins/event_bus.py contains `def purge_dead_letters(`
    - src/ztlctl/plugins/event_bus.py purge_dead_letters contains `dead_letter_retention_days`
    - src/ztlctl/infrastructure/vault.py contains `purge_dead_letters`
    - tests/plugins/test_event_bus.py -k dead_letter exits 0
    - tests/services/test_check.py -k dead_letter exits 0
  </acceptance_criteria>
  <done>Dead-letter events are visible in check output. Old dead-letters auto-purge at startup. purge_dead_letters method exists on EventBus.</done>
</task>

<task type="auto">
  <name>Task 2: Register event_purge action in ActionRegistry</name>
  <files>
    src/ztlctl/controllers/check.py,
    src/ztlctl/actions/_register_core.py,
    tests/controllers/test_check_controller.py
  </files>
  <read_first>
    src/ztlctl/controllers/check.py,
    src/ztlctl/actions/_register_core.py,
    src/ztlctl/actions/definitions.py,
    src/ztlctl/services/result.py
  </read_first>
  <action>
    Per D-21: Register `event_purge` as an action in ActionRegistry for manual dead-letter cleanup.

    1. Add `event_purge()` method to `CheckController` in `src/ztlctl/controllers/check.py`:
       ```python
       def event_purge(self, *, older_than_days: int | None = None) -> ServiceResult:
           """Purge dead-letter events from the WAL.

           Deletes dead-letter rows older than older_than_days (defaults to
           config dead_letter_retention_days). Returns count purged.
           """
           from ztlctl.services.result import ServiceResult

           bus = self._vault.event_bus
           if bus is None:
               return ServiceResult(
                   ok=False,
                   op="event_purge",
                   message="Event bus not initialized",
               )

           days = older_than_days if older_than_days is not None else self._vault.settings.eventbus.dead_letter_retention_days
           count = bus.purge_dead_letters(older_than_days=days)

           return ServiceResult(
               ok=True,
               op="event_purge",
               message=f"Purged {count} dead-letter event(s) from WAL",
               data={"purged_count": count, "older_than_days": days},
           )
       ```

    2. Register the action in `src/ztlctl/actions/_register_core.py` inside `_register_core_actions()`. Add after the existing check actions:
       ```python
       registry.register(
           ActionDefinition(
               name="event_purge",
               category="maintenance",
               description="Purge dead-letter events from the event WAL",
               handler=CheckController.event_purge,
               params=[
                   ActionParam(
                       name="older_than_days",
                       type="integer",
                       required=False,
                       description="Delete dead-letter events older than N days (default: config value)",
                   ),
               ],
           )
       )
       ```

    3. Add a basic test for the controller method (either in existing check controller test file or a new one). The test should verify that `event_purge()` returns a ServiceResult with `ok=True` and correct data shape when dead-letter rows exist.
  </action>
  <verify>
    <automated>cd /Users/shparki/Documents/Workspace/thatdev/ztlctl && uv run pytest tests/ -k "event_purge" -x -q --timeout=30 && uv run pytest tests/ -x -q --timeout=60</automated>
  </verify>
  <acceptance_criteria>
    - src/ztlctl/controllers/check.py contains `def event_purge(`
    - src/ztlctl/controllers/check.py contains `purge_dead_letters`
    - src/ztlctl/actions/_register_core.py contains `name="event_purge"`
    - src/ztlctl/actions/_register_core.py contains `category="maintenance"`
    - Full test suite `uv run pytest tests/ -x -q --timeout=60` exits 0
  </acceptance_criteria>
  <done>event_purge action registered in ActionRegistry under maintenance category. CheckController.event_purge method delegates to EventBus.purge_dead_letters. Full suite green.</done>
</task>

</tasks>

<verification>
- `uv run pytest tests/plugins/test_event_bus.py -k dead_letter -x -q` — dead-letter purge tests pass
- `uv run pytest tests/services/test_check.py -k dead_letter -x -q` — check reporting test passes
- `uv run pytest tests/ -k event_purge -x -q` — action registration test passes
- `uv run pytest tests/ -x -q --timeout=60` — full suite green
- `uv run ruff check src/` — no lint errors
- `uv run mypy src/` — no type errors
</verification>

<success_criteria>
- Dead-letter events appear in `ztlctl check` output at info severity (DEBT-03 observability)
- Old dead-letters auto-purge during startup (DEBT-03 accumulation prevention)
- `event_purge` action is registered and callable (DEBT-03 manual cleanup)
- EventBus.purge_dead_letters respects dead_letter_retention_days config
- SEVERITY_INFO constant added to CheckService
- All tests pass including full regression suite
</success_criteria>

<output>
After completion, create `.planning/phases/15-event-model-hardening/15-03-SUMMARY.md`
</output>
