---
phase: 08-retroactive-phase-verification
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - .planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md
autonomous: true
requirements:
  - CLUS-01
  - CLUS-02
  - CLUS-03
  - CLUS-04
  - CLUS-05
  - CLUS-06
  - CLUS-07

must_haves:
  truths:
    - "02-VERIFICATION.md exists in .planning/phases/02-core-vm-lifecycle/"
    - "YAML frontmatter includes phase, verified, status, score, and gaps fields"
    - "All 5 roadmap success criteria for Phase 2 have a row with status VERIFIED or human_needed"
    - "Each SC row cites an implementation file (with line reference), a test file, and a test name"
    - "Requirements coverage table maps CLUS-01 through CLUS-07 to plans and test counts"
    - "Per-plan must-have truth tables are present (one section per plan 02-01 through 02-04)"
  artifacts:
    - path: ".planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md"
      provides: "Phase 2 verification report with 3-source evidence"
      contains: "CLUS-01"
  key_links:
    - from: "02-VERIFICATION.md SC table"
      to: "vmcluster_mcp/cluster/__init__.py"
      via: "file:line evidence citations"
      pattern: "cluster/__init__"
    - from: "02-VERIFICATION.md requirements table"
      to: "CLUS-01..07"
      via: "requirements coverage section"
      pattern: "CLUS-0[1-7]"
---

<objective>
Audit the Phase 02 (Core VM Lifecycle) codebase against the roadmap success criteria and write the missing 02-VERIFICATION.md report. The report provides the 3-source evidence required to mark CLUS-01 through CLUS-07 as satisfied at milestone level.

Purpose: Close the audit gap that leaves CLUS-01..07 orphaned in the v1.0 milestone audit. Phase 2 was fully implemented and tested but no verification report was written.
Output: `.planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md`
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md

@.planning/phases/02-core-vm-lifecycle/02-01-SUMMARY.md
@.planning/phases/02-core-vm-lifecycle/02-02-SUMMARY.md
@.planning/phases/02-core-vm-lifecycle/02-03-SUMMARY.md
@.planning/phases/02-core-vm-lifecycle/02-04-SUMMARY.md

<!-- Canonical VERIFICATION.md format reference -->
@.planning/phases/07-network-node-fault-injection/07-VERIFICATION.md

<interfaces>
<!-- Key source locations confirmed in codebase. Use these as evidence citations. -->

vmcluster_mcp/cluster/libvirt_client.py:
  - Line 19: `_executor = ThreadPoolExecutor(max_workers=1)`
  - Line 35: `def domain_name(cluster_name: str, node_name: str) -> str`
  - Line 40: `def generate_mac(cluster_name: str, node_name: str) -> str` — returns 52:54:00:xx:xx:xx
  - Line 52: `def map_domain_state(state_int: int) -> str`
  - Line 62: `class LibvirtClient`
  - Line 76: `async def _run(self, func, *args)` — dispatches via _executor
  - Line 88: `async def lookup_domain(self, name: str) -> libvirt.virDomain | None`
  - Line 129: `async def lookup_network(self, name: str) -> libvirt.virNetwork | None`

vmcluster_mcp/cluster/__init__.py:
  - Line 35: `@mcp.tool()` / `@structured_tool_handler` / `async def cluster_define`
  - Line 59: `existing_dom = await _client.lookup_domain(first_dom_name)` — idempotency guard
  - Line 92: `overlay_dir = os.path.join(config.overlay_dir, topo.cluster_name)` — namespaced overlays
  - Line 111: `overlay_path = os.path.join(overlay_dir, f"{node.name}.qcow2")`
  - Line 114: `assert os.path.abspath(topo.base_image) != os.path.abspath(overlay_path)` — safety assert
  - Line 232: `async def cluster_start` — boots all nodes, SSH wait via asyncio.gather
  - Line 336: `async def cluster_stop` — shutdown or destroy mode
  - Line 398: `async def cluster_destroy` — undefines network + optional overlay removal (line 449–452)
  - Line 469: `async def cluster_status` — per-node domain state + SSH reachability in parallel
  - Line 542: `async def cluster_handle` — returns ClusterHandle with NodeDescriptor list

Test files and counts (all currently passing):
  - tests/test_libvirt_client.py — 31 tests
  - tests/test_builders.py — 17 tests
  - tests/test_cloud_init.py — 5 tests
  - tests/test_cluster_define.py — 4 tests
  - tests/test_cluster_lifecycle.py — 13 tests
  - tests/test_cluster_status.py — 13 tests (7 cluster_status + 6 cluster_handle)
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Audit Phase 02 codebase and write 02-VERIFICATION.md</name>
  <files>.planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md</files>
  <read_first>
    - .planning/phases/07-network-node-fault-injection/07-VERIFICATION.md (canonical format — replicate YAML frontmatter schema, table structure, section headings exactly)
    - .planning/phases/02-core-vm-lifecycle/02-01-SUMMARY.md
    - .planning/phases/02-core-vm-lifecycle/02-02-SUMMARY.md
    - .planning/phases/02-core-vm-lifecycle/02-03-SUMMARY.md
    - .planning/phases/02-core-vm-lifecycle/02-04-SUMMARY.md
    - vmcluster_mcp/cluster/__init__.py (confirm line numbers for SC evidence)
    - vmcluster_mcp/cluster/libvirt_client.py (confirm ThreadPoolExecutor line)
    - tests/test_cluster_define.py (confirm idempotency test names)
    - tests/test_cluster_lifecycle.py (confirm start/stop/destroy test names)
    - tests/test_cluster_status.py (confirm status/handle test names)
  </read_first>
  <action>
Run these verification commands first and record the outputs:

```
cd /home/hornc/repos/vmcluster-mcp
.venv/bin/python -m pytest tests/test_libvirt_client.py tests/test_builders.py tests/test_cloud_init.py tests/test_cluster_define.py tests/test_cluster_lifecycle.py tests/test_cluster_status.py -q --no-header
.venv/bin/python -m ruff check vmcluster_mcp/cluster/
```

Then write `.planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md` using the exact schema from 07-VERIFICATION.md:

**YAML frontmatter** (required fields, same keys as 07):
```yaml
---
phase: 02-core-vm-lifecycle
verified: 2026-03-23T00:00:00Z
status: passed
score: 5/5 roadmap success criteria verified; all must_have truths resolved
gaps: []
human_verification:
  - test: "SC-2 cluster_start SSH wait blocking behaviour"
    expected: "With wait_for_ssh=True, cluster_start blocks until all nodes SSH-reachable and returns time waited"
    why_human: "Requires live VMs with real SSH endpoints; unit tests mock asyncssh.connect"
  - test: "SC-3 cluster_status live SSH reachability"
    expected: "cluster_status returns ssh_reachable=True for running nodes and False for stopped nodes"
    why_human: "Requires live VMs; unit tests mock the SSH check"
---
```

**Phase Goal** header line and Verified/Status metadata identical to 07 format.

**Observable Truths table** — map each roadmap SC to code evidence + test evidence:

| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| SC-1 | `cluster_define` creates libvirt domain definitions and cluster-namespaced overlay qcow2 disks; second call returns success without modifying existing domains | ✓ VERIFIED | `cluster/__init__.py:59` idempotency guard (`lookup_domain` before define); `cluster/__init__.py:92` namespaced overlay path; `assert base_image != overlay_path` at line 114; `test_cluster_define.py` 4 tests |
| SC-2 | `cluster_start` boots all nodes; with `wait_for_ssh=True`, blocks until SSH-reachable and returns time waited | ✓ VERIFIED | `cluster/__init__.py:232`; `_wait_for_ssh` uses `asyncssh.connect`; `asyncio.gather` parallel SSH wait; `test_cluster_lifecycle.py`: test_cluster_start_ssh_success_sets_reachable, test_cluster_start_ssh_timeout_returns_failure |
| SC-3 | `cluster_status` returns domain state (running, shut off, etc.) and SSH reachability for each node | ✓ VERIFIED | `cluster/__init__.py:469`; parallel SSH check via `asyncio.gather` for running nodes only; returns `ToolResult[ClusterStatus]` with per-node `NodeStatus`; `test_cluster_status.py` 7 tests |
| SC-4 | `cluster_stop` and `cluster_destroy` cleanly remove cluster resources; `cluster_destroy` with `remove_overlays=True` deletes overlay disks | ✓ VERIFIED | `cluster/__init__.py:336` (cluster_stop, shutdown+destroy modes); `cluster/__init__.py:398` (cluster_destroy, always removes network, shutil.rmtree at line 449–452); `test_cluster_lifecycle.py` 8 tests |
| SC-5 | `cluster_handle` returns `ClusterHandle` with node names, IPs, SSH credentials for downstream use | ✓ VERIFIED | `cluster/__init__.py:542`; returns `ToolResult[ClusterHandle]` with `NodeDescriptor` list + `kernel_version`; `test_cluster_status.py` 6 cluster_handle tests |

**Required Artifacts table** — list all created files with actual line counts from summaries:

| Artifact | Status | Notes |
|----------|--------|-------|
| `vmcluster_mcp/cluster/libvirt_client.py` | ✓ VERIFIED | LibvirtClient, ThreadPoolExecutor(max_workers=1), 31 tests |
| `vmcluster_mcp/cluster/domain_builder.py` | ✓ VERIFIED | ElementTree XML builder for KVM domains |
| `vmcluster_mcp/cluster/network_builder.py` | ✓ VERIFIED | NAT network XML, no DHCP block |
| `vmcluster_mcp/cluster/cloud_init.py` | ✓ VERIFIED | NoCloud ISO via genisoimage/mkisofs |
| `vmcluster_mcp/cluster/__init__.py` | ✓ VERIFIED | 6 tools: cluster_define/start/stop/destroy/status/handle |
| `tests/test_libvirt_client.py` | ✓ VERIFIED | 31 tests |
| `tests/test_builders.py` | ✓ VERIFIED | 17 tests |
| `tests/test_cloud_init.py` | ✓ VERIFIED | 5 tests |
| `tests/test_cluster_define.py` | ✓ VERIFIED | 4 tests |
| `tests/test_cluster_lifecycle.py` | ✓ VERIFIED | 13 tests |
| `tests/test_cluster_status.py` | ✓ VERIFIED | 13 tests |

**Key Link Verification table** — confirm critical wiring:

| From | To | Via | Status |
|------|----|-----|--------|
| `cluster/__init__.py` | `libvirt_client.py` | `_client = LibvirtClient()` + all async calls via `await _client.*` | ✓ WIRED |
| `cluster/__init__.py` | `domain_builder.py` | `from vmcluster_mcp.cluster.domain_builder import build_domain_xml` | ✓ WIRED |
| `cluster/__init__.py` | `network_builder.py` | `from vmcluster_mcp.cluster.network_builder import build_network_xml` | ✓ WIRED |
| `cluster/__init__.py` | `cloud_init.py` | `from vmcluster_mcp.cluster.cloud_init import generate_cloud_init_iso` | ✓ WIRED |
| `cluster/__init__.py` | `server.py` | `from vmcluster_mcp import cluster` triggers `@mcp.tool()` registration | ✓ WIRED |
| `libvirt_client.py` | `ThreadPoolExecutor` | `_executor = ThreadPoolExecutor(max_workers=1)` at module level | ✓ WIRED |

**Per-Plan Must-Have Truth tables** — one section for each of plans 02-01 through 02-04, replicating the format from 07-VERIFICATION.md. Include the same column headings: `Truth | Status | Evidence`. Derive truths from each plan's SUMMARY.md self-check list.

**Requirements Coverage table**:

| Requirement | Description | Plan | Status | Evidence |
|-------------|-------------|------|--------|----------|
| CLUS-01 | Agent can define a cluster from a topology file | 02-02 | ✓ SATISFIED | cluster_define + 4 tests |
| CLUS-02 | Agent can start all nodes and optionally wait for SSH | 02-03 | ✓ SATISFIED | cluster_start + 5 tests |
| CLUS-03 | Agent can stop all nodes (graceful or forcibly) | 02-03 | ✓ SATISFIED | cluster_stop + 3 tests |
| CLUS-04 | Agent can destroy cluster and optionally remove overlays | 02-03 | ✓ SATISFIED | cluster_destroy + 5 tests |
| CLUS-05 | Agent can query current status of all nodes | 02-04 | ✓ SATISFIED | cluster_status + 7 tests |
| CLUS-06 | Agent can obtain a ClusterHandle for downstream SSH consumers | 02-04 | ✓ SATISFIED | cluster_handle + 6 tests |
| CLUS-07 | cluster_define is idempotent | 02-02 | ✓ SATISFIED | lookup_domain guard at line 59; test_cluster_define.py |

**Lightweight Verification Run** section — paste the actual pytest output from the commands run at start.

**Anti-Patterns Found** section — note no TODOs/FIXMEs/stubs found; note D-08 (single-worker executor) is correctly applied.

**Human Verification** section — one entry each for SC-2 and SC-3 explaining what live-VM testing would confirm.
  </action>
  <verify>
    <automated>grep -l "CLUS-01" /home/hornc/repos/vmcluster-mcp/.planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md && grep "phase: 02-core-vm-lifecycle" /home/hornc/repos/vmcluster-mcp/.planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md && grep "CLUS-07" /home/hornc/repos/vmcluster-mcp/.planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md && grep "SC-1\|SC-2\|SC-3\|SC-4\|SC-5" /home/hornc/repos/vmcluster-mcp/.planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md | grep "VERIFIED" | wc -l</automated>
  </verify>
  <acceptance_criteria>
    - `.planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md` exists
    - File contains `phase: 02-core-vm-lifecycle` in YAML frontmatter
    - File contains `status:` field in YAML frontmatter
    - File contains `score:` field in YAML frontmatter
    - File contains `CLUS-01` through `CLUS-07` — all 7 requirement IDs present
    - File contains at least 5 rows with `VERIFIED` in the SC table
    - File contains `human_verification:` block in frontmatter
    - `grep "02-01\|02-02\|02-03\|02-04" 02-VERIFICATION.md` returns at least 4 matches (per-plan sections)
    - File contains `ThreadPoolExecutor` as evidence citation
    - File contains `test_cluster_lifecycle` as evidence citation
  </acceptance_criteria>
  <done>02-VERIFICATION.md exists, all 5 roadmap SCs are VERIFIED with 3-source evidence (code location + test file + test name), CLUS-01..07 requirements table is present and all marked SATISFIED, per-plan must-have truth tables are present for all 4 plans.</done>
</task>

</tasks>

<verification>
After completion:
- `ls .planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md` — file exists
- `grep "CLUS-0[1-7]" .planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md | wc -l` — returns 7 or more
- `grep "SATISFIED" .planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md | wc -l` — returns 7
- `grep "VERIFIED" .planning/phases/02-core-vm-lifecycle/02-VERIFICATION.md | wc -l` — returns at least 5 (SC rows)
</verification>

<success_criteria>
- 02-VERIFICATION.md written and follows the exact schema of 07-VERIFICATION.md
- All 5 Phase 2 roadmap success criteria verified with 3-source evidence
- All CLUS-01 through CLUS-07 requirements marked SATISFIED in requirements coverage table
- Per-plan must-have truth tables present for all 4 plans (02-01 through 02-04)
- Human verification section documents which SCs require live-VM testing
</success_criteria>

<output>
After completion, create `.planning/phases/08-retroactive-phase-verification/08-01-SUMMARY.md`
</output>
