# Phase 5: Artifact Pipeline - Context

**Gathered:** 2026-03-23
**Status:** Ready for planning

<domain>
## Phase Boundary

Implement the full artifact pipeline — `artifact_register`, `artifact_sync`,
`artifact_install`, `artifact_list`, and `artifact_diff` — as content-addressed
build distribution tools. An agent can register a local build tree, push it to
cluster nodes with deduplication, install it using the appropriate method, and
compare two artifact versions to see what changed.

Requirements: ART-01, ART-02, ART-03, ART-04, ART-05.

Note: `artifact_build` is explicitly out of scope for v1 (deferred to ART-V2-01;
tracked in REQUIREMENTS.md). This phase does NOT implement the local build trigger.

</domain>

<decisions>
## Implementation Decisions

### Registry Storage Format
- **D-01:** Registry JSON is a **dict keyed by `artifact_id`** mapping to the
  full `BuildArtifact` model serialized as a dict. Layout:
  ```json
  {
    "<sha256>": { "artifact_id": "...", "source_path": "...", ... }
  }
  ```
  O(1) lookup by artifact_id. Deduplication is a key-exists check.
  File path: `config.artifact_registry` (default `/var/lib/vmcluster/artifacts/registry.json`).

- **D-02:** All registry writes use the **atomic write pattern established in STATE.md**:
  write to `{registry_path}.tmp` then `os.replace()` to the target path.
  Consistent with the phase 4 disk-based state pattern. The registry directory
  must exist before write; planner should ensure `artifact_register` creates it
  if missing.

### Content Hashing
- **D-03:** `artifact_id` = sha256 of the build tree content computed by:
  1. Walk the tree, collect all file paths (sorted, relative, excluding
     `__pycache__`, `.git`)
  2. For each file in sorted order: sha256(file contents)
  3. Concatenate all per-file hashes, then sha256 the concatenation
  This is deterministic across runs on the same content. Researcher confirms
  or adjusts the exact algorithm — planner must use whatever the researcher
  establishes as canonical.

### rsync Invocation
- **D-04:** Artifact sync uses **subprocess rsync** with SSH transport:
  ```
  rsync -avz -e "ssh -i {ssh_key_path} -o StrictHostKeyChecking=no"
        {local_tree}/  {ssh_user}@{node_ip}:{dest_base}/{artifact_id}/
  ```
  This is the direct approach the design doc names explicitly. No asyncssh
  SCP or Python rsync library. The asyncssh connection from Phase 3 is NOT
  re-used for the rsync tunnel — subprocess rsync controls its own SSH session.

- **D-05:** rsync runs **concurrently across nodes** using `asyncio.gather` with
  `asyncio.create_subprocess_exec` (async subprocess) — consistent with Phase 3
  fan-out pattern. Each node's rsync runs in a separate async subprocess; results
  are collected into `SyncNodeResult` per node.

### Node Deduplication Check
- **D-06:** Before running rsync for a node, check whether `{dest_base}/{artifact_id}/`
  directory already exists on the remote node via an SSH `test -d` command using
  the Phase 3 asyncssh infrastructure. If it exists and `force=False`, mark the
  node as `skipped=True` in `SyncNodeResult`. No server-side per-node tracking —
  stateless principle from PROJECT.md.

### ArtifactDiff Model
- **D-07:** `ArtifactDiff` is not yet in `models.py` and must be added:
  ```python
  class ArtifactDiff(BaseModel):
      artifact_id_a: str
      artifact_id_b: str
      changed_modules: list[str]    # .ko filenames that differ by hash
      changed_bins: list[str]       # userspace binary filenames that differ
      git_log: str | None           # `git log a_sha..b_sha` if both have git_sha
  ```
  The diff is computed locally by reading the registered `BuildArtifact` entries
  from the registry and comparing their `modules` and `userspace_bins` lists by
  content hash (re-hash each file in both `source_path` trees at diff time).

### Partial Failure Semantics
- **D-08:** `artifact_sync` per-node failures: **best-effort with full per-node
  reporting** — a failed rsync on one node does not abort others. `SyncStatus.per_node`
  reports exit codes for all nodes. Consistent with Phase 4 `snapshot_delete` (D-08).

- **D-09:** `artifact_install` per-node failures: **best-effort with full per-node
  reporting** — consistent with D-08 above. `InstallStatus.per_node` reports exit
  codes and installed paths for all nodes. An agent can retry failed nodes.

### install_mode Implementation
- **D-10:** `modprobe_path` (default): writes `/etc/modprobe.d/vmcluster-test.conf`
  on each node via SSH, pointing at `{dest_base}/{artifact_id}/` module paths.
  Runs `depmod -a` after writing. Does not touch system paths — trivially reverted
  by deleting the conf file. **Recommended default for test clusters.**

- **D-11:** `make_install`: SSH into synced tree dir, run `make install`. Writes
  to system paths.

- **D-12:** `rpm`/`deb`: Find package files within synced tree, install via
  `rpm -i` or `dpkg -i`. Researcher determines how to locate the package files
  (glob for `*.rpm` or `*.deb` within `{dest_base}/{artifact_id}/`).

### Concurrency Model
- **D-13:** All per-node artifact operations (`artifact_sync`, `artifact_install`)
  use `asyncio.gather` for parallel fan-out — same pattern as Phase 3 `node_exec_all`
  and Phase 4 snapshot operations. No libvirt calls involved, so the single-worker
  `ThreadPoolExecutor` is NOT needed here.

### Carried Forward (from prior phases)
- **D-14:** `@structured_tool_handler` wraps every tool handler — no undecorated tools.
- **D-15:** Pydantic v2 models for all return types — no dicts returned to agents.
  `BuildArtifact`, `SyncNodeResult`, `SyncStatus`, `InstallNodeResult`, `InstallStatus`
  already defined in `vmcluster_mcp/models.py`; `ArtifactDiff` must be added (D-07).
- **D-16:** Logging via `logging.getLogger("vmcluster_mcp.artifact")` — consistent
  module-level logger naming.
- **D-17:** `from __future__ import annotations` in every new module.

### Agent's Discretion
- Exact rsync flags beyond `-avz` (e.g., `--delete`, `--checksum`) — researcher
  determines what is appropriate for a build tree sync.
- Whether per-file hashing in `artifact_diff` re-reads `source_path` or caches
  hashes in the registry entry — researcher evaluates; planner decides.
- How to handle SSH `StrictHostKeyChecking` and `known_hosts` for rsync subprocess
  transport — consistent with the asyncssh `known_hosts=None` precedent from Phase 3.
- Error contract when `source_path` no longer exists at sync or diff time — planner
  defines the error message in `ToolResult.error`.
- Whether `artifact_register` auto-detects `kernel_version` from the tree or
  requires caller to pass it — researcher checks if there's a standard location.

</decisions>

<canonical_refs>
## Canonical References

**Downstream agents MUST read these before planning or implementing.**

### Design Specification
- `docs/vmcluster-mcp-design.md` §2.3 — `BuildArtifact` model, `artifact_register`,
  `artifact_sync`, `artifact_install`, `artifact_list`, `artifact_diff` signatures,
  parameters, and behavioral contracts
- `docs/vmcluster-mcp-design.md` §3.2 — Directory layout: `artifact/registry.py`,
  `artifact/syncer.py`, `artifact/installer.py`
- `docs/vmcluster-mcp-design.md` §3.3 — Config defaults for `artifact_registry`
  and `artifact_store_dir`

### Data Models
- `vmcluster_mcp/models.py` — `BuildArtifact`, `SyncNodeResult`, `SyncStatus`,
  `InstallNodeResult`, `InstallStatus` already defined; `ArtifactDiff` must be added

### Configuration
- `vmcluster_mcp/config.py` — `artifact_registry` and `artifact_store_dir` already
  defined; no new config fields needed

### Existing Infrastructure to Reuse
- `vmcluster_mcp/exec/ssh.py` — asyncssh SSH client; reuse for dedup check (`test -d`)
  and `make_install`/`rpm`/`deb` install commands via SSH
- `vmcluster_mcp/server.py` — FastMCP `mcp` instance + `structured_tool_handler`
  decorator + `config` global
- `vmcluster_mcp/snapshot/manager.py` — Reference for atomic registry writes and
  per-node asyncio.gather fan-out pattern (most recent prior implementation)

### Requirements
- `.planning/REQUIREMENTS.md` — ART-01..ART-05 acceptance criteria

### Prior Phase Decisions
- `.planning/phases/04-snapshot-management/04-CONTEXT.md` — D-08 (best-effort
  per-node semantics), atomic write pattern, asyncio.gather fan-out

</canonical_refs>

<code_context>
## Existing Code Insights

### Reusable Assets
- `BuildArtifact` (models.py ln 111): Complete model already defined — use as-is
- `SyncNodeResult` / `SyncStatus` (models.py ln 125–134): Return types for `artifact_sync` — ready
- `InstallNodeResult` / `InstallStatus` (models.py ln 137–145): Return types for `artifact_install` — ready
- `exec/ssh.py`: asyncssh SSH client — reuse for node dedup check and install commands
- `config.py`: `artifact_registry` + `artifact_store_dir` paths already configured
- `structured_tool_handler` (server.py): decorator wiring ready

### Established Patterns
- **Atomic writes**: write-to-`.tmp` + `os.replace()` — snapshot registry, STATE.md
- **asyncio.gather fan-out**: per-node parallel operations — Phase 3 node_exec_all, Phase 4 snapshot ops
- **Pydantic v2 BaseModel**: all data contracts
- **`from __future__ import annotations`** in every module
- **Cluster-namespaced paths**: `{overlay_dir}/{cluster_name}/...` (artifact paths: `{dest_base}/{artifact_id}/`)

### Integration Points
- `artifact/__init__.py` is empty — implement in `artifact/registry.py`, `artifact/syncer.py`, `artifact/installer.py` per design doc §3.2
- Tools register on the `mcp` FastMCP instance from `server.py`
- `artifact_sync` and `artifact_install` depend on `artifact_register` having been called (artifact must exist in registry)
- `artifact_diff` reads both artifacts from registry and re-hashes files in their `source_path`

### Missing Model
- `ArtifactDiff` (models.py): Not yet defined — must be added before Plan 5.4 can implement `artifact_diff`

</code_context>

<specifics>
## Specific Ideas

- `modprobe_path` is the **recommended default** install mode for test clusters per the design doc.
  The design doc notes it is "trivially reverted by deleting the conf file" — this should be
  emphasized in the tool docstring for agents.
- `artifact_build` (local make/configure trigger) is explicitly deferred to v2 — do not implement
  any part of it in this phase even if ArtifactBuildResult model is already in models.py.

</specifics>

<deferred>
## Deferred Ideas

- `artifact_build` — Local build trigger (`configure` + `make`). Tracked as ART-V2-01 in REQUIREMENTS.md.
  Not part of Phase 5 scope.
- Artifact store copy/caching (copying artifact tree to `artifact_store_dir`) — design doc mentions
  `artifact_store_dir` but the v1 tools reference `source_path` directly for sync. Whether to
  copy artifacts locally before sync is an open question the researcher should evaluate — if it
  adds complexity without v1 benefit, keep `source_path` direct reference.

</deferred>

---

*Phase: 05-artifact-pipeline*
*Context gathered: 2026-03-23*
