# Phase 5: Artifact Pipeline — Research

**Date:** 2026-03-23
**Phase:** 05-artifact-pipeline
**Discovery Level:** 1 (Quick Verification — all libraries known, confirming patterns)

## Research Questions

### 1. Content-Addressed Tree Hashing (D-03)

**Question:** How to deterministically hash a directory tree for `artifact_id`?

**Answer:** Python's `hashlib.sha256` with deterministic file ordering. The algorithm from CONTEXT.md D-03 is sound:

```python
import hashlib, os

def hash_tree(root: str) -> str:
    file_hashes = []
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = sorted(d for d in dirnames if d not in ("__pycache__", ".git"))
        for fname in sorted(filenames):
            fpath = os.path.join(dirpath, fname)
            h = hashlib.sha256()
            with open(fpath, "rb") as f:
                for chunk in iter(lambda: f.read(8192), b""):
                    h.update(chunk)
            rel = os.path.relpath(fpath, root)
            file_hashes.append(f"{rel}:{h.hexdigest()}")
    combined = hashlib.sha256("\n".join(file_hashes).encode())
    return combined.hexdigest()
```

Key details:
- `os.walk` with `sorted()` on both dirnames and filenames for determinism
- Exclude `__pycache__`, `.git` directories (per D-03)
- 8KB read chunks for memory efficiency with large files
- Relative paths prevent host-path dependence
- Include relative path in per-file hash string so file renames produce different artifact_ids

### 2. Async Subprocess rsync (D-04, D-05)

**Question:** How to invoke rsync as an async subprocess with `asyncio.create_subprocess_exec`?

**Answer:** Standard Python `asyncio.subprocess`:

```python
proc = await asyncio.create_subprocess_exec(
    "rsync", "-avz", "--delete",
    "-e", f"ssh -i {ssh_key_path} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null",
    f"{local_tree}/",
    f"{ssh_user}@{node_ip}:{dest_base}/{artifact_id}/",
    stdout=asyncio.subprocess.PIPE,
    stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await proc.communicate()
```

Rsync flags:
- `-avz`: archive mode, verbose, compression — standard for build tree sync
- `--delete`: remove files on remote that don't exist locally (complete tree replacement)
- `-e "ssh -i ... -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"`: matches Phase 3 asyncssh `known_hosts=None` policy
- Trailing `/` on source path is critical — rsync copies contents, not the directory itself

`asyncio.gather` wraps all per-node rsync coroutines for concurrent execution. No `return_exceptions=True` needed if we handle per-node errors within the coroutine.

### 3. modprobe.d Configuration (D-10)

**Question:** What is the correct format for `/etc/modprobe.d/*.conf` to use a custom module search path?

**Answer:** The `search` directive in `depmod.d` (not `modprobe.d`) tells `depmod` about additional module directories. For the `modprobe_path` install mode:

1. Write `/etc/depmod.d/vmcluster-test.conf`:
   ```
   search updates vmcluster
   ```
2. Create a symlink or copy `.ko` files into `/lib/modules/$(uname -r)/vmcluster/{artifact_id}/`
3. Run `depmod -a`

**Alternative (simpler for test clusters):** Use `insmod` with absolute paths, or set `MODPROBE_OPTIONS` — but `depmod.d` is the standard mechanism.

**Recommended approach for v1:** Write a shell script via SSH that:
1. Creates `/lib/modules/{kernel_version}/extra/vmcluster/` 
2. Symlinks or copies `.ko` files from `{dest_base}/{artifact_id}/` into it
3. Runs `depmod -a {kernel_version}`
4. Verifies with `modprobe --dry-run {module_name}`

This is cleaner than writing modprobe.d conf files and is what `make modules_install` effectively does.

### 4. Remote Directory Existence Check (D-06)

**Question:** How to check if an artifact is already synced to a node?

**Answer:** Use the Phase 3 `SSHConnectionManager.execute()` to run `test -d {dest_base}/{artifact_id}`:

```python
result = await ssh_manager.execute(ip, port, user, key_path, f"test -d {dest_path}", timeout_s=5)
if result.exit_code == 0:
    # Already synced
```

No new infrastructure needed — reuse the existing SSH client.

### 5. kernel_version Auto-Detection

**Question:** Should `artifact_register` auto-detect `kernel_version` from the tree?

**Answer:** Check for `include/generated/utsrelease.h` or `include/config/kernel.release` in the source tree. If found, parse the version string. If not found, require the caller to pass `kernel_version`. This is safe because:
- Kernel build trees always contain `include/generated/utsrelease.h` after build
- External module trees may not — caller must specify

### 6. Module and Binary Detection

**Question:** How to scan a build tree for `.ko` files and userspace binaries?

**Answer:**
- **Modules:** `pathlib.Path(root).rglob("*.ko")` — returns relative paths
- **Userspace binaries:** Walk tree, check `os.access(path, os.X_OK)` for files that are not `.ko`, `.py`, `.sh`, or other script types. Alternatively, look for known binary patterns or just use a simple heuristic: ELF magic bytes (`\x7fELF`).

For v1, the simplest approach:
- `.ko` files: `rglob("*.ko")`
- Userspace bins: `rglob("*.bin")` plus any ELF files in `bin/` or `sbin/` subdirectories

### 7. Source Path Missing at Sync/Diff Time

**Question:** What error contract when `source_path` no longer exists?

**Answer:** Return `ToolResult(success=False, error=ErrorDetail(code="SOURCE_NOT_FOUND", message="Artifact source path '{source_path}' no longer exists on disk", recoverable=False))`. This is a fatal error — the agent must re-register the artifact from a valid path.

### 8. Package File Location for rpm/deb Install (D-12)

**Question:** How to find `.rpm` or `.deb` files in a synced tree?

**Answer:** Glob within the synced directory:
- RPM: `find {dest_path} -name '*.rpm' -type f`
- DEB: `find {dest_path} -name '*.deb' -type f`

Install all found packages: `rpm -ivh {files}` or `dpkg -i {files}`.

## Standard Stack

| Component | Choice | Reason |
|-----------|--------|--------|
| Hashing | `hashlib.sha256` (stdlib) | No external deps needed |
| File transfer | `rsync` subprocess | Per D-04, direct subprocess call |
| Async subprocess | `asyncio.create_subprocess_exec` | Per D-05, async subprocess |
| SSH dedup check | `SSHConnectionManager` (Phase 3) | Per D-06, reuse existing infra |
| Registry storage | JSON file + atomic write | Per D-01, D-02 |
| Parallel fan-out | `asyncio.gather` | Per D-13, consistent with prior phases |

## Architecture Patterns

- **Registry pattern:** Single JSON file, dict keyed by `artifact_id`, atomic writes via tmp+rename
- **Subprocess pattern:** `asyncio.create_subprocess_exec` for rsync, `asyncio.gather` for fan-out
- **SSH reuse:** Phase 3 `SSHConnectionManager` for dedup checks and install commands
- **Error pattern:** Best-effort per-node with full reporting (consistent with Phase 4 D-08)

## Don't Hand-Roll

- SHA-256 hashing: Use `hashlib` directly, don't implement custom hashing
- rsync: Use the system binary, don't implement file transfer in Python
- SSH: Reuse Phase 3 `SSHConnectionManager`, don't create a new SSH client

## Common Pitfalls

1. **Non-deterministic hash:** File ordering in `os.walk` varies by filesystem — always sort
2. **rsync trailing slash:** `source/` vs `source` — trailing slash copies contents, not directory
3. **Stale source_path:** Build tree may be deleted after registration — check existence at sync/diff time
4. **depmod requires kernel version:** Must pass `-a {kernel_version}` explicitly on remote
5. **Permission errors:** `.ko` install may require root — tool already runs as root via SSH

---
*Research completed: 2026-03-23*
