# `atex-virt-helper` JSON protocol

## Basic idea

It's a bi-directional line-based protocol, one fully-complete JSON per one line
(so each side can `.readline()` and parse it via JSON).

Commands are given by the client via the `cmd` key, ie. `{"cmd": "ping"}`,
replies can be either without `cmd` (if they pertain to a global state)
or with `cmd` (if they are generated by the `cmd`-handling code).

Success is relayed in replies with the `success` key with a boolean value.

Free-form text in replies is often done via the `reply` key, typically for
error text when `success` is `false`.

The replies almost always contain all valid arguments that were passed in
the request, ie. if a `data` key was in the request, it is copied to the
reply as well, to help the caller associate a success/failure with what
was requested.

The server always takes JSON lines on stdin and always outputs them on
stdout, with stderr sometimes being used as an auxiliary `cmd`-specific
channel.

## Valid commands

- `ping`
  - A simple no-op to query whether the helper is listening and alive.

  ```json
  C: {"cmd": "ping"}
  S: {"cmd": "ping", "success": true, "reply": "atex-virt-helper v1 pong"}
  ```

- `setname`
  - Set a process name that others can see (when querying for `reservations`
    or otherwise in `top` or `/proc/$pid/comm`.
  - The name is limited to 15 characters (by `prctl(PR_SET_NAME)`).

  ```json
  C: {"cmd": "setname", "name": "fanthomas"}
  S: {"cmd": "setname", "success": true, "name": "fanthomas"}
  ```

- `virsh`
  - Bypass to `virsh` running on the server (so the client doesn't require it).
  - The `-c` (connect) argument is handled automatically.
  - Any outputs are always from `--quiet` mode (stripping off the extra `\n`).

  ```json
  C: {"cmd": "virsh", "args": ["list"]}
  S: {"cmd": "virsh", "args": ["list"], "success": true, "reply": " 747   vmname   running\n"}

  C: {"cmd": "virsh", "args": ["sdsd"]}
  S: {"cmd": "virsh", "args": ["list"], "success": false, "reply": "error: unknown command: 'sdsd'\n"}
  ```

- `reserve`
  - Reserve (lock) any one domain for exclusive use by this session.
  - The lock is released by `release` or session (helper) exit.
  - The locking is advisory - it doesn't prevent users messing with the domain.
    It only works when everyone respects it (uses it).
  - Optional `filter` arg makes it operate only on domain names matching
    the filter. The filter is a regex matched using `re.fullmatch()`.

  ```json
  C: {"cmd": "reserve"}
  S: {"cmd": "reserve", "success": true, "domain": "vmname1"}
  C: {"cmd": "reserve"}
  S: {"cmd": "reserve", "success": true, "domain": "vmname2"}
  C: {"cmd": "reserve"}
  S: {"cmd": "reserve", "success": false, "reply": "no domain could be reserved"}

  C: {"cmd": "reserve", "filter": "blabla.*"}
  S: {"cmd": "reserve", "success": false, "filter": "blabla.*", "reply": "no domain matches the filter"}
  ```

- `release`
  - Un-reserve (unlock) any one domain, as a flip side to `reserve`.

  ```json
  C: {"cmd": "release", "domain": "vmname2"}
  S: {"cmd": "release", "success": true, "domain": "vmname2"}
  ```

- `reservations`
  - List all domains and whether anybody has them reserved.
  - The response contains `domains` as a JSON object (dict) with keys being
    domain names and values one of:
    - `null` when not reserved,
    - a string containing process name that holds the reservation, see also `setname`,
    - special string `(reserved for us)` when our process holds the reservation,
    - other special strings inside `()`, such as when a process name is empty

  ```json
  C: {"cmd": "reservations"}
  S: {"success": true, "cmd": "reservations", "domains": {"vmname1": "(reserved for us)", "vmname3": null, "vmname2": "fanthomas"}}
  ```

- `upload`
  - Upload a single binary file under the specified name.
  - Intended for use with the `virsh` or `virt-install` commands, ie. for
    providing an install-time kickstart as `--initrd-inject ks.cfg`.
  - The file is uploaded to a temporary directory that is deleted when the
    helper session ends.
    - The helper changes CWD to the temporary directory upon startup, so that
      the file name can have a simple relative name, and other commands can
      access it via that name (no `/`).
  - The file cannot have `/` in its name.
  - The file data is transferred using the same channel as JSON control lines,
    the helper reads exactly the amount of bytes specified in `length` before
    resuming line-JSON parsing.

  ```json
  C: {"cmd": "upload", "file": "ks.cfg", "length": 4}
  C: abc\n
  S: {"cmd": "upload", "success": true, "file": "ks.cfg", "length": 4}
  ```

- `virt-install`
  - Bypass to `virt-install` running on the server.
  - The `--connect` argument is handled automatically.
  - Any `virt-install` output is routed to the stderr of the server-side helper,
    and the client must read it (to empty buffers).
  - Standard error output (stderr) of the server-side helper is used to output
    `virt-install`.
  - The `virt-install` command is run via pseudotty to allow it to do
    `virsh console` (`--autoconsole text`) on the domain under installation,
    helping provide debug output.
    - This means the helper **does not** need to have a valid TTY, a big
      improvement over running bare `virt-install`.
  - If `destroy_on_error` is passed and `true`, attempt to find a `-n` / `--name`
    passed to `virt-install` and call `virsh destroy $name` if an error happens
    while running `virt-install` (ie. ssh running the helper disconnects).
    - While `virt-install` has its own `--destroy-on-exit`, it currently (2026)
      does not work with `--transient`. This argument works around that and
      lets you use `--transient` with `"destroy_on_error": true`, effectively
      doing full cleanup on a sudden disconnect / error.

  ```json
  C: {"cmd": "virt-install", "args": ["--name", "vmname", "--memory", "4096", ...], "destroy_on_error": true}
  S (stderr): ... virt-install output ...
  S (stdout): {"cmd": "virt-install", "success": true, "args": ["--name", "vmname", "--memory", "4096", ...]}

  C: {"cmd": "virt-install", "args": ["asds"]}
  S (stderr): usage: virt-install OPTIONS
  S (stderr): virt-install: error: unrecognized arguments: asds
  S (stdout): {"success": false, "args": ["asds"], "reply": "virt-install exited with 2"}
  ```

- `vol-copy`
  - An alternative to `virsh vol-clone` that is actually fast and atomic.
  - Useful for keeping a set of volumes with pre-installed OSes (installed via
    `virt-install`) and copying them as domain-backing volumes instead of
    re-installing the domain.
  - Tries to use reflink (COW) copy on XFS/btrfs to be MUCH faster than
    `virsh vol-clone`.
  - Always copies to a temporary file, then does atomic rename, potentially
    replacing the destination.
  - If `move: true` is given, it just does the atomic replace, renaming
    `from` to `to`.
  - Can copy either one volume to another (within the same pool) or a volume
    to a domain (the helper looks up the domain's volume in the specified pool).
    - If replacing a domain's volume, the helper clears NVRAM if it exists,
      given that old EFI vars are likely not relevant to the new volume.
    - The domain **is not restarted**, the caller needs to take care of
      destroying / starting it.
  - The storage pool is refreshed after copying, incl. working around (retrying)
    a race condition in libvirt (`has asynchronous jobs running`).
  - Only `<pool type='dir'>` (directory) storage pools are supported,
    with domains that use them via `<disk type='volume'>`.
    - Domains with `<disk type='file'>` linking to the absolute file path
      are not supported (it's hard to associate the file with a storage pool).

  ```json
  C: {"cmd": "vol-copy", "pool": "default", "from": "abc", "to": "xyz"}
  S: {"cmd": "vol-copy", "success": true, "pool": "default", "from": "abc", "to": "xyz"}

  C: {"cmd": "vol-copy", "pool": "default", "from": "9.6.qcow2", "to_domain": "vmname"}
  S: {"cmd": "vol-copy", "success": true, "pool": "default", "from": "9.6.qcow2", "to_domain": "vmname"}
  ```

## Final notes

- We avoid python-libvirt and use `virsh` instead to prevent any protocol issues
  seen in the past (connection resets, event loop broken, etc.) and to avoid
  a dependency on libvirt-devel OS package when installing atex from PyPI (to compile
  python-libvirt).
  - TL;DR: `virsh` is a slow, but reliable battle-tested solution.

- We also try to offload any libvirt-related commands and tools to the server,
  allowing the client to use any language that can do JSON (even Javascript).
