# PET Macro Registry — Design Proposal

> **Status: Deferred.** This feature is fully designed but not implemented.
> It will be reconsidered when the user base justifies the development and
> maintenance cost. This document is intended for LLMs resuming the work and
> for developers who stumble upon it and want to contribute or comment.

---

## Background

PET ships with a built-in macro library installed into `.pet/` by `pet init`.
As users write their own macros, there is a natural desire to share them across
projects and with other users. This document describes a macro registry and
install system that would make that possible.

The design deliberately avoids a centralised package index (like PyPI or npm).
Instead it builds on two ideas:

- **`~/.pet/repo/`** — a user-local macro cache that the user populates however
  they like, bypassing all authentication and network constraints.
- **`.pet-req`** — a project-level manifest (TOML, committed to version control)
  that records what is installed, where it came from, and where it lives in the
  cache.

---

## Goals

- Install macros from GitHub, GitLab, any HTTPS URL, or the local cache
- Work offline once a macro has been fetched once
- Support multiple versions of the same macro across different projects without
  a formal version manager
- Keep the project self-contained and reproducible: `pet init && pet install`
  restores everything from scratch
- Stay simple: no dependency resolution, no lock files, no registry server

## Non-Goals (v1)

- Centralised macro index or discovery
- Automatic version management
- Authentication against private remote repositories (handled out-of-band)
- `--as` flag for renaming macros at install time
- Per-macro version pinning without manual cache management

---

## Directory Structure

### User-global cache: `~/.pet/`

`~/.pet/` is reserved for PET user-level state. The macro cache lives under
`~/.pet/repo/`. Other subdirectories may be added in future versions.

```
~/.pet/
    repo/                   # macro cache — the only location pet install reads
        wordcount.py
        nlp/
            tokenize.py
            stem.py
        nlp-v1/             # manually managed version copy
            tokenize.py
            stem.py
        acme-corp/          # placed here manually (private / auth-required source)
            invoice.py
    config                  # reserved for future global settings
```

`~/.pet/repo/` holds exactly one copy of each macro path. There is no built-in
version management — if the user needs multiple versions, they manage them as
separate subdirectories (see [Version Management](#version-management) below).

### Project-local: `.pet/` and `.pet-req`

```
<project>/
    .pet/                   # generated — not committed to version control
        chapter.py          # standard library (from pet init)
        wordcount.py        # symlink → ~/.pet/repo/wordcount.py
        nlp/                # symlink → ~/.pet/repo/nlp-v1/
        acme-corp/          # copy (when symlinks not supported)
        .hash               # hash of standard library files
    .pet-req                # committed to version control
```

---

## `.pet-req` Manifest

`.pet-req` is a TOML file in the project root, alongside `.pet/`. It is
committed to version control. It is created empty by `pet init` and updated
automatically by `pet install`.

### Schema

```toml
# .pet-req

[macros.<name>]
source = "<source-uri>"      # where the macro came from (informational + used by --update)
path   = "<cache-path>"      # path relative to ~/.pet/repo/ — the actual install source
sha    = "<hex>"             # content hash; written for remote sources; optional for local
```

### Field semantics

**`source`** — the original URI passed to `pet install`. Used by
`pet install --update` to re-fetch. Never used for the actual file lookup —
that is always done through `path`. The user can change `path` freely without
touching `source`.

**`path`** — path relative to `~/.pet/repo/` that is linked or copied into
`.pet/`. This is the field the user edits for manual version management.
Can point to a file (`wordcount.py`) or a directory (`nlp-v1/`).

**`sha`** — SHA-256 (or MD5, consistent with existing `.hash`) of the content
at `~/.pet/repo/<path>` at the time of install. Written automatically for
remote sources. Optional for `local:` sources (the user manages the cache).
On `pet install` (no-arg), a mismatch between the recorded `sha` and the
current content of `~/.pet/repo/<path>` produces a **warning**, not an error —
the install proceeds. This handles the case where the user has intentionally
updated the cache.

### Example

```toml
# .pet-req

[macros.wordcount]
source = "local:wordcount.py"
path   = "wordcount.py"

[macros.nlp]
source = "github:verhasp/pet-macros/nlp/@v1.2.0"
path   = "nlp-v1"
sha    = "a3f8c2..."

[macros.myformat]
source = "https://example.com/macros/myformat.py"
path   = "myformat.py"
sha    = "9d1e47..."
```

---

## Source URI Syntax

```
github:<user>/<repo>/<path>[@<ref>]
gitlab:<user>/<repo>/<path>[@<ref>]
https://<url>
local:<path-relative-to-repo>
```

- `<ref>` is a branch, tag, or commit SHA. Omitting it fetches the default branch.
- `<path>` within a GitHub/GitLab URI may point to a file or a directory.
- `local:` sources are already in `~/.pet/repo/`; no network access occurs.
- `https://` fetches a single file only (no directory listing).

### Resolving GitHub/GitLab to a download URL

For a file:
```
github:verhasp/pet-macros/nlp/tokenize.py@v1.2.0
→ https://raw.githubusercontent.com/verhasp/pet-macros/v1.2.0/nlp/tokenize.py
```

For a directory, use the GitHub/GitLab archive API or the contents API to
enumerate and fetch all `.py` files recursively.

---

## Command Reference

### `pet install <source>`

Fetch (if remote), cache in `~/.pet/repo/`, link or copy into `.pet/`, update
`.pet-req`.

```
pet install github:verhasp/pet-macros/nlp/@v1.2.0
pet install https://example.com/macros/myformat.py
pet install local:wordcount.py
pet install local:acme-corp/
```

Options:

| Flag | Effect |
|------|--------|
| `-c`, `--copy` | Copy instead of symlink, even if symlinks are supported |
| `--force` | Overwrite existing `.pet/<name>` and `~/.pet/repo/<path>` |

**Behaviour:**

1. Parse the source URI.
2. For `github:`, `gitlab:`, `https://`: fetch to `~/.pet/repo/<derived-path>`.
   Create `~/.pet/repo/` if it does not exist.
   For `local:`: verify `~/.pet/repo/<path>` exists; error if not.
3. Compute content hash; record as `sha` for remote sources.
4. Symlink `~/.pet/<name>` → `~/.pet/repo/<path>`, or copy if `--copy` or if
   the OS does not support symlinks.
5. Write/update the `[macros.<name>]` entry in `.pet-req`.

The `<name>` used in `.pet/` and in `.pet-req` is derived from the final
path component of `<path>` (filename without extension for files; directory
name for directories).

### `pet install` (no arguments)

Read `.pet-req` and reinstall every entry. This is the standard command after
cloning a project.

```
pet install
```

**Behaviour:**

1. Error if `.pet-req` does not exist.
2. For each `[macros.*]` entry:
   a. Verify `~/.pet/repo/<path>` exists. Error with clear message if not
      (the user must populate the cache manually for `local:` sources, or
      re-run `pet install <source>` to re-fetch remote ones).
   b. If `sha` is recorded, compute current hash of `~/.pet/repo/<path>`.
      Warn if mismatched; continue.
   c. Symlink or copy into `.pet/<name>`.
3. Works fully offline once all paths are in `~/.pet/repo/`.

### `pet install --update [<name>]`

Re-fetch remote sources and update the cache. Does not affect `local:` entries
(the user updates the cache manually).

```
pet install --update          # update all remote entries in .pet-req
pet install --update nlp      # update only the 'nlp' entry
```

**Behaviour:**

1. Read `.pet-req`.
2. For each targeted entry whose `source` is not `local:`:
   a. Re-fetch to `~/.pet/repo/<path>` (overwrite).
   b. Recompute and update `sha` in `.pet-req`.
   c. Re-link or re-copy into `.pet/<name>`.

### `pet init` changes

`pet init` gains two new behaviours:

1. **Create `.pet-req`** if it does not exist (empty file with a comment header).
2. **Run `pet install`** automatically if `.pet-req` exists and has one or more
   `[macros.*]` entries.

This keeps `pet init` as the single onboarding command.

---

## Version Management

`~/.pet/repo/` has no built-in version management. The user manages versions
manually by using different directory names in the cache.

**Scenario:** two projects use `nlp` at different versions.

```bash
# Fetch v1 — lands in ~/.pet/repo/nlp
pet install github:verhasp/pet-macros/nlp/@v1.2.0
mv ~/.pet/repo/nlp ~/.pet/repo/nlp-v1

# Fetch v2 — lands in ~/.pet/repo/nlp
pet install github:verhasp/pet-macros/nlp/@v2.0.0
mv ~/.pet/repo/nlp ~/.pet/repo/nlp-v2
```

In project A, edit `.pet-req`:
```toml
[macros.nlp]
source = "github:verhasp/pet-macros/nlp/@v1.2.0"
path   = "nlp-v1"
sha    = "..."
```

In project B, edit `.pet-req`:
```toml
[macros.nlp]
source = "github:verhasp/pet-macros/nlp/@v2.0.0"
path   = "nlp-v2"
sha    = "..."
```

`pet install` in each project links `.pet/nlp` to the correct version.
The `source` field remains accurate for documentation and future `--update` use.

---

## Symlink vs Copy

Symlinks are preferred because:

- Changes to `~/.pet/repo/` are immediately reflected in all projects that
  link to that path.
- `pet install --update` only needs to update the cache; the link stays valid.

Copy is used when:

- `--copy` / `-c` is passed explicitly.
- The OS does not support symlinks (detected at runtime; silent fallback).

The `sha` field in `.pet-req` serves as a consistency check regardless of
whether a symlink or copy was used.

---

## Content Hashing

Consistent with the existing `.hash` mechanism in `pet init`:

- **Single file:** MD5 of the file content.
- **Directory:** MD5 of the sorted concatenation of `<relative-path><content>`
  for all `.py` files, excluding `__pycache__` and `__init__.py`.

This reuses `_pet_hash()` from `cli.py`, extended to accept a file path as well
as a directory.

---

## Data Flow Summary

```
pet install github:user/repo/nlp/@v1.2.0
    │
    ├─ 1. fetch via HTTPS → ~/.pet/repo/nlp/
    ├─ 2. compute sha
    ├─ 3. write .pet-req  [macros.nlp]
    │                     source = "github:user/repo/nlp/@v1.2.0"
    │                     path   = "nlp"
    │                     sha    = "<computed>"
    └─ 4. symlink .pet/nlp → ~/.pet/repo/nlp

pet install   (no args — e.g. after git clone)
    │
    └─ for each [macros.*] in .pet-req:
         check sha (warn on mismatch)
         symlink/copy ~/.pet/repo/<path> → .pet/<name>
         works offline ✓
```

---

## Implementation Sketch

The implementation fits cleanly into the existing codebase:

| Component | Location | Notes |
|-----------|----------|-------|
| `cmd_install(args)` | `cli.py` | New function; analogous to `cmd_init` |
| `_pet_req_path()` | `cli.py` | Returns `Path(".pet-req")` |
| `_read_pet_req()` | `cli.py` | Returns parsed TOML dict |
| `_write_pet_req()` | `cli.py` | Writes updated TOML dict |
| `_fetch_remote(uri)` | `cli.py` or new `fetch.py` | Handles `github:`, `gitlab:`, `https://` |
| `_link_or_copy(src, dst, copy)` | `cli.py` | Symlink with copy fallback |
| `_pet_hash(path)` | `cli.py` | Extend existing function to accept files |
| `pet install` subcommand | `cli.py` `main()` | Add argparse subcommand |
| `pet init` changes | `cli.py` `cmd_init()` | Create `.pet-req`; call `cmd_install` |

The only new external dependency is an HTTP client for remote fetches.
`urllib.request` (stdlib) suffices for simple HTTPS downloads. The GitHub
contents API requires JSON parsing (`json` stdlib). No third-party dependencies
are strictly needed for v1.

---

## Open Questions for Future Versions

- **`--as <name>`** — install a macro under a different name in `.pet/`
- **Per-macro copy/link preference** recorded in `.pet-req`
- **`pet install --list`** — show installed macros and their sources
- **`pet uninstall <name>`** — remove from `.pet/` and `.pet-req`
- **Curated index** — a community-maintained list of known macro packages
- **Integrity algorithm** — upgrade from MD5 to SHA-256 for security-conscious users
- **GitLab and other forges** — the GitHub raw URL pattern may differ; needs
  per-forge resolution logic
