# Issue #1280: brew.packages doesn't handle packages from third-party taps in an idempotent way

## Issue Summary

**Reporter:** @karlicoss
**Created:** 2025-01-26
**Status:** Open
**URL:** https://github.com/pyinfra-dev/pyinfra/issues/1280

When using `brew.packages` with a package from a third-party tap specified with its full name (e.g., `koekeishiya/formulae/skhd`), pyinfra incorrectly reports that changes are needed on subsequent runs, even though the package is already installed.

### Reproduction Steps

1. Run: `pyinfra @local brew.packages koekeishiya/formulae/skhd`
2. Run the same command again
3. pyinfra incorrectly reports a change is needed

### Expected Behavior

The second run should report "no changes" since the package is already installed.

### Actual Behavior

pyinfra reports a change is needed because it cannot match the full tap name (`koekeishiya/formulae/skhd`) with the short name (`skhd`) returned by `brew list --versions`.

---

## Root Cause Analysis

### The Problem Flow

1. **User requests package:** `brew.packages koekeishiya/formulae/skhd`

2. **`BrewPackages` fact collection** (file: `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/brew.py`, lines 65-88):
   ```python
   class BrewPackages(FactBase):
       def command(self) -> str:
           return "brew list --versions"

       def process(self, output):
           return parse_packages(BREW_REGEX, output)
   ```

3. **Output from `brew list --versions`:**
   ```
   skhd 0.3.9
   yabai 7.1.6
   ```
   Note: This returns SHORT names only, not full tap-qualified names.

4. **Package comparison** in `ensure_packages()` (file: `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/util/packaging.py`, lines 136-231):
   - The function receives `packages_to_ensure = ["koekeishiya/formulae/skhd"]`
   - It creates `PkgInfo(name="koekeishiya/formulae/skhd", ...)`
   - It looks for this in `current_packages = {"skhd": {"0.3.9"}}`
   - **Lookup fails** because `"koekeishiya/formulae/skhd" not in {"skhd": ...}`
   - Result: Package is considered "not installed" and scheduled for installation

### The Key Files Involved

| File | Role |
|------|------|
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/brew.py` | Defines `BrewPackages` fact that runs `brew list --versions` |
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/brew.py` | Defines `packages()` operation that uses the fact |
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/util/packaging.py` | Contains `ensure_packages()` that compares requested vs installed packages |
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/util/packaging.py` | Contains `parse_packages()` that parses `brew list` output |

### Why `brew list --versions` Is Insufficient

The `brew list --versions` command outputs packages by their short name regardless of which tap they came from:

```bash
$ brew list --versions | grep skhd
skhd 0.3.9
```

There is `brew list --full-name` but it is mutually exclusive with `--versions`.

---

## Proposed Fix

### Option A: Use `brew info --json --installed` (Recommended)

This is the most robust solution as suggested by the issue reporter.

#### Changes Required

**1. Modify `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/brew.py`:**

```python
import json

class BrewPackages(FactBase):
    """
    Returns a dict of installed brew packages with both full and short names:

    .. code:: python

        {
            "package_name": ["version"],
            "tap/repo/package_name": ["version"],  # Also includes full name for tap packages
        }
    """

    @override
    def command(self) -> str:
        return "brew info --json=v2 --installed"

    @override
    def requires_command(self) -> str:
        return "brew"

    default = dict

    @override
    def process(self, output):
        packages: dict[str, set[str]] = {}

        try:
            # Join all output lines and parse as JSON
            data = json.loads("\n".join(output))
        except json.JSONDecodeError:
            return packages

        for formula in data.get("formulae", []):
            name = formula.get("name", "")
            full_name = formula.get("full_name", name)

            # Get installed versions
            versions = set()
            for installed in formula.get("installed", []):
                version = installed.get("version", "")
                if version:
                    versions.add(version)

            if name and versions:
                # Add by short name
                packages[name] = versions
                # Also add by full name if different (for tap packages)
                if full_name != name:
                    packages[full_name] = versions

        return packages
```

**2. Update the `BrewCasks` class similarly:**

```python
class BrewCasks(FactBase):
    """
    Returns a dict of installed brew casks with both full and short names.
    """

    @override
    def command(self) -> str:
        return "brew info --json=v2 --installed"

    @override
    def requires_command(self) -> str:
        return "brew"

    default = dict

    @override
    def process(self, output):
        packages: dict[str, set[str]] = {}

        try:
            data = json.loads("\n".join(output))
        except json.JSONDecodeError:
            return packages

        for cask in data.get("casks", []):
            token = cask.get("token", "")
            full_token = cask.get("full_token", token)
            version = cask.get("installed", "")

            if token and version:
                packages[token] = {version}
                if full_token != token:
                    packages[full_token] = {version}

        return packages
```

**3. Update test file `/Users/nick/Dev/Webroot/pyinfra/src/tests/facts/brew.BrewPackages/packages.json`:**

The test would need to be updated with JSON input/output format.

### Option B: Normalize Package Names in `ensure_packages()`

A simpler but less complete fix would be to extract the short name from full tap-qualified names when looking up packages.

**Modify `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/util/packaging.py`:**

Add a helper function and modify `_has_package`:

```python
def _normalize_brew_package_name(package: str) -> list[str]:
    """
    Return possible names for a brew package.
    For 'koekeishiya/formulae/skhd' returns ['koekeishiya/formulae/skhd', 'skhd']
    For 'git' returns ['git']
    """
    names = [package]
    if "/" in package:
        # Extract short name (last component after last /)
        short_name = package.rsplit("/", 1)[-1]
        names.append(short_name)
    return names
```

Then use this in brew.packages operation to check both forms.

### Option C: Document the Workaround

At minimum, document that users should:
1. First call `brew.tap("koekeishiya/formulae")`
2. Then call `brew.packages("skhd")` (short name)

This ensures idempotency but requires users to know the workaround.

---

## Recommendation

**Option A is recommended** because:

1. It uses structured JSON output which is more reliable than regex parsing
2. It handles both full and short names seamlessly
3. It provides version information alongside full name in a single command
4. It's future-proof against output format changes
5. It aligns with brew's recommended approach for scripting

### Potential Concerns

1. **Performance:** `brew info --json --installed` may be slightly slower than `brew list --versions`, but the difference should be minimal.

2. **Backward Compatibility:** The change maintains backward compatibility since:
   - Short names will still work (indexed by both short and full name)
   - Existing code using short names won't break

3. **brew Version Compatibility:** Need to verify `--json=v2` is available in older brew versions. May need fallback logic similar to what exists for cask CLI changes.

---

## Related Files to Modify

| File | Changes Needed |
|------|----------------|
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/brew.py` | Update `BrewPackages` and `BrewCasks` classes to use JSON output |
| `/Users/nick/Dev/Webroot/pyinfra/src/tests/facts/brew.BrewPackages/packages.json` | Update test with new JSON format |
| `/Users/nick/Dev/Webroot/pyinfra/src/tests/facts/brew.BrewCasks/casks.json` | Update test with new JSON format |
| `/Users/nick/Dev/Webroot/pyinfra/src/tests/operations/brew.packages/*.json` | May need updates to include full tap names in test facts |

---

## Testing Plan

1. **Unit Tests:**
   - Update fact tests with JSON input/output
   - Add test case with third-party tap package (full name)
   - Verify both short and full names work for lookup

2. **Integration Tests:**
   - Test `brew.packages("koekeishiya/formulae/skhd")` runs idempotently
   - Test `brew.packages("git")` still works (short name from default tap)
   - Test `brew.packages("python@3.13")` still works (versioned packages)

3. **Edge Cases:**
   - Empty installed packages list
   - Malformed JSON output (graceful fallback)
   - Packages with `@version` suffix from third-party taps
