# GitHub Issue #1026: operations.apt.packages shows "SUCCESS/Changed" for packages that are already installed in some cases

**Issue URL:** https://github.com/pyinfra-dev/pyinfra/issues/1026
**Status:** Open
**Reporter:** @stone-w4tch3r
**Labels:** bug
**Created:** November 6, 2023

## Issue Summary

When using `apt.packages` with package names that are "aliases" (i.e., virtual packages or packages that resolve to different actual package names), pyinfra incorrectly reports "SUCCESS/Changed" on every run, even when the package is already installed.

### Reproduction Steps

1. Run Ubuntu/Debian VM
2. Run pyinfra with:
```python
from pyinfra.operations import apt

apt.update(_sudo=True, cache_time=86400)
apt.packages(_sudo=True, packages=['fortune'])
```
3. Run again - pyinfra reports "Changed: 1" even though the package was already installed

### The Problem

APT has a concept of package aliases/virtual packages. When you request `fortune`, APT resolves it to `fortune-mod`:

```
sudo apt install fortune -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'fortune-mod' instead of 'fortune'  # <-- THIS
fortune-mod is already the newest version (1:1.99.1-7build1).
```

pyinfra's `DebPackages` fact uses `dpkg -l` to list installed packages, which only shows the real package name (`fortune-mod`), not the alias (`fortune`). When the operation checks if `fortune` is installed, it doesn't find it because only `fortune-mod` is in the installed packages list.

## Root Cause Analysis

### Current Code Flow

1. **`apt.packages` operation** (`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/apt.py`, lines 398-496):
   - Calls `ensure_packages()` with the list of packages and current installed packages from `DebPackages` fact

2. **`DebPackages` fact** (`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/deb.py`, lines 30-58):
   - Uses `dpkg -l` command to get installed packages
   - Returns dict of `{package_name: [versions]}`
   - Only knows about **actual** package names, not aliases

3. **`ensure_packages` utility** (`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/util/packaging.py`, lines 136-231):
   - Takes an `expand_package_fact` parameter that can resolve package names to their actual counterparts
   - Uses `_has_package()` function to check if a package is installed
   - **APT operations do NOT use `expand_package_fact`** - this is the root cause

### How Pacman Solved This

The pacman package manager has the same problem with groups and virtual packages. pyinfra solves it with:

1. **`PacmanUnpackGroup` fact** (`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/pacman.py`, lines 14-39):
   ```python
   class PacmanUnpackGroup(FactBase):
       def command(self, package):
           return 'pacman -S --print-format "%n" {0} || true'.format(shlex.quote(package))
   ```
   This resolves package aliases/groups to their actual package names.

2. **pacman.packages operation** (`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/pacman.py`, lines 74-82):
   ```python
   yield from ensure_packages(
       host,
       packages,
       host.get_fact(PacmanPackages),
       present,
       install_command="pacman --noconfirm -S",
       uninstall_command="pacman --noconfirm -R",
       expand_package_fact=lambda package: host.get_fact(PacmanUnpackGroup, package=package),
   )
   ```

### The Missing Piece for APT

The `apt.packages` operation does NOT pass an `expand_package_fact` parameter:

```python
# Current code in apt.py (line 486-496)
yield from ensure_packages(
    host,
    packages,
    host.get_fact(DebPackages),
    present,
    install_command=noninteractive_apt(install_command, force=force),
    uninstall_command=noninteractive_apt(uninstall_command, force=force),
    upgrade_command=noninteractive_apt(upgrade_command, force=force),
    version_join="=",
    latest=latest,
)
```

## Proposed Fix

### 1. Create a new `DebResolvePackage` fact

Add to `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/deb.py`:

```python
class DebResolvePackage(FactBase):
    """
    Resolves a package name (which may be an alias/virtual package) to the actual
    package name(s) that would be installed by apt.

    Returns a list of package names, or the original package if resolution fails.
    """

    @override
    def requires_command(self, package) -> str:
        return "apt-cache"

    default = list

    @override
    def command(self, package):
        # apt-cache show will show the resolved package info
        # Use apt-get --simulate install to see what would actually be installed
        # The -qq flag minimizes output, and we parse the "Inst" lines
        return (
            "apt-get -qq --simulate install {0} 2>/dev/null | "
            "grep -oP '^Inst \\K[^ ]+' || echo {0}"
        ).format(shlex.quote(package))

    @override
    def process(self, output):
        # Return list of package names that would be installed
        # Filter out empty lines
        packages = [line.strip() for line in output if line.strip()]
        return packages if packages else []
```

**Alternative approach using `apt-cache`:**

```python
class DebResolvePackage(FactBase):
    """
    Resolves a package name (which may be an alias/virtual package) to the actual
    package name(s) that would be installed by apt.
    """

    @override
    def requires_command(self, package) -> str:
        return "apt-cache"

    default = list

    @override
    def command(self, package):
        # apt-cache policy shows the resolved package name
        # For virtual packages, it shows what provides them
        return "apt-cache policy {0} 2>/dev/null | head -1 | cut -d: -f1".format(
            shlex.quote(package)
        )

    @override
    def process(self, output):
        result = []
        for line in output:
            line = line.strip()
            if line:
                result.append(line)
        return result
```

**Recommended approach using `apt-cache showpkg`** (as suggested in GitHub comment):

```python
class DebResolvePackage(FactBase):
    """
    Resolves a package name (which may be an alias/virtual package) to the actual
    package name(s) that would be installed by apt.

    For virtual packages like 'fortune', this returns the real package name(s)
    that provide the virtual package (e.g., 'fortune-mod').
    """

    @override
    def requires_command(self, package) -> str:
        return "apt-cache"

    default = list

    @override
    def command(self, package):
        # apt-cache showpkg shows reverse provides which tells us what packages
        # provide a virtual package. We check if it's a real package first,
        # and if not, look for providers.
        return (
            "if apt-cache show {0} >/dev/null 2>&1; then "
            "echo {0}; "
            "else "
            "apt-cache showpkg {0} 2>/dev/null | "
            "awk '/^Reverse Provides:/{{flag=1; next}} flag && NF{{print $1}}'; "
            "fi"
        ).format(shlex.quote(package))

    @override
    def process(self, output):
        packages = [line.strip() for line in output if line.strip()]
        return packages if packages else []
```

### 2. Update apt.packages to use the new fact

Modify `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/apt.py`:

```python
# Add import at top
from pyinfra.facts.deb import DebPackage, DebPackages, DebResolvePackage

# Update ensure_packages call (around line 486)
yield from ensure_packages(
    host,
    packages,
    host.get_fact(DebPackages),
    present,
    install_command=noninteractive_apt(install_command, force=force),
    uninstall_command=noninteractive_apt(uninstall_command, force=force),
    upgrade_command=noninteractive_apt(upgrade_command, force=force),
    version_join="=",
    latest=latest,
    expand_package_fact=lambda package: host.get_fact(DebResolvePackage, package=package),
)
```

### 3. Add tests

Create test cases in `/Users/nick/Dev/Webroot/pyinfra/src/tests/`:

```python
# Test for the new fact
def test_deb_resolve_package_virtual():
    """Test that virtual packages are resolved to their providers."""
    # Test with a known virtual package like 'fortune' -> 'fortune-mod'
    pass

def test_deb_resolve_package_real():
    """Test that real packages resolve to themselves."""
    pass

# Test for the operation
def test_apt_packages_virtual_package_already_installed():
    """Test that apt.packages doesn't report changes for already-installed virtual packages."""
    pass
```

## Implementation Notes

### Choice of apt command

Several options were discussed in the GitHub comments:

1. **`apt-cache showpkg`** - Shows package information including "Reverse Provides" which lists what packages provide a virtual package
2. **`apt-file`** - Can find which package provides a file (overkill for this use case)
3. **`apt-get --simulate install`** - Shows exactly what would be installed

The recommended approach is `apt-cache showpkg` because:
- It's available on all Debian/Ubuntu systems with apt
- It directly shows the "Reverse Provides" relationship
- It doesn't require simulating an install (faster)

### Edge Cases to Consider

1. **Multiple providers**: A virtual package might have multiple providers. The current implementation would need to check if ANY of them is installed.
2. **Version-specific aliases**: Some package aliases might be version-specific.
3. **Architecture-specific packages**: Packages like `libc6:amd64` need proper handling.
4. **Transitional packages**: Packages that exist but recommend/depend on their successor.

### Performance Considerations

Using `expand_package_fact` will result in additional fact-gathering for each package. This is the same approach used by pacman and is acceptable. The facts are memoized, so repeated calls for the same package don't result in extra commands.

## Related Files

| File Path | Purpose |
|-----------|---------|
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/apt.py` | APT operations including `apt.packages` |
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/deb.py` | DEB package facts - needs new `DebResolvePackage` fact |
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/util/packaging.py` | `ensure_packages` utility that supports `expand_package_fact` |
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/pacman.py` | Reference implementation using `expand_package_fact` |
| `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/pacman.py` | Reference `PacmanUnpackGroup` fact |

## Conclusion

The issue occurs because pyinfra's APT package management doesn't resolve package aliases/virtual packages to their actual names before checking if they're installed. The fix requires:

1. Creating a new `DebResolvePackage` fact that uses `apt-cache showpkg` to resolve package names
2. Updating `apt.packages` to pass the new fact as `expand_package_fact` to `ensure_packages`
3. Adding appropriate tests

This mirrors the existing solution for pacman packages and leverages the already-implemented `expand_package_fact` parameter in `ensure_packages()`.
