# Issue #1061: files.sync does not remove existing symlinks

## Issue Summary

**GitHub Issue:** https://github.com/pyinfra-dev/pyinfra/issues/1061
**Reporter:** @Snektron
**Status:** Open (bug, operations labels)
**Created:** 2024-01-22

### Description
When using `files.sync` with `delete=True` on a remote directory that contains symlinks not present in the local source directory, those symlinks are NOT deleted. This breaks the expected behavior of syncing a directory where all files not in the source should be removed.

### Reproduction Example
```python
files.sync(
    name="Update MOTD",
    src="files/update-motd.d/",
    dest="/etc/update-motd.d",
    delete=True,
)
```

On Ubuntu 22.04, the `/etc/update-motd.d` directory contains symlinks like `50-landscape-sysinfo`. After sync, the directory contains:
- `something` (the file from local source - correct)
- `50-landscape-sysinfo` (symlink that should have been deleted - bug)

### Maintainer Response
@Fizzadar confirmed that the sync operation should handle symlinks, ideally behaving like rsync's archive mode.

---

## Root Cause Analysis

### The Problem Location
**File:** `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/files.py`
**Function:** `sync()` (lines 552-696)

### How Deletion Currently Works

The sync operation's deletion logic (lines 686-695) works as follows:

```python
# Delete any extra files
if delete:
    remote_filenames = set(host.get_fact(FindFiles, path=dest) or [])
    wanted_filenames = set([remote_filename for _, remote_filename in put_files])
    files_to_delete = remote_filenames - wanted_filenames
    for filename in files_to_delete:
        # Should we exclude this file?
        if exclude and any(fnmatch(filename, match) for match in exclude):
            continue

        yield from file._inner(path=filename, present=False)
```

### The Root Cause

**The `FindFiles` fact only finds regular files, NOT symlinks.**

Looking at the fact definitions in `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/files.py`:

```python
class FindFiles(FindFilesBase):
    """
    Returns a list of files from a start path, recursively using ``find``.
    """
    type_flag = "f"  # <-- Only finds regular files (-type f)


class FindLinks(FindFilesBase):
    """
    Returns a list of links from a start path, recursively using ``find``.
    """
    type_flag = "l"  # <-- Finds symlinks (-type l)
```

The `find` command generated uses `-type f` which explicitly excludes symlinks (type `l`).

### Secondary Issue: `file._inner()` Cannot Delete Symlinks

Even if symlinks were found, the current deletion approach calls `file._inner(path=filename, present=False)`.

Looking at the `file` operation (lines 1508-1608), when called with `present=False`:
1. It gets the `File` fact for the path
2. The `File` fact returns `False` for symlinks (since symlinks are not files)
3. The `file` operation then raises an error (via `_raise_or_remove_invalid_path`) because it found something that's "not a file"

The file operation is designed for regular files, not symlinks. To delete symlinks, you would need to use `link._inner(path=filename, present=False)`.

---

## Proposed Fix

### Solution Overview

1. Also find symlinks in the remote directory using `FindLinks`
2. Delete symlinks using `link._inner()` instead of `file._inner()`

### Code Changes

**File:** `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/files.py`

#### Change 1: Import FindLinks

At line 52, add `FindLinks` to the imports:

```python
from pyinfra.facts.files import (
    MARKER_BEGIN_DEFAULT,
    MARKER_DEFAULT,
    MARKER_END_DEFAULT,
    Block,
    Directory,
    File,
    FileContents,
    FindFiles,
    FindInFile,
    FindLinks,  # ADD THIS
    Flags,
    Link,
    Md5File,
    Sha1File,
    Sha256File,
    Sha384File,
)
```

#### Change 2: Update the sync deletion logic

Replace lines 685-695:

**Before:**
```python
    # Delete any extra files
    if delete:
        remote_filenames = set(host.get_fact(FindFiles, path=dest) or [])
        wanted_filenames = set([remote_filename for _, remote_filename in put_files])
        files_to_delete = remote_filenames - wanted_filenames
        for filename in files_to_delete:
            # Should we exclude this file?
            if exclude and any(fnmatch(filename, match) for match in exclude):
                continue

            yield from file._inner(path=filename, present=False)
```

**After:**
```python
    # Delete any extra files and symlinks
    if delete:
        remote_filenames = set(host.get_fact(FindFiles, path=dest) or [])
        remote_links = set(host.get_fact(FindLinks, path=dest) or [])
        wanted_filenames = set([remote_filename for _, remote_filename in put_files])

        # Delete extra regular files
        files_to_delete = remote_filenames - wanted_filenames
        for filename in files_to_delete:
            # Should we exclude this file?
            if exclude and any(fnmatch(filename, match) for match in exclude):
                continue

            yield from file._inner(path=filename, present=False)

        # Delete extra symlinks
        links_to_delete = remote_links - wanted_filenames
        for linkname in links_to_delete:
            # Should we exclude this link?
            if exclude and any(fnmatch(linkname, match) for match in exclude):
                continue

            yield from link._inner(path=linkname, present=False)
```

---

## Additional Considerations

### 1. Syncing Symlinks FROM Local Source

The current implementation also does not sync symlinks that exist in the local source directory. If the local `src` directory contains symlinks, they are not uploaded/recreated on the remote. This is a separate but related issue.

To fully support rsync-like behavior (as @Fizzadar mentioned), the sync operation would need to:
1. Detect symlinks in the local source using `os.walk` or similar
2. Create corresponding symlinks on the remote using `link._inner()`

This would require more extensive changes to the file iteration logic in lines 620-648.

### 2. Documentation Update

The docstring for `sync()` at line 566 states:
> Note that delete will remove extra files on the remote side, but not extra directories.

This should be updated to mention symlinks once the fix is applied:
> Note that delete will remove extra files and symlinks on the remote side, but not extra directories.

---

## Files to Modify

1. **`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/files.py`**
   - Add `FindLinks` import
   - Update `sync()` deletion logic

2. **`/Users/nick/Dev/Webroot/pyinfra/src/tests/operations/files.sync/`** (new test file needed)
   - Add test case `sync_delete_symlinks.json` to verify symlink deletion works

---

## Test Plan

### New Test Case: `sync_delete_symlinks.json`

```json
{
    "require_platform": ["Darwin", "Linux"],
    "args": ["/somedir/", "/home/somedir"],
    "kwargs": {
        "delete": true
    },
    "local_files": {
        "files": {},
        "dirs": {
            "somedir": {
                "files": {
                    "somefile.txt": null
                },
                "dirs": {}
            }
        }
    },
    "facts": {
        "files.File": {
            "path=/home/somedir/somefile.txt": null
        },
        "files.Directory": {
            "path=/home/somedir/somefile.txt": null,
            "path=/home/somedir": null
        },
        "files.FindFiles": {
            "args=None, fname=None, iname=None, max_size=None, maxdepth=None, min_size=None, path=/home/somedir, quote_path=True, regex=None, size=None": []
        },
        "files.FindLinks": {
            "args=None, fname=None, iname=None, max_size=None, maxdepth=None, min_size=None, path=/home/somedir, quote_path=True, regex=None, size=None": [
                "/home/somedir/some-symlink"
            ]
        },
        "files.Link": {
            "path=/home/somedir": false,
            "path=/home/somedir/some-symlink": {
                "user": "root",
                "group": "root",
                "mode": 777,
                "link_target": "/some/target"
            }
        }
    },
    "commands": [
        "mkdir -p /home/somedir",
        "chmod 755 /home/somedir",
        ["upload", "/somedir/somefile.txt", "/home/somedir/somefile.txt"],
        "chmod 644 /home/somedir/somefile.txt",
        "rm -f /home/somedir/some-symlink"
    ]
}
```

---

## Summary

The bug is caused by the sync operation only using `FindFiles` (which finds regular files with `-type f`) to determine what to delete. Symlinks are excluded from this search because they have type `l`, not `f`.

The fix requires:
1. Also querying `FindLinks` to find symlinks in the remote directory
2. Deleting symlinks using `link._inner()` instead of `file._inner()`
