# Issue #807: Hard to remove file-or-link

## Issue Summary

**Reporter:** drewp
**Created:** 2022-05-07
**Status:** Open
**Labels:** bug
**Related Issue:** #791 (files.link did not remove old link pointing to "different" target)

### Problem Description

Users face difficulty when they need to remove a filesystem path that could be either a file or a symlink, then create a new file in its place. The core problem is that pyinfra's type-checking operations (`files.file`, `files.link`) raise errors when the path exists but is of a different type than expected.

### Example from Issue

```python
# User wants to ensure /etc/resolv.conf is a templated file (not a symlink)
files.file(path='/etc/resolv.conf', present=False)      # Line 1
files.link(path='/etc/resolv.conf', present=False)      # Line 2
files.template(src='templates/resolv.conf.j2', dest='/etc/resolv.conf', ns='10.2.0.1')  # Line 3
```

**Results:**
- Running 1,2,3: `pyinfra error: /etc/resolv.conf exists and is not a file`
- Running 2,3 only: `pyinfra error: /etc/resolv.conf exists and is not a link`
- Running 3 only: Works but inconsistently - some hosts retain symlinks, others have files

**Expected behavior:** `files.template` (Line 3 alone) should replace symlinks with the templated file.

---

## Root Cause Analysis

### 1. Type-Checking in `files.file` and `files.link`

Both operations use fact checks that return `False` when the path exists but is of the wrong type:

**File Fact (`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/files.py`):**
```python
class File(FactBase[Union[FileDict, Literal[False], None]]):
    """
    If the path does not exist: returns ``None``
    If the path exists but is not a file: returns ``False``
    """
    type = "file"

    def process(self, output):
        # ...
        if path_type != self.type:
            return False  # Path exists but is wrong type
```

**Link Fact inherits from File:**
```python
class Link(File):
    type = "link"
```

### 2. Error Handling in Operations (`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/files.py`)

When `info is False` (path exists but wrong type), both `files.file` and `files.link` call `_raise_or_remove_invalid_path`:

**In `files.file` (lines 1561-1569):**
```python
info = host.get_fact(File, path=path)

if info is False:  # not a file
    yield from _raise_or_remove_invalid_path(
        "file",
        path,
        force,
        force_backup,
        force_backup_dir,
    )
    info = None
```

**In `files.link` (lines 1453-1463):**
```python
info = host.get_fact(Link, path=path)

if info is False:  # not a link
    yield from _raise_or_remove_invalid_path(
        "link",
        path,
        force,
        force_backup,
        force_backup_dir,
    )
    info = None
```

### 3. The `_raise_or_remove_invalid_path` Function (lines 1387-1398)

```python
def _raise_or_remove_invalid_path(fs_type, path, force, force_backup, force_backup_dir):
    if force:
        if force_backup:
            backup_path = "{0}.{1}".format(path, get_timestamp())
            if force_backup_dir:
                backup_path = os.path.basename(backup_path)
                backup_path = "{0}/{1}".format(force_backup_dir, backup_path)
            yield StringCommand("mv", QuoteString(path), QuoteString(backup_path))
        else:
            yield StringCommand("rm", "-rf", QuoteString(path))
    else:
        raise OperationError("{0} exists and is not a {1}".format(path, fs_type))
```

### 4. Key Issue: `present=False` with Wrong Type

The critical problem is in the deletion logic. When `present=False` and the path is the wrong type:

**In `files.file` (lines 1571-1576):**
```python
if not present:
    if info:
        yield StringCommand("rm", "-f", QuoteString(path))
    else:
        host.noop("file {0} does not exist")
    return
```

**In `files.link` (lines 1471-1476):**
```python
if not present:
    if info:
        yield remove_cmd
    else:
        host.noop("link {link} does not exist")
    return
```

**The Problem:** When a symlink exists and you call `files.file(path=X, present=False)`:
1. `info = host.get_fact(File, path=X)` returns `False` (exists but not a file)
2. `_raise_or_remove_invalid_path` is called
3. Since `force=False` by default, an error is raised
4. The code never reaches the `if not present:` block

This means you cannot use `files.file(present=False)` to remove a symlink, or `files.link(present=False)` to remove a regular file.

### 5. The `files.template` Issue

`files.template` delegates to `files.put`, which checks the File fact:

```python
remote_file = host.get_fact(File, path=dest)
```

If `remote_file is False` (path exists but is not a file, e.g., it's a symlink), the code falls through to the upload logic without checking or removing the symlink. The upload succeeds but the symlink remains, and the file is written to wherever the symlink points.

---

## Proposed Fix

### Option A: Make `present=False` Ignore Type Mismatch (Recommended)

The most intuitive fix is to allow `present=False` to delete any existing path regardless of its type. Users calling `files.file(path=X, present=False)` want to ensure X doesn't exist - they don't care if it's a file, link, or directory.

**Proposed change to `files.file`:**

```python
@operation()
def file(
    path: str,
    present=True,
    # ... other args
):
    path = _validate_path(path)
    mode = ensure_mode_int(mode)

    # When present=False, we just want to remove whatever exists
    if not present:
        # Check if anything exists at this path (file, link, or directory)
        file_info = host.get_fact(File, path=path)
        link_info = host.get_fact(Link, path=path)

        if file_info or link_info:
            # Remove it regardless of type
            yield StringCommand("rm", "-f", QuoteString(path))
        else:
            host.noop("file {0} does not exist".format(path))
        return

    # Rest of existing logic for present=True...
    info = host.get_fact(File, path=path)
    if info is False:
        yield from _raise_or_remove_invalid_path(...)
    # ...
```

**Similarly for `files.link`:**

```python
@operation()
def link(
    path: str,
    target: str | None = None,
    present=True,
    # ... other args
):
    path = _validate_path(path)

    if present and not target:
        raise OperationError("If present is True target must be provided")

    # When present=False, remove whatever exists
    if not present:
        file_info = host.get_fact(File, path=path)
        link_info = host.get_fact(Link, path=path)

        if file_info or link_info:
            yield StringCommand("rm", "-f", QuoteString(path))
        else:
            host.noop("link {0} does not exist".format(path))
        return

    # Rest of existing logic for present=True...
    info = host.get_fact(Link, path=path)
    if info is False:
        yield from _raise_or_remove_invalid_path(...)
    # ...
```

### Option B: Add `force` Support for `present=False`

Allow `force=True` to work with `present=False` to remove any type:

```python
# User code would become:
files.file(path='/etc/resolv.conf', present=False, force=True, force_backup=False)
files.template(src='templates/resolv.conf.j2', dest='/etc/resolv.conf', ns='10.2.0.1')
```

This requires moving the type check after the `present=False` handling.

### Option C: Fix `files.template`/`files.put` to Handle Symlinks

Modify `files.put` (which `files.template` uses) to detect and optionally remove symlinks:

```python
remote_file = host.get_fact(File, path=dest)

# If path is a symlink, decide what to do
if remote_file is False:
    link_info = host.get_fact(Link, path=dest)
    if link_info:
        # Remove the symlink before uploading
        yield StringCommand("rm", "-f", QuoteString(dest))
        remote_file = None  # Now treat as non-existent
```

---

## Recommended Implementation

I recommend **Option A** as the primary fix, combined with **Option C** for completeness.

### Rationale:

1. **Option A** follows the principle of least surprise - when a user says "I don't want this to exist", we should remove it regardless of its type
2. **Option C** makes `files.template` and `files.put` more robust by handling symlinks gracefully
3. The combination allows both explicit removal and implicit replacement to work correctly

### Files to Modify:

1. `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/files.py`:
   - Modify `file()` function to handle `present=False` before type checking
   - Modify `link()` function to handle `present=False` before type checking
   - Optionally modify `put()` to handle symlinks at destination

2. `/Users/nick/Dev/Webroot/pyinfra/src/tests/operations/files.file/` - Add test cases:
   - `delete_symlink.json` - Test removing a symlink with `files.file(present=False)`

3. `/Users/nick/Dev/Webroot/pyinfra/src/tests/operations/files.link/` - Add test cases:
   - `delete_file.json` - Test removing a file with `files.link(present=False)`

### Proposed Test Case: `files.file/delete_symlink.json`

```json
{
    "args": ["testpath"],
    "kwargs": {
        "present": false
    },
    "facts": {
        "files.File": {
            "path=testpath": false
        },
        "files.Link": {
            "path=testpath": {
                "user": "root",
                "group": "root",
                "mode": 777,
                "link_target": "/some/target"
            }
        }
    },
    "commands": [
        "rm -f testpath"
    ]
}
```

---

## Workarounds (Current)

Until the fix is implemented, users can use:

### 1. Use `force=True` with `present=True` (not ideal semantically)

```python
# This works but creates the file instead of just removing
files.file(path='/etc/resolv.conf', force=True, force_backup=False)
```

### 2. Use `server.shell` with POSIX-compatible syntax

```python
# Use test -L for POSIX shell compatibility (not [[ ]])
server.shell(commands=["test -L /etc/resolv.conf && rm /etc/resolv.conf || true"])
files.template(src='templates/resolv.conf.j2', dest='/etc/resolv.conf')
```

### 3. Check and conditionally remove

```python
from pyinfra import host
from pyinfra.facts.files import Link

if host.get_fact(Link, path='/etc/resolv.conf'):
    files.link(path='/etc/resolv.conf', present=False, force=True, force_backup=False)

files.template(src='templates/resolv.conf.j2', dest='/etc/resolv.conf')
```

---

## Summary

The issue stems from type-checking logic that raises an error before considering the `present=False` intent. The fix should allow deletion operations to succeed regardless of the existing path's type, making the operations more intuitive and consistent with user expectations.
