# GitHub Issue #615: Exception raised trying to collect facts for file

## Issue Summary

**Issue URL:** https://github.com/pyinfra-dev/pyinfra/issues/615
**Status:** Open (state marked as open in GitHub)
**Type:** Bug
**Reporter:** @blarghmatey
**First Reported:** June 28, 2021

### Description

When using `files.file` or `files.directory` operations, an exception is raised while collecting the `File` fact. The error occurs when a `Path` object (from Python's `pathlib` module) is passed as a path argument instead of a string.

The traceback shows:
```
TypeError: expected string or bytes-like object
```

This occurs at `shlex.quote()` which expects a string but receives a `Path` object.

### Root Cause Analysis

After investigating the codebase, the root cause has been identified in multiple locations:

#### Primary Issue: `File.command()` in `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/files.py`

```python
# Line 229-234
@override
def command(self, path):
    if path.startswith("~/"):  # <-- PROBLEM: Path objects don't have startswith for strings
        # Do not quote leading tilde to ensure that it gets properly expanded by the shell
        path = f"~/{shlex.quote(path[2:])}"
    else:
        path = QuoteString(path)
```

When a `pathlib.Path` object is passed:
1. `path.startswith("~/")` fails because `Path.startswith()` requires a `PurePath` object, not a string
2. This results in a `TypeError`

#### Secondary Issue: `HashFileFactBase.command()` and `HashFileFactBase.process()`

```python
# Line 382-395
@override
def command(self, path):
    self.path = path  # <-- Stores the Path object as-is
    return make_formatted_string_command(self._raw_cmd, QuoteString(path))

@override
def process(self, output) -> Optional[str]:
    output = output[0]
    escaped_path = re.escape(self.path)  # <-- re.escape converts to string, but regex matching may fail
    for regex in self._regexes:
        matches = re.match(regex % escaped_path, output)
        if matches:
            return matches.group(1)
    return None
```

While `re.escape()` will convert the `Path` object to a string, the behavior is inconsistent. The path should be converted to a string early to ensure consistent handling.

#### Flow of the Problem

1. User passes a `Path` object to an operation (e.g., `files.file(path=Path("/some/path"))`)
2. Operation calls `host.get_fact(File, path=path)`
3. Fact's `command()` method is called with the `Path` object
4. `path.startswith("~/")` is called, which fails for `Path` objects with string arguments

### Historical Context from Issue Comments

1. Initial fix in v1.4.7 (commit `b91281e9`) tried to handle non-string `StringCommand` bits
2. That fix was incomplete - it addressed the symptom in `StringCommand._get_all_bits()` but not the root cause
3. A second fix (commit `48d6ff7e`) attempted to handle `Path` objects more broadly
4. The reporter (@blarghmatey) noted that the fix wasn't complete because the type conversion was happening "too far down in the stack"
5. The recommended approach is to convert `Path` objects to strings "at the perimeter of the system" rather than doing "whack-a-mole" fixes

### Current State

The `StringCommand` class in `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/api/command.py` does handle non-string types:

```python
# Line 113-114
if not isinstance(bit, str):
    bit = "{0}".format(bit)
```

However, this happens too late in the processing. The issue occurs earlier when fact `command()` methods are called with `Path` objects and they perform string operations (like `startswith`) on them.

## Proposed Fix

The fix should be applied at the fact level, converting `Path` objects to strings at the entry point of each fact's `command()` method. This follows the recommended approach of handling type conversion "at the perimeter."

### Option 1: Fix Individual Fact Classes (Minimal Change)

**File:** `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/files.py`

```python
# Add import at the top
from os import fspath

# Modify File.command() - around line 229
@override
def command(self, path):
    path = fspath(path)  # Convert Path objects to strings
    if path.startswith("~/"):
        # Do not quote leading tilde to ensure that it gets properly expanded by the shell
        path = f"~/{shlex.quote(path[2:])}"
    else:
        path = QuoteString(path)
    # ... rest of method

# Modify HashFileFactBase.command() - around line 383
@override
def command(self, path):
    path = fspath(path)  # Convert Path objects to strings
    self.path = path
    return make_formatted_string_command(self._raw_cmd, QuoteString(path))
```

The `os.fspath()` function:
- Returns the string representation of a path-like object
- Passes through strings unchanged
- Raises `TypeError` for non-path-like, non-string objects

### Option 2: Central Fix in `_handle_fact_kwargs()` (Comprehensive)

**File:** `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/api/facts.py`

Add a helper function to convert Path-like objects to strings:

```python
from os import fspath
from pathlib import PurePath

def _normalize_fact_kwargs(fact_kwargs):
    """Convert Path-like objects to strings in fact kwargs."""
    normalized = {}
    for key, value in fact_kwargs.items():
        if isinstance(value, PurePath):
            normalized[key] = fspath(value)
        else:
            normalized[key] = value
    return normalized

# Then in _handle_fact_kwargs(), before returning:
def _handle_fact_kwargs(state: "State", host: "Host", cls, args, kwargs):
    # ... existing code ...

    if args or fact_kwargs:
        # Merges args & kwargs into a single kwargs dictionary
        fact_kwargs = getcallargs(cls().command, *args, **fact_kwargs)

    # Normalize Path objects to strings
    fact_kwargs = _normalize_fact_kwargs(fact_kwargs)

    return fact_kwargs, global_kwargs
```

### Option 3: Decorator Approach (Most Comprehensive)

Create a decorator that automatically converts `Path` arguments to strings for any fact `command()` method:

**File:** `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/api/facts.py`

```python
from functools import wraps
from os import fspath
from pathlib import PurePath

def normalize_path_args(func):
    """Decorator to convert Path-like arguments to strings."""
    @wraps(func)
    def wrapper(*args, **kwargs):
        args = tuple(fspath(a) if isinstance(a, PurePath) else a for a in args)
        kwargs = {k: fspath(v) if isinstance(v, PurePath) else v for k, v in kwargs.items()}
        return func(*args, **kwargs)
    return wrapper
```

Then apply to facts that accept path arguments.

## Recommended Approach

**Option 2** is recommended because:
1. It's a single-location fix that handles all facts
2. It doesn't require modifying individual fact classes
3. It's consistent with the reporter's suggestion of converting at the "perimeter"
4. It's backward compatible - strings pass through unchanged

## Files That Need Modification

1. **Primary fix location:**
   - `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/api/facts.py` - Add Path normalization in `_handle_fact_kwargs()`

2. **Alternative/additional fix locations (if using Option 1):**
   - `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/files.py` - Modify `File.command()` and `HashFileFactBase.command()`

3. **Testing:**
   - `/Users/nick/Dev/Webroot/pyinfra/src/tests/test_facts.py` - Add test cases with `Path` objects
   - Create test file at `/Users/nick/Dev/Webroot/pyinfra/src/tests/facts/files.File/path_object.json` with a `Path` object argument

## Related Files

- `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/api/command.py` - `StringCommand` already handles non-string types in `_get_all_bits()`
- `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/files.py` - File operations that call facts
- `/Users/nick/Dev/Webroot/pyinfra/src/tests/test_api/test_api_command.py` - Has test for `Path` with `StringCommand`

## Testing Considerations

1. Test that `Path` objects work with:
   - `File` fact
   - `Directory` fact
   - `Link` fact
   - `Sha256File` and other hash facts
   - `FindInFile` fact
   - `Block` fact
   - `FileContents` fact

2. Test edge cases:
   - `Path("~/.bashrc")` (home directory expansion)
   - `Path("/path/with spaces/file.txt")`
   - `PurePosixPath` objects
   - Mixed usage (some `str`, some `Path`)

## Notes

- The issue was originally reported in v1.4.5/v1.4.6 (2021)
- Multiple fix attempts were made but incomplete
- The codebase has evolved since then (now on v3.x branch)
- The `StringCommand` class already handles `Path` objects, but the issue occurs before that code is reached
