# GitHub Issue #1019: files.line grep inserts wrong regex

## Issue Summary

**Reported by:** JustScreaMy (October 4, 2023)
**Status:** Open (bug label)
**Assignee:** Fizzadar

### Problem Description

When using `files.line` to replace a single line, the operation produces an incorrect grep regex pattern, causing the line to never be matched and therefore automatically inserted at the end of the file instead of replacing the existing line.

### Reproduction Code

```python
from pyinfra.operations import files

files.line(
    name="Disable password login",
    path="/etc/ssh/sshd_config",
    line="PasswordAuthentication .+",
    replace="PasswordAuthentication no"
)
```

### Expected Behavior

The code should match a line containing `PasswordAuthentication yes` and replace it with `PasswordAuthentication no`.

### Actual Behavior (at time of report)

The grep command generated was:
```bash
sh -c 'grep -e '\"'\"'^.*.+PasswordAuthentication.+.*$'\"'\"' /etc/ssh/sshd_config
```

The problem was that the original regex `PasswordAuthentication .+` was being transformed into `^.*.+PasswordAuthentication.+.*$` which places the `.+` from the original pattern **before** the literal text instead of after it.

---

## Root Cause Analysis

### Investigation Findings

Based on the comments on the issue and analysis of the current codebase:

1. **Issue Comment from "bad" (June 19, 2025):** States that the `.+` insertion issue appears to be fixed on 3.x HEAD, with the example grep now showing:
   ```
   sh -c 'grep -e '\"'\"'^.*10.0.2.2[        ]\\{1,\\}host.*$'\"'\"' /etc/hosts [...]
   ```

2. **Current Code Path:**

   The `files.line` operation (in `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/files.py`, line 278-475) calls `adjust_regex()` (line 372) to transform the user-provided line pattern.

   The `adjust_regex()` function (in `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/util/files.py`, lines 200-216):
   ```python
   def adjust_regex(line: str, escape_regex_characters: bool) -> str:
       """
       Ensure the regex starts with '^' and ends with '$' and escape regex characters if requested
       """
       match_line = line

       if escape_regex_characters:
           match_line = re.sub(r"([\.\\\+\*\?\[\^\]\$\(\)\{\}\-])", r"\\\1", match_line)

       # Ensure we're matching a whole line, note: match may be a partial line so we
       # put any matches on either side.
       if not match_line.startswith("^"):
           match_line = "^.*{0}".format(match_line)
       if not match_line.endswith("$"):
           match_line = "{0}.*$".format(match_line)

       return match_line
   ```

3. **Historical Bug Context:**

   Based on the original report's grep output `^.*.+PasswordAuthentication.+.*$`, it appears the old code was incorrectly inserting `^.*.+` at the beginning regardless of the pattern contents. This has been fixed in the current implementation which simply prepends `^.*` (without `.+`).

### Current Behavior Analysis

For the input `line="PasswordAuthentication .+"`:
- Current code produces: `^.*PasswordAuthentication .+.*$`
- This regex will match any line containing "PasswordAuthentication " followed by one or more characters

The current implementation **appears correct** for the reported issue.

### Remaining Concerns

As noted in the second comment by "bad", there are still potential optimization issues:

1. **Unnecessary `^.*` prefix:** Adding `^.*` serves no purpose in grep matching and causes excessive backtracking, slowing down pattern matching on large files.

2. **Unnecessary `.*$` suffix:** This is also redundant since grep already matches partial lines by default.

These are performance concerns rather than correctness issues.

---

## Current Status Assessment

Based on the investigation, **the original bug appears to have been fixed** in the current 3.x codebase. The `adjust_regex()` function now correctly prepends `^.*` (without the erroneous `.+`) and appends `.*$`.

However, there are two perspectives on whether the issue should be closed:

### Option 1: Close as Fixed
The original reported behavior (`.+` being incorrectly inserted) is no longer present.

### Option 2: Address Performance Concerns
While functionally correct, the regex transformation adds unnecessary overhead:
- `^.*` causes backtracking in grep
- `.*$` is redundant

---

## Proposed Fix (Performance Optimization)

If the decision is to address the performance concerns raised in the comments, the `adjust_regex()` function could be simplified:

### Current Code (`/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/util/files.py`):
```python
def adjust_regex(line: str, escape_regex_characters: bool) -> str:
    """
    Ensure the regex starts with '^' and ends with '$' and escape regex characters if requested
    """
    match_line = line

    if escape_regex_characters:
        match_line = re.sub(r"([\.\\\+\*\?\[\^\]\$\(\)\{\}\-])", r"\\\1", match_line)

    # Ensure we're matching a whole line, note: match may be a partial line so we
    # put any matches on either side.
    if not match_line.startswith("^"):
        match_line = "^.*{0}".format(match_line)
    if not match_line.endswith("$"):
        match_line = "{0}.*$".format(match_line)

    return match_line
```

### Proposed Optimized Code:
```python
def adjust_regex(line: str, escape_regex_characters: bool) -> str:
    """
    Ensure the regex matches a whole line and escape regex characters if requested.

    The pattern is wrapped to ensure it matches complete lines, not partial matches
    within a line.
    """
    match_line = line

    if escape_regex_characters:
        match_line = re.sub(r"([\.\\\+\*\?\[\^\]\$\(\)\{\}\-])", r"\\\1", match_line)

    # Ensure we're matching a whole line by anchoring at start and end.
    # Note: We only add anchors, not wildcards, since grep already matches
    # partial lines and adding ^.* causes unnecessary backtracking.
    if not match_line.startswith("^"):
        match_line = "^{0}".format(match_line)
    if not match_line.endswith("$"):
        match_line = "{0}$".format(match_line)

    return match_line
```

**IMPORTANT:** This change would alter the matching behavior:
- Current: `PasswordAuthentication .+` becomes `^.*PasswordAuthentication .+.*$` (matches line containing the pattern anywhere)
- Proposed: `PasswordAuthentication .+` becomes `^PasswordAuthentication .+$` (matches line starting with the pattern)

This is a **breaking change** and may not be desirable. The docstring for `files.line` states:
> "Unless line matches a line (starts with ^, ends $), pyinfra will wrap it such that it does, like: `^.*LINE.*$`. This means we don't swap parts of lines out."

So the current behavior is **intentional** to support matching partial lines. The performance cost is a documented trade-off for flexibility.

---

## Related Files

1. **Operation definition:** `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/files.py` (lines 278-475)
2. **Regex adjustment utility:** `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/operations/util/files.py` (lines 200-216)
3. **FindInFile fact (grep execution):** `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/facts/files.py` (lines 423-456)

---

## Recommendations

1. **Confirm the original bug is fixed:** The core issue (`.+` being inserted incorrectly) appears resolved.

2. **Decision needed on performance optimization:**
   - If keeping current behavior (partial line matching with `^.*...*$`), document that this is intentional
   - If changing to strict anchoring (`^...$`), this is a breaking change requiring a major version bump or deprecation period

3. **Consider closing the issue** with a note that:
   - The original regex insertion bug is fixed
   - The `^.*` and `.*$` wrapping is intentional per the documented behavior
   - Users who want exact line matching should use explicit `^` and `$` anchors in their patterns

4. **Update documentation** to clarify the performance implications of the default partial-line matching behavior and recommend explicit anchors for performance-critical use cases.
