# Issue #1389: Positional arguments in any operations result in `Too few arguments` mypy errors

## Issue Summary

**GitHub Issue**: https://github.com/pyinfra-dev/pyinfra/issues/1389
**Reporter**: @karlicoss
**Status**: Open
**Created**: 2025-07-03

### Problem Description

When using positional arguments with pyinfra operations, mypy reports type errors:

```python
from pyinfra.operations import server
server.shell('echo')
```

Results in:
```
test.py:2: error: Too few arguments for "__call__" of "PyinfraOperation"  [call-arg]
test.py:2: error: Argument 1 to "__call__" of "PyinfraOperation" has incompatible type "str"; expected "bool"  [arg-type]
```

### Workaround

Using explicit keyword arguments works correctly:
```python
from pyinfra.operations import server
server.shell(commands='echo')  # This passes mypy
```

---

## Root Cause Analysis

### The Core Problem: Python Typing Limitations with ParamSpec

The issue stems from a fundamental limitation in Python's type system when trying to combine:
1. **ParamSpec** for preserving the original operation's signature
2. **Additional keyword arguments** (global arguments like `_sudo`, `_sudo_user`, etc.)

### How pyinfra Operations Work

1. Operations are defined as regular Python functions:
   ```python
   @operation(is_idempotent=False)
   def shell(commands: str | list[str]):
       ...
   ```

2. The `@operation` decorator wraps the function and returns a `PyinfraOperation[P]` type that:
   - Accepts the original operation's parameters (`P.args`, `P.kwargs`)
   - Adds "global arguments" (`_sudo`, `_sudo_user`, `name`, `_parallel`, etc.)

### The PyinfraOperation Protocol

Located at `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/api/arguments_typed.py`:

```python
class PyinfraOperation(Generic[P], Protocol):
    _inner: Callable[P, Generator]

    def __call__(
        self,
        #
        # op args - needs to be first (PR #1105 fix)
        #
        *args: P.args,
        #
        # ConnectorArguments
        #
        _sudo: bool = False,
        _sudo_user: Optional[str] = None,
        # ... many more global arguments ...
        #
        # op kwargs
        #
        **kwargs: P.kwargs,
    ) -> "OperationMeta": ...
```

### Why the Error Occurs

The problem is **Python's typing system limitation with ParamSpec and Concatenate**:

1. **PEP 612** (ParamSpec) defines `P.args` and `P.kwargs` for capturing function signatures
2. **PEP 612 explicitly states** that you can only prepend **positional-only** arguments before `P.args` using `Concatenate`
3. There is **no way** in the current Python type system to:
   - Add keyword arguments **between** the original positional args and kwargs
   - Have positional arguments from `P.args` work correctly when other keyword-only parameters are defined

From PEP 612 specification (https://peps.python.org/pep-0612/#concatenating-keyword-parameters):
> "Only positional argument can be prepended... keyword argument cannot be prepended"

### The Technical Details

When mypy sees:
```python
server.shell('echo')
```

It interprets `'echo'` as being passed to the first positional parameter in `PyinfraOperation.__call__`, which is `*args: P.args`. However, because there are keyword-only parameters defined after `*args`, mypy gets confused about how to match positional arguments from the ParamSpec.

The PR #1105 (merged 2024-05-25) moved `*args: P.args` to be first in the parameter list, which partially fixed the issue for keyword-only usage but did not solve the fundamental problem of positional arguments.

### Why Keyword Arguments Work

When using keyword arguments:
```python
server.shell(commands='echo')
```

mypy correctly matches `commands` to `P.kwargs`, and the global arguments (all with defaults) don't interfere.

---

## Related History

### PR #1105 (Merged)
- **Title**: "PyinfraOperation: Put *args first so that mypy correctly handles inner operation args"
- **Change**: Moved `*args: P.args` to be first in the `__call__` signature
- **Reason**: When `*args` was after the keyword parameters, mypy was assigning positional args to `_sudo`, `_sudo_user`, etc.

### PR #1082 (Merged)
- **Title**: "V3: type hints for operations"
- Added type hints to all operations, establishing the current typing infrastructure

---

## Proposed Fixes

### Option 1: Accept as Documented Limitation (Current Approach)

**Recommendation from maintainers**: Use keyword arguments for all operation calls.

**Pros**:
- No code changes required
- More explicit and readable code
- Aligns with pyinfra documentation and examples

**Cons**:
- Users may be surprised by mypy errors
- Existing code using positional arguments will fail type checking

**Implementation**:
- Update documentation to clearly state this limitation
- Add a note to the README or typing documentation

### Option 2: Add Type Ignore Comment Guidance

Document how users can suppress these specific errors:

```python
from pyinfra.operations import server
server.shell('echo')  # type: ignore[call-arg]
```

### Option 3: Explore Alternative Typing Approaches (Complex)

**Note**: These are theoretical and may not be fully feasible.

#### 3a. Use Overloads for Common Cases

Create explicit overloads for the most common operations:

```python
from typing import overload

class PyinfraOperation(Generic[P], Protocol):
    @overload
    def __call__(self, __arg1: P.args[0], /, **kwargs: P.kwargs) -> "OperationMeta": ...
    @overload
    def __call__(self, *args: P.args, **kwargs: P.kwargs) -> "OperationMeta": ...
    # ... more overloads
```

**Problems**:
- ParamSpec doesn't support indexing (`P.args[0]`)
- Would require separate overloads for each arity, which is impractical

#### 3b. Per-Operation Type Stubs

Generate `.pyi` stub files for each operation with explicit signatures:

```python
# server.pyi
def shell(
    commands: str | list[str],
    *,
    _sudo: bool = ...,
    _sudo_user: Optional[str] = ...,
    # ... etc
) -> OperationMeta: ...
```

**Pros**: Full type safety for both positional and keyword args
**Cons**:
- Significant maintenance burden
- Must keep stubs in sync with actual code
- Need to generate stubs for ~100+ operations

#### 3c. Wait for Python Typing Improvements

Future Python versions may support:
- `Concatenate` with keyword arguments (not currently possible)
- Better ParamSpec features for this use case

**Related PEPs to watch**:
- PEP 612 extensions
- Any future PEP addressing keyword argument prepending

### Option 4: Runtime Enforcement (Non-Typing Solution)

Add a runtime check that warns or errors when positional arguments are used:

```python
def _wrap_operation(func: Callable[P, Generator], _set_in_op: bool = True) -> PyinfraOperation[P]:
    @wraps(func)
    def decorated_func(*args: P.args, **kwargs: P.kwargs) -> OperationMeta:
        if args:
            import warnings
            warnings.warn(
                f"Positional arguments to {func.__name__} may cause mypy errors. "
                "Consider using keyword arguments instead.",
                UserWarning,
                stacklevel=2
            )
        # ... rest of implementation
```

**Pros**: Educates users at runtime
**Cons**: Doesn't fix the mypy errors, just explains them

---

## Affected Files

1. **Primary**: `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/api/arguments_typed.py`
   - Contains `PyinfraOperation` Protocol definition
   - Lines 29-79

2. **Secondary**: `/Users/nick/Dev/Webroot/pyinfra/src/src/pyinfra/api/operation.py`
   - Contains `operation` decorator and `_wrap_operation` function
   - Lines 240-381

3. **Test Files**:
   - `/Users/nick/Dev/Webroot/pyinfra/src/tests/typing/valid/operation.py` - Only uses keyword args
   - `/Users/nick/Dev/Webroot/pyinfra/src/tests/typing/invalid/operation_kwarg.py`

---

## Recommendations

### Immediate Actions

1. **Document the limitation** clearly in:
   - README/docs under "Type Checking" section
   - Add inline comment in `arguments_typed.py` explaining why this limitation exists

2. **Update valid typing tests** to explicitly test that keyword arguments work

3. **Consider adding** an invalid typing test case for positional arguments (to document the expected behavior)

### Long-Term Considerations

1. Monitor Python typing PEPs for improvements that might allow a proper fix
2. Consider per-operation stubs if type safety becomes a higher priority
3. Evaluate if making operations keyword-only (at runtime) is acceptable for v4

---

## Conclusion

This is a **fundamental limitation of Python's type system**, specifically PEP 612's ParamSpec design. The pyinfra codebase is already using the best available approach. The recommended solution is to:

1. Accept the limitation
2. Document it clearly
3. Encourage users to use keyword arguments (which is already the convention in pyinfra examples)

The maintainer @Fizzadar has confirmed this understanding in the issue comments, stating: "IMO the correct 'fix' here is just to use keyword arguments for all operation calls, which the docs and examples do (or should). Personally I think it makes for more explicit code which is a good thing."
