# Fix Pre-commit Test Failures - Test Fixture Setup

**Status: ✅ COMPLETED**

All 34 failing tests have been fixed. The full `make pre-commit` suite now passes with 427 tests passing.

## Problem Summary

34 tests failing in `make pre-commit` due to test fixture setup issues. All static checks (format, lint, typecheck) passed.

**Root Cause**: Tests create `mock_http_request = MagicMock()` without properly setting up `app.state.request_tracker`. When metrics are enabled (globally set in conftest), `RequestOrchestrator._initialize_metrics()` calls `get_request_tracker(http_request)` which fails because MagicMock's `app.state.request_tracker` is another MagicMock (not a real RequestTracker).

### Two Failure Patterns

1. **TypeError: app.state.request_tracker is not a RequestTracker** (20 tests)
   - `tests/api/orchestrator/test_request_orchestrator.py` (6 tests)
   - `tests/api/orchestrator/test_request_orchestrator_error_paths.py` (14 tests)

2. **HTTPException 401: Provider requires API key passthrough** (14 tests)
   - Same files - caused by MagicMock `name` kwargot behavior

## Work Completed

### Step 1: Fixed conftest.py module clearing ✅
- Added `src.core.dependencies` to `modules_to_clear` list in `tests/conftest.py:160`
- Added `AsyncMock` import to support fixture creation
- Added `mock_http_request_with_app_state()` fixture helper for reusable mock requests

### Step 2: Added helper functions to orchestrator test files ✅

**tests/api/orchestrator/test_request_orchestrator.py:**
- Created `_create_mock_provider_manager()` helper with `spec_set` for attribute control
- Created `_create_mock_model_manager()` helper for model manager mocking
- Created `_create_mock_config()` helper that properly delegates provider_manager methods and uses `spec_set` to prevent auto-attribute creation
- Fixed `mock_provider_config` creation to set attributes via assignment instead of constructor kwargs
- Added `mock_http_request.app.state.request_tracker` setup using `create_request_tracker()`

**tests/api/orchestrator/test_request_orchestrator_error_paths.py:**
- Added same helper functions for consistency
- Updated all tests to use new helpers

### Step 3: Fixed patch targets ✅ (CORRECTED)
- Initially changed from `"src.api.orchestrator.request_orchestrator.get_request_tracker"` to `"src.core.metrics.runtime.get_request_tracker"`
- **This was incorrect** - patches must target the import location, not definition location
- **Final fix:** Changed back to `"src.api.orchestrator.request_orchestrator.get_request_tracker"` (import location)
- **Lesson:** When patching, always target where the function is **imported and used**, not where it's defined

### Step 4: Removed module clearing for request_orchestrator ✅
- Removed `"src.api.orchestrator.request_orchestrator"` from `modules_to_clear` in `tests/conftest.py` to fix mock issues caused by module clearing
- Restored top-level import of `RequestOrchestrator` in test files

## Critical Issues Discovered

### Issue 1: MagicMock Constructor Behavior

**The `name` kwargot in `MagicMock(name="openai", ...)` does NOT set the `name` attribute!**

It sets the mock's internal name for repr/debugging. The `name` attribute becomes a MagicMock which is truthy, causing the passthrough validation to fail.

**Correct pattern:**
```python
mock_provider_config = MagicMock()
mock_provider_config.name = "openai"
mock_provider_config.uses_passthrough = False
```

**Incorrect pattern (causes failures):**
```python
mock_provider_config = MagicMock(name="openai", uses_passthrough=False)
# ^ This sets the mock's repr name, not the attributes!
```

### Issue 2: MagicMock Auto-Attribute Creation

When using `MagicMock` for `mock_config`, accessing any undefined attribute (like `middleware_chain`) auto-creates a new MagicMock. This breaks the `hasattr()` check in `_apply_middleware_preprocessing()`.

**Solution:** Use `spec_set` on MagicMock to prevent auto-attribute creation:
```python
spec = [
    "log_request_metrics",
    "provider_manager",
    "get_provider_config",
    "get_client",
    "get_next_provider_api_key",
    # Only include "middleware_chain" if has_middleware=True
]
mock_config = MagicMock(spec_set=spec)
```

### Issue 3: Module Clearing Interference

The autouse fixture clears `src.api.orchestrator.request_orchestrator` before each test. When patches are applied and then `RequestOrchestrator` is imported inside the patch context, a fresh module is loaded that may not properly interact with pre-created mocks.

**Solution:** Remove `request_orchestrator` from the module clearing list since it doesn't have module-level config imports that would cause issues.

## Current Status

- **Started with:** 34 failing tests
- **After all fixes:** 0 failing tests
  - All 25 tests in `test_request_orchestrator.py` pass
  - All 14 tests in `test_request_orchestrator_error_paths.py` pass
  - Full `make pre-commit` suite passes (427 passed, 7 skipped)

## Root Cause of Final Fix

The issue was the **incorrect patch target** for `get_request_tracker`. The patch was targeting:
- **Wrong:** `"src.core.metrics.runtime.get_request_tracker"` (definition location)
- **Correct:** `"src.api.orchestrator.request_orchestrator.get_request_tracker"` (import location)

When patching imports, you must patch where the function is **imported and used**, not where it's defined. The `request_orchestrator` module imports `get_request_tracker` at line 21, so that's the import path that needs to be patched.

Additionally, two tests were using the incorrect `MagicMock` constructor pattern:
```python
# WRONG - kwargs create mock methods, not attributes with values
mock_provider_config = MagicMock(name="openai", uses_passthrough=False)

# CORRECT - set attributes after creation
mock_provider_config = MagicMock()
mock_provider_config.name = "openai"
mock_provider_config.uses_passthrough = False
```

## Final Changes Made

1. **Fixed patch targets** in both test files:
   - Changed from `"src.core.metrics.runtime.get_request_tracker"` to `"src.api.orchestrator.request_orchestrator.get_request_tracker"`
   - This affected 2 tests in `test_request_orchestrator.py` and 3 tests in `test_request_orchestrator_error_paths.py`

2. **Fixed mock setup** in 2 tests:
   - `test_orchestrator_no_middleware_when_not_configured`
   - `test_orchestrator_context_contains_all_required_fields`
   - Changed from direct `MagicMock()` with kwargs to using `_create_mock_config()` helper
   - Changed `MagicMock(name="openai", uses_passthrough=False)` to setting attributes via assignment

## Remaining Work

**None.** All tasks have been completed:
- ✅ Fixed 34 failing tests
- ✅ All helper functions implemented
- ✅ Patch targets corrected
- ✅ Module clearing configured correctly
- ✅ Full `make pre-commit` suite passes (427 passed, 7 skipped)

---

# Lessons Learned

This section documents the key pitfalls encountered during this debugging session. Understanding these will help you avoid similar issues when writing or maintaining tests.

## 1. The Patch Target Rule (Most Critical)

**When patching imports, always target where the function is IMPORTED, not where it's DEFINED.**

### The Problem

```python
# In src/api/orchestrator/request_orchestrator.py:
from src.core.metrics.runtime import get_request_tracker  # Line 21

# In your test, this WON'T WORK:
patch("src.core.metrics.runtime.get_request_tracker", ...)  # ❌ Definition location

# This WILL WORK:
patch("src.api.orchestrator.request_orchestrator.get_request_tracker", ...)  # ✅ Import location
```

### Why This Matters

When you patch at the definition location (`src.core.metrics.runtime`), you're patching the original function. But the module under test (`request_orchestrator`) has already imported the function and holds a reference to the original. Your patch doesn't affect that reference.

When you patch at the import location (`request_orchestrator.get_request_tracker`), you're replacing the reference that the module under test actually uses.

### How to Find the Correct Patch Target

1. Look at the imports in the file you're testing
2. Find the line that imports the function you want to patch
3. Use that full path as your patch target

```python
# In your test file:
from src.api.orchestrator.request_orchestrator import RequestOrchestrator

# The patch target must match the import path in request_orchestrator.py
patch("src.api.orchestrator.request_orchestrator.get_request_tracker")
```

## 2. MagicMock Constructor Kwargs Don't Set Attributes

**The `name` kwarg (and other kwargs) in `MagicMock()` do NOT set attributes - they create mock methods.**

### The Problem

```python
# ❌ WRONG - This does NOT set the 'name' attribute!
mock_config = MagicMock(name="openai", uses_passthrough=False)
print(mock_config.name)  # <MagicMock name='mock.name' id='...'>
# The 'name' attribute is a MagicMock object, which is truthy!

# ✅ CORRECT - Set attributes after creation
mock_config = MagicMock()
mock_config.name = "openai"
mock_config.uses_passthrough = False
print(mock_config.name)  # 'openai' (actual string value)
print(mock_config.uses_passthrough)  # False (actual bool value)
```

### Why This Matters

When you use `MagicMock(name="value")`, the `name` kwarg sets the mock's **internal repr name** (used for debugging/display), NOT an attribute called `name`. The attribute `mock.name` becomes a new MagicMock object, which is always truthy.

This causes conditions like `if provider_config.uses_passthrough:` to evaluate to `True` even when you intended `False`, leading to unexpected test failures.

### The Special Case of the `name` Kwarg

The `name` kwarg is particularly confusing because:
- It has a dual purpose (repr name vs potential attribute name)
- It's commonly used as both a real attribute name and a mock kwarg
- The behavior differs subtly between `Mock`, `MagicMock`, and `AsyncMock`

### Best Practice

```python
# Always create mocks first, then set attributes
mock_config = MagicMock(spec_set=["name", "uses_passthrough", "other_attr"])
mock_config.name = "actual_value"
mock_config.uses_passthrough = False
```

## 3. MagicMock Auto-Creates Attributes

**Accessing any undefined attribute on a MagicMock creates a new MagicMock automatically.**

### The Problem

```python
mock_config = MagicMock()  # No spec_set
print(hasattr(mock_config, "middleware_chain"))  # True (auto-created!)
print(type(mock_config.middleware_chain))  # <class 'unittest.mock.MagicMock'>
```

This breaks `hasattr()` checks in your code:

```python
# In your production code:
if hasattr(config, "middleware_chain"):
    apply_middleware(config.middleware_chain)  # This runs even when you don't want it to!
```

### The Solution: Use `spec_set`

```python
# ✅ Prevents auto-attribute creation
mock_config = MagicMock(spec_set=["log_request_metrics", "provider_manager"])
print(hasattr(mock_config, "middleware_chain"))  # False (correct!)
mock_config.middleware_chain  # Raises AttributeError (correct!)
```

## 4. Module Clearing Interferes with Patching

**Be careful when using pytest autouse fixtures that clear modules - they can break your patches.**

### The Problem

```python
# In conftest.py:
@pytest.fixture(autouse=True)
def clear_modules():
    yield
    sys.modules.pop("src.api.orchestrator.request_orchestrator", None)

# In your test:
@patch("src.api.orchestrator.request_orchestrator.get_request_tracker")
def test_something(mock_get_tracker):
    from src.api.orchestrator.request_orchestrator import RequestOrchestrator
    # The import happens AFTER module clearing, loading a fresh module
    # The patch may not apply to the freshly imported code!
```

### The Fix

Only clear modules that have module-level state that needs resetting. Don't clear modules just for testing purposes:

```python
# ✅ Clear modules with actual module-level state
modules_to_clear = ["src.core.dependencies"]  # Has singletons

# ❌ Don't clear modules without state
# modules_to_clear = ["src.api.orchestrator.request_orchestrator"]  # No state, only classes
```

## 5. Use Helper Functions for Consistent Mock Setup

**Creating helper functions for mock setup prevents inconsistencies and reduces boilerplate.**

### Before (Inconsistent and Error-Prone)

```python
def test_1():
    mock_config = MagicMock()
    mock_config.log_request_metrics = True
    mock_config.provider_manager = MagicMock()
    mock_config.provider_manager.get_provider_config = MagicMock(return_value=...)
    # ...lots of repetitive setup

def test_2():
    mock_config = MagicMock()
    mock_config.log_request_metrics = True
    mock_config.provider_manager = MagicMock()
    # ...slightly different setup, potential bugs
```

### After (Consistent and Maintainable)

```python
def _create_mock_config(log_request_metrics=False, provider_config=None):
    """Create a consistently configured mock config."""
    spec = ["log_request_metrics", "provider_manager", "get_provider_config", ...]
    mock_config = MagicMock(spec_set=spec)
    mock_config.log_request_metrics = log_request_metrics
    # ...centralized setup logic
    return mock_config

def test_1():
    mock_config = _create_mock_config(log_request_metrics=True)

def test_2():
    mock_config = _create_mock_config(log_request_metrics=True)
```

## 6. RequestTracker Setup with Global Metrics

**When `LOG_REQUEST_METRICS` is globally enabled in conftest, tests must properly set up `app.state.request_tracker`.**

### The Problem

```python
# In conftest.py:
os.environ["LOG_REQUEST_METRICS"] = "true"  # Globally enabled

# In your test:
mock_http_request = MagicMock()  # ❌ Missing app.state.request_tracker!

# When code calls get_request_tracker(mock_http_request):
# It fails because app.state.request_tracker is a MagicMock, not a real RequestTracker
```

### The Fix

```python
# Always set up the request tracker when metrics are enabled
from src.core.metrics import create_request_tracker

mock_http_request = MagicMock()
mock_http_request.app = MagicMock()
mock_http_request.app.state.request_tracker = create_request_tracker()  # ✅ Real tracker
```

---

## Summary

All 34 failing tests have been fixed. The key learnings from this debugging session:

1. **Patch Target Rule:** When patching imports, target the **import location**, not the definition location.
   - Target: `"module.that.imports.function_name"`
   - Not: `"module.where.function.is.defined"`

2. **MagicMock Constructor Kwargs:** Using kwargs like `MagicMock(name="value")` does NOT set attributes - it creates mock methods.
   - Use `mock.attr = value` instead of `MagicMock(attr=value)` for attributes

3. **Use Helper Functions:** The `_create_mock_config()` helper ensures consistent, correct mock setup across tests.

4. **Module Clearing Side Effects:** Be careful when clearing modules with pytest autouse fixtures - it can interfere with patching.

## Tips for Next Person

1. **Always verify MagicMock attribute access:**
   - Use `mock.attr = value` not `MagicMock(attr=value)` for attributes
   - The `name` kwargot is special - it sets repr name, not an attribute!

2. **RequestOrchestrator uses config as client_factory:**
   - When `client_factory` parameter is None, it defaults to `config`
   - So `config` must have `get_provider_config`, `get_client`, and `get_next_provider_api_key` methods
   - Delegating from `provider_manager` works: `config.get_provider_config = provider_manager.get_provider_config`

3. **Test isolation requires proper module clearing:**
   - `src.core.dependencies` must be in `modules_to_clear` to reset singletons
   - BUT `src.api.orchestrator.request_orchestrator` should NOT be cleared to avoid mock issues

4. **MagicMock auto-attribute creation:**
   - `hasattr(mock, "anything")` returns `True` for MagicMock
   - Use `spec_set` to limit available attributes
   - Consider using `SimpleNamespace` for plain objects without mocking behavior

5. **Module clearing and patching interaction:**
   - Module clearing happens BEFORE test function runs (via autouse fixture)
   - Patches are applied INSIDE test function
   - If a module is cleared and re-imported inside a patch context, the patch may not affect the freshly imported code
   - Solution: Either don't clear the module, or import it AFTER patches are applied

6. **For HTTP mock setup:**
   - Always set up `mock_http_request.app.state.request_tracker` using `create_request_tracker()` when `LOG_REQUEST_METRICS` is globally enabled
   - This is required even if you patch `get_request_tracker`, because the real function is called first

## Files Modified

- `tests/conftest.py` - Removed `request_orchestrator` from module clearing
- `tests/api/orchestrator/test_request_orchestrator.py` - Added helper functions, fixed mock setup
- `tests/api/orchestrator/test_request_orchestrator_error_paths.py` - Added helper functions, fixed mock setup
