# Story 11.8: Configure Loop Stall Detection Thresholds

**Status:** Draft
**Priority:** Low
**Depends on:** None

---

## Context / Background

The test suite includes loop stall detection to catch async code that blocks the event loop. The threshold is configured via `FAPILOG_TEST_MAX_LOOP_STALL_SECONDS`:

**Current configuration (inconsistent):**

| Location | Value | Purpose |
|----------|-------|---------|
| `tox.ini` | 0.25s | Default for tox runs |
| `pyproject.toml` | 0.25s | Default for direct pytest |
| `.env.test` | 0.10s | Very strict (may cause flakiness) |
| `test_rotating_file_sink_integration.py` | 0.035s | Override for specific test |
| `test_load_metrics.py` | 0.20s / 0.25s | Per-test overrides |

The 0.25s default may be too tight for slow CI runners, causing false positives. The review recommends increasing to 0.5s or making it configurable per-environment.

---

## Scope (In / Out)

### In Scope

- Standardize loop stall threshold configuration
- Remove per-test overrides where possible
- Make threshold CI-aware (higher for slow runners)
- Document threshold tuning guidelines

### Out of Scope

- Changing the loop stall detection mechanism itself
- Adding new stall detection features
- Production code changes

---

## Acceptance Criteria

### AC1: Single Source of Truth for Threshold

**Description:** Loop stall threshold is configured in one place, not scattered across files.

**Validation:**
```bash
grep -r "LOOP_STALL" tests/ --include="*.py" | grep -v "os.getenv" | wc -l
# Expected: 0 (all uses go through env var)
```

### AC2: CI Uses Appropriate Threshold

**Description:** CI environments use a higher threshold (0.5s) than local development (0.25s).

**Validation:**
```yaml
# tox.ini or CI config
FAPILOG_TEST_MAX_LOOP_STALL_SECONDS = 0.5  # For CI
```

### AC3: Threshold Helper Function

**Description:** Tests use a helper to get the threshold, allowing environment-based adjustment.

**Validation:**
```python
from tests.conftest import get_loop_stall_threshold

threshold = get_loop_stall_threshold()  # Returns env-appropriate value
```

### AC4: Documentation Updated

**Description:** CI configuration docs explain threshold tuning.

**Validation:**
- `docs/ci-configuration.md` explains the setting
- Clear guidance on when to increase threshold

---

## Implementation Notes

### Standardize Configuration

Remove per-test overrides and use environment variable consistently:

```python
# Before (in test file)
if "FAPILOG_TEST_MAX_LOOP_STALL_SECONDS" not in os.environ:
    os.environ["FAPILOG_TEST_MAX_LOOP_STALL_SECONDS"] = "0.035"

# After (use conftest helper)
from tests.conftest import get_loop_stall_threshold

threshold = get_loop_stall_threshold()
```

### Helper Function

Add to `tests/conftest.py`:

```python
import os

def get_loop_stall_threshold(default: float = 0.25) -> float:
    """Get loop stall detection threshold.

    In CI environments (detected via CI env var), uses a higher threshold
    to account for slower runners.

    Args:
        default: Default threshold for local development

    Returns:
        Threshold in seconds
    """
    # Check for explicit override first
    explicit = os.getenv("FAPILOG_TEST_MAX_LOOP_STALL_SECONDS")
    if explicit:
        return max(float(explicit), 0.10)  # Minimum 0.10s

    # Use higher threshold in CI
    if os.getenv("CI"):
        return 0.50

    return default
```

### Configuration Hierarchy

1. **Explicit env var** - `FAPILOG_TEST_MAX_LOOP_STALL_SECONDS` takes precedence
2. **CI detection** - If `CI=true`, use 0.5s
3. **Default** - Use 0.25s for local development

### Remove .env.test Override

The `.env.test` file sets 0.10s which is too strict:

```bash
# .env.test (before)
FAPILOG_TEST_MAX_LOOP_STALL_SECONDS=0.10

# .env.test (after) - remove or increase
FAPILOG_TEST_MAX_LOOP_STALL_SECONDS=0.25
```

### File Changes

```
tests/conftest.py (MODIFIED - add get_loop_stall_threshold)
tests/integration/test_rotating_file_sink_integration.py (MODIFIED - use helper)
tests/integration/test_load_metrics.py (MODIFIED - use helper)
.env.test (MODIFIED - remove strict override)
tox.ini (MODIFIED - set 0.5s for CI)
docs/ci-configuration.md (MODIFIED - document threshold)
```

---

## Tasks

### Phase 1: Create Helper

- [ ] Add `get_loop_stall_threshold()` to conftest.py
- [ ] Add unit test for helper function
- [ ] Document in docstring

### Phase 2: Standardize Configuration

- [ ] Update tox.ini to use 0.5s for CI
- [ ] Remove or update .env.test strict value
- [ ] Remove per-test overrides in test_rotating_file_sink_integration.py
- [ ] Remove per-test overrides in test_load_metrics.py

### Phase 3: Migrate Tests

- [ ] Update tests to use `get_loop_stall_threshold()` helper
- [ ] Verify tests pass with new thresholds

### Phase 4: Documentation

- [ ] Update docs/ci-configuration.md
- [ ] Add CHANGELOG entry

---

## Tests

### Helper Function Test

```python
def test_loop_stall_threshold_default(monkeypatch):
    monkeypatch.delenv("FAPILOG_TEST_MAX_LOOP_STALL_SECONDS", raising=False)
    monkeypatch.delenv("CI", raising=False)
    assert get_loop_stall_threshold() == 0.25

def test_loop_stall_threshold_ci(monkeypatch):
    monkeypatch.delenv("FAPILOG_TEST_MAX_LOOP_STALL_SECONDS", raising=False)
    monkeypatch.setenv("CI", "true")
    assert get_loop_stall_threshold() == 0.50

def test_loop_stall_threshold_explicit(monkeypatch):
    monkeypatch.setenv("FAPILOG_TEST_MAX_LOOP_STALL_SECONDS", "0.75")
    assert get_loop_stall_threshold() == 0.75
```

### Regression

```bash
pytest tests/integration/test_rotating_file_sink_integration.py -v
pytest tests/integration/test_load_metrics.py -v
```

---

## Definition of Done

### Code Complete

- [ ] Helper function implemented
- [ ] Per-test overrides removed
- [ ] Configuration standardized

### Quality Assurance

- [ ] Tests pass locally with default threshold
- [ ] Tests pass in CI with CI threshold
- [ ] `ruff check` passes

### Documentation

- [ ] CI configuration documented
- [ ] CHANGELOG updated

---

## Risks / Rollback

### Risks

1. **Risk:** Higher threshold misses real stalls
   - **Mitigation:** 0.5s still catches significant stalls; keep local at 0.25s

2. **Risk:** Some tests need specific thresholds
   - **Mitigation:** Allow per-test override via parameter, not env manipulation

### Rollback Plan

If issues occur:
1. Re-add per-test overrides where needed
2. Document why specific tests need custom thresholds

---

## Related Stories

- **Related:** Story 11.4 - CI timeout multipliers (similar pattern)
- **Related:** Story 11.2 - flaky tests may be affected by stall detection

---

## Change Log

| Date | Change | Author |
|------|--------|--------|
| 2026-01-14 | Initial draft | Claude |
