# Story 11.1: Replace Hard Sleeps with Polling in Integration Tests

**Status:** Ready
**Priority:** Low
**Depends on:** None

---

## Context / Background

Integration tests use fixed `asyncio.sleep()` calls when waiting for external systems:

| File | Line | Sleep Duration | Purpose | Impact |
|------|------|----------------|---------|--------|
| `tests/integration/test_loki_sink.py` | 37 | 1.0 second | Wait for Loki to index data | Low (integration test, skipped in normal CI) |
| `tests/integration/test_rotating_file_sink_integration.py` | 99 | 1.2 seconds | Cross time-based rotation boundary | Very Low (intentional delay for testing interval rotation) |

**Note:** These are `@pytest.mark.integration` tests that require external services (real Loki server). They are **skipped in normal CI runs** and only execute in dedicated integration test environments. This lowers their priority compared to unit test flakiness.

### Loki Test Analysis

The Loki test sleep is **not** waiting for the client to flush — `sink.stop()` already ensures the push completes. The sleep waits for **Loki server-side indexing** so the data can be queried:

```python
# test_loki_sink.py:33-44
await sink.start()
await sink.write({"level": "INFO", "message": message})
await sink.stop()  # Push is complete here

await asyncio.sleep(1.0)  # Waiting for Loki to INDEX the data

resp = await client.get(query_url)  # Query may fail if indexing incomplete
assert message in resp
```

**Current pattern (flaky):**
```python
await sink.stop()
await asyncio.sleep(1.0)  # Hope Loki indexes within 1s
assert message in query_response  # Fails if Loki is slow
```

**Target pattern (reliable):**
```python
await sink.stop()
# Poll until data appears or timeout
for _ in range(50):
    if message in (await client.get(query_url)).text:
        break
    await asyncio.sleep(0.2)
else:
    pytest.fail("Loki did not index within 10s")
```

### Rotating File Test Analysis

The 1.2s sleep in `test_rotating_file_sink_integration.py` is **intentional** — it tests time-based rotation by crossing a 1-second interval boundary. This is not a synchronization issue but a deliberate test of the feature. Consider reducing `interval_seconds` to make the test faster, but the pattern itself is correct.

---

## Scope (In / Out)

### In Scope

- Replace the 1.0s sleep in `test_loki_sink.py` with polling-based wait
- Optionally reduce the rotation interval in `test_rotating_file_sink_integration.py` to speed up the test

### Out of Scope

- Adding coordination primitives to production sink code (not needed)
- Changing the rotating file test pattern (it's testing time-based rotation correctly)
- Unit test timing issues — covered in Story 11.2
- Smaller sleeps (< 0.5s) — covered in Story 11.5

---

## Acceptance Criteria

### AC1: Loki Test Uses Polling Instead of Fixed Sleep

**Description:** `test_loki_sink.py` polls Loki until data is indexed, rather than sleeping for a fixed duration.

**Validation:**
```python
# Before: await asyncio.sleep(1.0)
# After:
async def test_loki_sink_push_and_query():
    ...
    await sink.stop()

    # Poll Loki until data is indexed (max 10s)
    async with httpx.AsyncClient() as client:
        for _ in range(50):
            resp = await client.get(query_url, params={"query": query})
            if message in resp.text:
                break
            await asyncio.sleep(0.2)
        else:
            pytest.fail(f"Loki did not index message within 10s")

        assert resp.status_code == 200
        # ... rest of assertions
```

### AC2: Polling Has Reasonable Timeout

**Description:** The polling loop has a maximum timeout (10s) to fail fast if Loki is broken.

**Validation:**
- Polling interval: 0.2s (short enough to detect quickly)
- Max attempts: 50 (10s total timeout)
- Clear failure message if timeout exceeded

### AC3: Test Passes Consistently

**Description:** The Loki integration test passes reliably when Loki is available.

**Validation:**
```bash
# Requires LOKI_URL to be set
for i in {1..5}; do
    pytest tests/integration/test_loki_sink.py -v || exit 1
done
```

### AC4: (Optional) Rotating File Test Runs Faster

**Description:** Reduce `interval_seconds` from 1.0 to 0.5 and sleep from 1.2s to 0.6s.

**Validation:**
```python
# Before
interval_seconds=1,
await asyncio.sleep(1.2)

# After
interval_seconds=0.5,
await asyncio.sleep(0.6)
```

---

## Implementation Notes

### Polling Helper Function

Create a reusable polling helper for integration tests:

```python
# tests/integration/conftest.py or inline
async def poll_until(
    predicate: Callable[[], Awaitable[bool]],
    timeout: float = 10.0,
    interval: float = 0.2,
    description: str = "condition",
) -> None:
    """Poll until predicate returns True or timeout is reached."""
    deadline = time.monotonic() + timeout
    while time.monotonic() < deadline:
        if await predicate():
            return
        await asyncio.sleep(interval)
    pytest.fail(f"{description} not met within {timeout}s")
```

### Loki Test Fix

```python
# test_loki_sink.py
async def test_loki_sink_push_and_query() -> None:
    message = f"loki-test-{int(time.time())}"
    sink = LokiSink(LokiSinkConfig(...))
    await sink.start()
    await sink.write({"level": "INFO", "message": message})
    await sink.stop()

    # Poll until Loki has indexed the data
    async with httpx.AsyncClient() as client:
        query_url = f"{LOKI_URL.rstrip('/')}/loki/api/v1/query"

        async def check_indexed() -> bool:
            resp = await client.get(query_url, params={"query": '{service="fapilog-integration"}'})
            return resp.status_code == 200 and message in resp.text

        await poll_until(check_indexed, timeout=10.0, description="Loki indexing")

        # Final assertion
        resp = await client.get(query_url, params={"query": '{service="fapilog-integration"}'})
        assert resp.status_code == 200
        data = resp.json()
        results = data.get("data", {}).get("result", [])
        assert any(message in v for r in results for _, v in r.get("values", []))
```

### File Changes

```
tests/integration/test_loki_sink.py (MODIFIED - replace sleep with polling)
tests/integration/test_rotating_file_sink_integration.py (OPTIONALLY MODIFIED - reduce interval)
```

**No production code changes required.**

---

## Tasks

### Phase 1: Loki Test Fix

- [ ] Add `poll_until` helper to `tests/integration/conftest.py`
- [ ] Replace `asyncio.sleep(1.0)` with polling loop in `test_loki_sink.py`
- [ ] Verify test passes with real Loki (requires `LOKI_URL`)

### Phase 2: (Optional) Rotating File Test Optimization

- [ ] Reduce `interval_seconds` from 1.0 to 0.5
- [ ] Reduce sleep from 1.2s to 0.6s
- [ ] Verify test still validates time-based rotation correctly

### Phase 3: Validation

- [ ] Run integration tests with Loki available
- [ ] Verify no regressions

---

## Tests

### Verification

The Loki integration test itself serves as verification — it should pass consistently when Loki is available.

```bash
# Requires LOKI_URL environment variable
LOKI_URL=http://localhost:3100 pytest tests/integration/test_loki_sink.py -v
```

---

## Definition of Done

### Code Complete

- [ ] `poll_until` helper added to integration test utilities
- [ ] Loki test uses polling instead of fixed sleep
- [ ] Polling has 10s timeout with clear failure message

### Quality Assurance

- [ ] Test passes with real Loki instance
- [ ] `ruff check` passes
- [ ] `mypy` passes

### Documentation

- [ ] CHANGELOG updated

---

## Risks / Rollback

### Risks

1. **Risk:** Polling adds complexity to test
   - **Mitigation:** Use simple, readable helper function

2. **Risk:** 10s timeout too short for very slow Loki
   - **Mitigation:** Timeout is configurable; 10s is generous for indexing

### Rollback Plan

If issues occur:
1. Revert to original `asyncio.sleep(1.0)`
2. Increase sleep duration as temporary fix

---

## Related Stories

- **Related:** Story 11.2 - may use similar polling patterns for other flaky tests
- **Related:** Story 11.5 - addresses smaller sleep instances in unit tests

---

## Change Log

| Date | Change | Author |
|------|--------|--------|
| 2026-01-14 | Initial draft | Claude |
