# Story 11.7: Optimize Disk I/O in Rotating File Tests

**Status:** Draft
**Priority:** Low
**Depends on:** None

---

## Context / Background

The rotating file sink tests create and delete many files during execution. On shared CI runners with slow or contended I/O, this can cause timeouts and flaky behavior.

**Current approach:**
```python
def test_rotation_creates_new_file(tmp_path):
    sink = RotatingFileSink(path=tmp_path / "test.log", max_size=1024)
    # Write enough to trigger rotation
    for i in range(100):
        await sink.write(large_record)
    # Disk I/O for each write + rotation
```

**Potential optimizations:**
1. **Memory-backed storage** - Use `tempfile` with memory-backed filesystem
2. **RAM disk in CI** - Mount `/tmp` as tmpfs
3. **Batch operations** - Reduce number of individual I/O operations
4. **Smaller test files** - Use smaller rotation thresholds

---

## Scope (In / Out)

### In Scope

- Profile disk I/O in rotating file tests
- Implement memory-backed storage where feasible
- Reduce unnecessary file operations
- Document CI optimization options (RAM disk)

### Out of Scope

- Changing production sink behavior
- Rewriting file sink implementation
- CI infrastructure changes (documenting only)

---

## Acceptance Criteria

### AC1: Tests Use Memory-Backed Storage Where Possible

**Description:** Tests that don't require real filesystem use memory-backed alternatives.

**Validation:**
```python
# Using pytest's tmp_path with potential memory backing
def test_rotation_logic(tmp_path):
    # tmp_path already uses system temp, which may be RAM-backed
    ...
```

### AC2: Reduced File Operations

**Description:** Tests minimize unnecessary file creates/deletes.

**Validation:**
- Tests use smaller rotation thresholds (e.g., 1KB instead of 1MB)
- Tests clean up files in `finally` blocks, not separate operations
- Batch writes where possible

### AC3: CI Documentation Updated

**Description:** Documentation explains how to configure RAM-backed `/tmp` for faster tests.

**Validation:**
```markdown
# docs/ci-configuration.md
## Optimizing Disk I/O

For faster rotating file tests, mount /tmp as tmpfs:
```yaml
- name: Setup RAM disk
  run: sudo mount -t tmpfs -o size=512m tmpfs /tmp
```
```

### AC4: Test Execution Time Reduced

**Description:** Rotating file tests complete faster after optimization.

**Validation:**
```bash
# Before optimization
time pytest tests/unit/test_rotating_file_sink.py
# real: X.XXs

# After optimization
time pytest tests/unit/test_rotating_file_sink.py
# real: < 0.8 * X.XXs (20% improvement)
```

---

## Implementation Notes

### Option 1: Memory-Mapped Temp Directory

Python's `tempfile` module can use memory-backed storage on systems with tmpfs:

```python
import tempfile
import os

@pytest.fixture
def memory_tmp_path():
    """Create temp directory in memory-backed storage if available."""
    # Check if /dev/shm exists (Linux shared memory)
    if os.path.exists("/dev/shm"):
        base = "/dev/shm"
    else:
        base = None  # Fall back to system default

    with tempfile.TemporaryDirectory(dir=base) as tmpdir:
        yield Path(tmpdir)
```

### Option 2: Smaller Test Files

Reduce rotation thresholds for faster tests:

```python
# Before
sink = RotatingFileSink(max_size=1024 * 1024)  # 1MB rotation

# After
sink = RotatingFileSink(max_size=1024)  # 1KB rotation for tests
```

### Option 3: Mock Filesystem for Unit Tests

For pure unit tests, mock the filesystem:

```python
from unittest.mock import MagicMock, patch

def test_rotation_logic_unit():
    with patch("builtins.open", MagicMock()):
        sink = RotatingFileSink(...)
        # Test logic without actual I/O
```

### Option 4: CI RAM Disk Configuration

Document in CI configuration:

```yaml
# .github/workflows/test.yml
jobs:
  test:
    steps:
      - name: Setup RAM disk for tests
        run: |
          sudo mkdir -p /mnt/ramdisk
          sudo mount -t tmpfs -o size=256m tmpfs /mnt/ramdisk
          echo "TMPDIR=/mnt/ramdisk" >> $GITHUB_ENV
```

### File Changes

```
tests/unit/test_rotating_file_sink.py (MODIFIED - use smaller thresholds)
tests/integration/test_rotating_file_sink_integration.py (MODIFIED - optimize I/O)
tests/conftest.py (MODIFIED - add memory_tmp_path fixture)
docs/ci-configuration.md (MODIFIED - document RAM disk option)
```

---

## Tasks

### Phase 1: Profile Current State

- [ ] Measure current test execution time
- [ ] Identify tests with most I/O operations
- [ ] Count file creates/deletes per test

### Phase 2: Optimize Tests

- [ ] Reduce rotation thresholds in unit tests
- [ ] Add memory-backed temp directory fixture
- [ ] Update tests to use optimized fixtures
- [ ] Remove unnecessary intermediate file operations

### Phase 3: Documentation

- [ ] Document RAM disk setup for CI
- [ ] Add comments explaining I/O optimizations
- [ ] Update CHANGELOG

### Phase 4: Validation

- [ ] Measure execution time improvement
- [ ] Verify tests still pass
- [ ] Run on CI to verify improvement

---

## Tests

### Performance Verification

```bash
# Before optimization
time pytest tests/unit/test_rotating_file_sink.py -v 2>&1 | tail -5

# After optimization
time pytest tests/unit/test_rotating_file_sink.py -v 2>&1 | tail -5
```

### Regression

```bash
pytest tests/unit/test_rotating_file_sink.py -v
pytest tests/integration/test_rotating_file_sink_integration.py -v
```

---

## Definition of Done

### Code Complete

- [ ] Memory-backed temp directory fixture available
- [ ] Rotation thresholds reduced for tests
- [ ] Unnecessary file operations removed

### Quality Assurance

- [ ] Tests pass locally
- [ ] Tests pass in CI
- [ ] Execution time improved by 20%+

### Documentation

- [ ] CI RAM disk setup documented
- [ ] CHANGELOG updated

---

## Risks / Rollback

### Risks

1. **Risk:** Memory-backed storage behaves differently than real disk
   - **Mitigation:** Keep integration tests on real filesystem

2. **Risk:** Smaller thresholds miss edge cases
   - **Mitigation:** Keep some tests with realistic thresholds

### Rollback Plan

If issues occur:
1. Revert to original fixtures
2. Keep documentation for future optimization

---

## Related Stories

- **Related:** Story 11.6 - split files may reduce per-file I/O
- **Related:** Story 11.9 - parallel execution may stress I/O more

---

## Change Log

| Date | Change | Author |
|------|--------|--------|
| 2026-01-14 | Initial draft | Claude |
