# Task Recon: Implement cache metadata and staleness detection

**Task ID:** 5.3
**Status:** pending
**Generated:** 2026-01-28T12:00:00Z
**Agent:** claude-opus-4-5-20251101

---

## Task Summary

Add methods to track CSV cache metadata including last update time and staleness detection based on CSV_STALE_DAYS constant. Implement get_csv_metadata, get_cache_age_days, and is_cache_stale methods.

### Parent Task Context

**Parent Task:** Implement CacheManager (ID: 5)
**Parent Goal:** Create the CacheManager for token caching, CSV file reading, and response caching with TTL support.
**This Subtask's Role:** This subtask completes the CacheManager by adding cache staleness detection. The CSV cache (locations.csv, policies.csv) is populated via PowerShell export, and the CLI needs to detect when data is outdated to prompt users to refresh.

### Dependencies
- Depends on: 5.1 (Token storage - provides CacheManager class structure) - **DONE**
- Depends on: 5.2 (CSV reading - provides read_locations_csv, read_policies_csv) - **DONE**

### Acceptance Criteria
1. Define `CacheMetadata` Pydantic model with fields:
   - `last_updated: datetime | None`
   - `locations_count: int`
   - `policies_count: int`
2. `get_csv_metadata() -> CacheMetadata | None` that:
   - Checks locations.csv and policies.csv existence
   - Returns mtime as `last_updated` (oldest of the two files)
   - Counts rows in each file
   - Returns None if neither file exists
3. `get_cache_age_days() -> int | None` that returns days since last_updated or None if no metadata
4. `is_cache_stale() -> bool` that returns True if:
   - `cache_age_days` is None (no cache exists)
   - `cache_age_days` exceeds CSV_STALE_DAYS (30 days per constants.py)
5. Use `datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc)` for cross-platform mtime handling

---

## Insights from Previous Tasks

### Dependency Task Notes
| Task ID | Title | Key Insight |
|---------|-------|-------------|
| 5.1 | Token storage and retrieval | CacheManager class structure, `_cache_dir` property, graceful error handling patterns |
| 5.2 | CSV reading | `read_locations_csv()`, `read_policies_csv()`, `_read_csv()` helper already implemented |

### Reusable Patterns from Dependencies

**From Subtask 5.1 Implementation Notes (task_005.md:46-82):**
- **Path property pattern**: `_cache_dir` is a private Path instance, exposed via `cache_dir` property
- **Silent failure pattern**: Return None or empty values for missing files (graceful degradation)
- **Pydantic JSON serialization**: `model.model_dump_json()` for saving, `Model.model_validate_json(content)` for loading
- **PydanticValidationError alias**: Use `from pydantic import ValidationError as PydanticValidationError` to avoid name collision

**From Subtask 5.2 Implementation Notes (task_005.md:84-118):**
- **UTF-8-sig encoding**: CSV files use `encoding="utf-8-sig"` for BOM handling
- **Empty list on error**: CSV methods return `[]` for missing/malformed files
- **csv.DictReader pattern**: `list(reader)` converts iterator to list of dicts

### Warnings and Gotchas
- **Timezone handling**: Always use `timezone.utc` when working with datetime. See `models/auth.py:50-55` for pattern.
- **Path.stat() may raise**: `FileNotFoundError` if file was deleted between existence check and stat call - handle gracefully.
- **CSV row counts use existing methods**: Call `read_locations_csv()` and `read_policies_csv()` rather than re-implementing row counting to avoid duplication.

### Open Items Inherited
- [x] (From 5.1) Subtask 5.2: Implement CSV file reading methods - **DONE**
- [ ] (From 5.1) Subtask 5.3: Implement staleness detection - **THIS TASK**

---

## Architecture Context

### Relevant Design Decisions
- **CacheManager is lowest-level infrastructure** - has no dependencies on other components (5-components.md:143)
- **CSV_STALE_DAYS = 30** - defined in constants.py:44, used for cache staleness threshold
- **Pydantic strict mode** - all models use `ConfigDict(extra="forbid")` for fail-fast validation

### Module Structure
```
src/teams_phone/
├── infrastructure/
│   ├── __init__.py           # Already exports CacheManager
│   └── cache_manager.py      # MODIFY - add metadata and staleness methods
├── models/
│   ├── __init__.py           # ADD CacheMetadata to exports
│   └── ??? (see recommendation)  # ADD CacheMetadata model
├── constants.py              # CSV_STALE_DAYS = 30 (already defined)
└── exceptions.py             # Not needed - graceful degradation
```

### Conventions to Follow
- Docstrings follow Google style (see existing methods in cache_manager.py)
- Type hints required for all functions (mypy strict mode)
- Pydantic models use `ConfigDict(extra="forbid")` (see auth.py:37)
- Use `datetime.now(timezone.utc)` for current time (see auth.py:50)
- Use `datetime.fromtimestamp(ts, tz=timezone.utc)` for mtime conversion

---

## Related Code References

### Primary References (Must Read)
| File | Lines | Purpose |
|------|-------|---------|
| `src/teams_phone/infrastructure/cache_manager.py` | 44-51 | `cache_dir` property - use to build CSV file paths |
| `src/teams_phone/infrastructure/cache_manager.py` | 149-193 | `_read_csv()`, `read_locations_csv()`, `read_policies_csv()` - reuse for row counting |
| `src/teams_phone/constants.py` | 44-45 | `CSV_STALE_DAYS = 30` - staleness threshold |
| `src/teams_phone/models/auth.py` | 24-55 | `CachedToken` model pattern with datetime field and `is_expired()` method |
| `src/teams_phone/models/auth.py` | 37 | `model_config = ConfigDict(extra="forbid")` pattern |
| `src/teams_phone/models/auth.py` | 50-55 | Datetime comparison pattern with timezone handling |

### Secondary References (Helpful Context)
| File | Lines | Purpose |
|------|-------|---------|
| `src/teams_phone/models/config.py` | 8-14 | How to import constants for defaults |
| `src/teams_phone/models/__init__.py` | 1-19 | How to export new models |
| `src/teams_phone/infrastructure/__init__.py` | 1-6 | Infrastructure exports (CacheManager already exported) |
| `.taskmaster/tasks/task_005.md` | 84-118 | Subtask 5.2 implementation notes |

### Code Patterns to Follow

**Pydantic Model Pattern (auth.py:24-37):**
```python
from datetime import datetime
from pydantic import BaseModel, ConfigDict

class CachedToken(BaseModel):
    """Docstring with Google-style attributes."""

    model_config = ConfigDict(extra="forbid")

    access_token: str
    expires_at: datetime
    tenant_id: str
    scopes: list[str]
```

**Datetime Now with Timezone (auth.py:50):**
```python
from datetime import datetime, timezone

now = datetime.now(timezone.utc)
```

**Datetime from Timestamp (recommended pattern):**
```python
from datetime import datetime, timezone
from pathlib import Path

mtime = Path("file.csv").stat().st_mtime
dt = datetime.fromtimestamp(mtime, tz=timezone.utc)
```

---

## Dependencies

### Internal Modules
- `teams_phone.constants` - `CSV_STALE_DAYS` for staleness threshold
- `teams_phone.models` - Will add `CacheMetadata` export
- `teams_phone.infrastructure.cache_manager` - Existing `CacheManager` class, `read_locations_csv()`, `read_policies_csv()`

### External Packages
- `pydantic` - Already in dependencies, used for `CacheMetadata` model
- `datetime` - stdlib, for datetime operations with timezone

### New Dependencies Needed
- None required - all dependencies already in pyproject.toml

---

## Testing Guidance

### Test File Location
`tests/unit/test_cache_manager.py` (extend existing file)

### Test Patterns in Use
```python
# Example test structure (tests/unit/test_cache_manager.py:352-364)
class TestReadLocationsCsv:
    """Tests for read_locations_csv method."""

    @pytest.mark.unit
    def test_returns_list_of_dicts(self, tmp_path: Path) -> None:
        """read_locations_csv returns list of dicts with correct keys."""
        manager = CacheManager(cache_dir=str(tmp_path))
        (tmp_path / "locations.csv").write_text(LOCATIONS_CSV, encoding="utf-8")

        result = manager.read_locations_csv()

        assert len(result) == 2
        assert result[0]["locationId"] == "..."
```

### Test Cases Required
Based on task test strategy:
1. `test_get_csv_metadata_returns_correct_counts_and_mtime` - both CSVs exist, verify counts and oldest mtime
2. `test_get_csv_metadata_returns_none_when_no_csv_files` - neither CSV exists
3. `test_get_csv_metadata_with_only_locations` - only locations.csv exists
4. `test_get_csv_metadata_with_only_policies` - only policies.csv exists
5. `test_get_cache_age_days_calculates_correctly` - verify day calculation
6. `test_get_cache_age_days_returns_none_when_no_cache` - no metadata
7. `test_is_cache_stale_returns_true_when_exceeds_threshold` - age > 30 days
8. `test_is_cache_stale_returns_true_when_no_cache_exists` - no files
9. `test_is_cache_stale_returns_false_for_fresh_cache` - age < 30 days

### Fixtures Available
- `tmp_path` (pytest built-in) - temporary directory for test isolation
- `LOCATIONS_CSV` (test_cache_manager.py:336-338) - sample locations CSV content
- `POLICIES_CSV` (test_cache_manager.py:340-343) - sample policies CSV content

### Test Utilities
- `from unittest.mock import patch` - for mocking datetime.now() in age tests
- `from freezegun import freeze_time` - NOT in dependencies, use patch instead

---

## Risks and Blockers

### Technical Risks
- [x] **Race condition risk**: File may be deleted between existence check and stat() - mitigate with try/except around stat()
- [x] **Timezone consistency**: Ensure all datetime comparisons use UTC - follow auth.py:50-55 pattern

### Dependency Risks
- None - all dependencies (5.1, 5.2) are complete

### Open Questions
- [ ] **CacheMetadata location**: Task details say "in models (or inline in cache_manager.py)" - recommend placing in `models/cache.py` for consistency with other domain models, then export from `models/__init__.py`
- [ ] **Row count behavior for empty/malformed CSV**: Should count be 0 or should get_csv_metadata return None? Recommend: return 0 count for empty files, None only if file doesn't exist.

---

## Implementation Recommendations

### Suggested Approach
1. **Create CacheMetadata model** in `src/teams_phone/models/cache.py`:
   - Follow existing model patterns from `auth.py`
   - Fields: `last_updated: datetime | None`, `locations_count: int`, `policies_count: int`
   - Export from `models/__init__.py`

2. **Implement get_csv_metadata()** in `cache_manager.py`:
   - Check if locations.csv and/or policies.csv exist
   - Return None if neither exists
   - Get mtime from both files, use the OLDER one (indicates when cache was last complete)
   - Use existing `read_locations_csv()` and `read_policies_csv()` to get row counts

3. **Implement get_cache_age_days()** in `cache_manager.py`:
   - Call `get_csv_metadata()` to get last_updated
   - Return None if metadata is None or last_updated is None
   - Calculate days: `(datetime.now(timezone.utc) - last_updated).days`

4. **Implement is_cache_stale()** in `cache_manager.py`:
   - Call `get_cache_age_days()`
   - Return True if None (no cache) or > CSV_STALE_DAYS (30)
   - Return False otherwise

5. **Add tests** in `tests/unit/test_cache_manager.py`:
   - New test classes: `TestGetCsvMetadata`, `TestGetCacheAgeDays`, `TestIsCacheStale`
   - Use tmp_path fixture for file isolation
   - Mock datetime.now() for deterministic age testing

### Files to Create/Modify
| Action | File | Reason |
|--------|------|--------|
| Create | `src/teams_phone/models/cache.py` | CacheMetadata Pydantic model |
| Modify | `src/teams_phone/models/__init__.py` | Export CacheMetadata |
| Modify | `src/teams_phone/infrastructure/cache_manager.py` | Add get_csv_metadata, get_cache_age_days, is_cache_stale |
| Modify | `tests/unit/test_cache_manager.py` | Add tests for new methods |

### Quick Wins
- Start with `CacheMetadata` model - simple Pydantic class, quick to implement and test
- `is_cache_stale()` is the simplest method - delegates to `get_cache_age_days()`

---

## Cheat Sheet

### Key Imports
```python
# For cache_manager.py
from datetime import datetime, timezone
from teams_phone.constants import CSV_STALE_DAYS
from teams_phone.models import CacheMetadata

# For models/cache.py
from datetime import datetime
from pydantic import BaseModel, ConfigDict
```

### Key Functions to Use
- `path.stat().st_mtime` - Get file modification time as float timestamp
- `datetime.fromtimestamp(ts, tz=timezone.utc)` - Convert timestamp to datetime
- `datetime.now(timezone.utc)` - Get current UTC time
- `(dt1 - dt2).days` - Get days between two datetimes
- `len(self.read_locations_csv())` - Count locations rows (reuse existing method)
- `len(self.read_policies_csv())` - Count policies rows (reuse existing method)

### Common Patterns
- **Graceful degradation**: Return None/0/empty when files don't exist
- **Pydantic model config**: `model_config = ConfigDict(extra="forbid")`
- **Path operations**: `self._cache_dir / "filename.csv"` for path construction
- **File existence check**: `path.exists()` before stat()

### Method Signatures
```python
def get_csv_metadata(self) -> CacheMetadata | None: ...
def get_cache_age_days(self) -> int | None: ...
def is_cache_stale(self) -> bool: ...
```

---

*Generated by task-recon (claude-opus-4-5-20251101) | Ready for implementation*
