# Task Recon: Implement Debug Logging

**Task ID:** 29
**Status:** pending
**Generated:** 2026-01-30T14:30:00Z
**Agent:** claude-opus-4-5-20251101

---

## Task Summary

Add comprehensive debug logging with a `--debug-log` flag that enables JSON Lines logging for API calls, timing, and troubleshooting. The implementation should:

1. Add `--debug-log <path>` global option to the CLI callback in `main.py`
2. Configure Python's logging module to write to the specified file when enabled
3. Add logging statements to GraphClient for request/response tracking
4. Implement secret masking to prevent tokens, certificates, and credentials from appearing in logs
5. Include timing information for API calls

### Dependencies
- Depends on: Task 7 (GraphClient with Retry Logic) - **DONE**
- Depends on: Task 11 (Root CLI App with Global Options) - **DONE**
- Depends on: Task 41 (Validation Checkpoint 3) - **DONE**

### Acceptance Criteria
From task details and CLI specification:
- `--debug-log <path>` option writes JSON Lines to specified file
- Each log entry is self-contained JSON with timestamp, correlation_id, level, event
- API calls logged with method, URL, timing, status
- Secrets NEVER appear in logs (access tokens, client secrets, certificate contents)
- UPNs are masked/truncated in logs (e.g., `j***@contoso.com`)

---

## Insights from Previous Tasks

### Dependency Task Notes

| Task ID | Title | Key Insight |
|---------|-------|-------------|
| 7 | Implement GraphClient with Retry Logic | GraphClient already has `logging.getLogger(__name__)` and uses `logger.warning()` for retry attempts. Logging infrastructure exists but is minimal. |
| 11 | Implement Root CLI App with Global Options | CLIContext dataclass stores global options; new options should be added to CLIContext and initialized in the main callback. OutputFormatter is eagerly initialized, other services are lazy. |
| 41 | Validation Checkpoint 3 | Fixed Windows Unicode encoding issues with ASCII fallback - same pattern may be needed for debug log file encoding. |

### Reusable Patterns from Dependencies

- **Global option pattern**: See `src/teams_phone/cli/main.py:30-76` for how `--json`, `--verbose`, `--tenant`, `--no-color` are defined and stored in CLIContext
- **Lazy initialization pattern**: See `src/teams_phone/cli/context.py:41-73` for `get_output_formatter()` and `get_config_manager()` lazy init
- **Logging setup**: See `src/teams_phone/infrastructure/graph_client.py:9,34` for existing logging import and logger creation

### Warnings and Gotchas

- **Token acquisition is sync**: `auth_service.get_token()` is synchronous - be careful not to log the token itself
- **Windows encoding**: Task 41 discovered Windows cp1252 encoding issues - debug log file should use UTF-8 explicitly
- **Exit code 2 for help**: When testing with no subcommand after global options, expect exit code 2 (Typer behavior)

### Open Items Inherited

- None from dependency tasks

---

## Architecture Context

### Relevant Design Decisions

From `.taskmaster/docs/cli-specification/logging-diagnostics.md`:
- JSON Lines format (newline-delimited JSON)
- Required fields: `ts`, `correlation_id`, `level`, `event`
- Event types: `command_start`, `command_complete`, `graph_request`, `graph_response`, `cache_hit`, `cache_miss`, `retry`, `error`
- Security: Never log tokens, secrets, or full UPNs

### Module Structure

The implementation spans two layers:
1. **CLI Layer** (`src/teams_phone/cli/main.py`): Add `--debug-log` option to callback
2. **Infrastructure Layer** (`src/teams_phone/infrastructure/`): Add logging to GraphClient

### Conventions to Follow

- Use `from __future__ import annotations` at file top
- Use `pathlib.Path` for file paths (see OutputFormatter pattern)
- Use dataclass for structured data (CLIContext pattern)
- Strict mypy type hints required
- Docstrings in Google style with Args/Returns/Raises sections

---

## Related Code References

### Primary References (Must Read)

| File | Lines | Purpose |
|------|-------|---------|
| `src/teams_phone/cli/main.py` | 30-76 | Main callback with global options - where to add `--debug-log` |
| `src/teams_phone/cli/context.py` | 19-57 | CLIContext dataclass - add debug_log_path field |
| `src/teams_phone/infrastructure/graph_client.py` | 107-124, 293-378 | Existing logging in GraphClient - enhance with request/response logging |

### Secondary References (Helpful Context)

| File | Lines | Purpose |
|------|-------|---------|
| `src/teams_phone/constants.py` | 1-97 | Constants module - may need new constants for log formats |
| `src/teams_phone/services/auth_service.py` | 30-39 | Where secrets are defined - avoid logging these |
| `src/teams_phone/infrastructure/output_formatter.py` | 23-39 | Unicode detection pattern for Windows compatibility |
| `.taskmaster/docs/cli-specification/logging-diagnostics.md` | 1-32 | JSON Lines format specification |

### Code Patterns to Follow

```python
# Global option pattern from main.py:45-55
tenant: Optional[str] = typer.Option(
    None,
    "--tenant",
    "-t",
    help="Override default tenant for this command.",
),

# CLIContext field pattern from context.py:34-39
@dataclass
class CLIContext:
    json_output: bool = False
    verbose: bool = False
    tenant: str | None = None
    no_color: bool = False
    # Add: debug_log_path: Path | None = None
```

```python
# Existing logging in graph_client.py:34,117-123
logger = logging.getLogger(__name__)

def _log_retry(retry_state: RetryCallState) -> None:
    """Log retry attempts before sleeping."""
    # ... logs warning for retry attempts
    logger.warning(
        "Retrying request after HTTP %d (attempt %d/%d, waiting %.1fs)",
        response.status_code,
        retry_state.attempt_number,
        MAX_RETRIES,
        wait_time,
    )
```

---

## Dependencies

### Internal Modules
- `teams_phone.cli.context` - CLIContext dataclass to extend
- `teams_phone.cli.main` - Main callback to modify
- `teams_phone.infrastructure.graph_client` - GraphClient to add logging
- `teams_phone.constants` - May need new logging-related constants

### External Packages
- `logging` (stdlib) - Python's logging module, already imported in graph_client.py
- `json` (stdlib) - For JSON Lines output
- `datetime` (stdlib) - For ISO 8601 timestamps
- `uuid` (stdlib) - For correlation IDs
- `pathlib` (stdlib) - For Path handling (already used throughout)
- `typing` (stdlib) - For type annotations (already used)

### New Dependencies Needed
- None required - all needed packages are in stdlib or already dependencies

---

## Testing Guidance

### Test File Location
`tests/unit/test_cli_main.py` - Extend existing tests for new global option

Consider creating: `tests/unit/test_debug_logging.py` - Dedicated tests for logging functionality

### Test Patterns in Use

```python
# From tests/unit/test_cli_main.py:78-105
@pytest.fixture
def test_app_with_command(self) -> tuple[typer.Typer, dict[str, object]]:
    """Create a copy of the app with a test command to inspect context."""
    from teams_phone.cli.main import main

    test_app = typer.Typer(...)
    test_app.callback()(main)

    captured: dict[str, object] = {}

    @test_app.command()
    def test_cmd(ctx: typer.Context) -> None:
        cli_ctx = get_context(ctx)
        captured["debug_log"] = cli_ctx.debug_log_path  # Add this
        # ...
    return test_app, captured
```

```python
# From tests/unit/test_graph_client.py:29-34
@pytest.fixture
def mock_auth_service(tmp_path: Path) -> MagicMock:
    """Create a mock AuthService that returns a test token."""
    mock_service = MagicMock(spec=AuthService)
    mock_service.get_token.return_value = "test-access-token-12345"
    return mock_service
```

### Fixtures Available
- `tmp_path` (pytest) - Temporary directory for log file tests
- `mock_auth_service` - Mock AuthService in test_graph_client.py
- `CliRunner` (typer.testing) - For CLI invocation tests

### Test Utilities
- `tests/conftest.py` - Minimal, defines custom markers (unit, integration, contract)
- `respx` - HTTP mocking for async requests (already used in graph_client tests)
- `unittest.mock.patch` - For environment variable and logging tests

---

## Risks and Blockers

### Technical Risks

- [ ] **Secret exposure in logs**: Must carefully mask access tokens in Authorization header, client secrets, certificate contents. Mitigation: Create a dedicated masking function, unit test with known secrets.

- [ ] **Performance impact**: Logging every request could impact performance for bulk operations. Mitigation: Only configure logging when `--debug-log` is specified; use efficient JSON serialization.

- [ ] **File encoding on Windows**: Windows may default to cp1252 encoding. Mitigation: Explicitly open log file with `encoding='utf-8'`.

### Dependency Risks

- [ ] None - all dependencies are complete (Tasks 7, 11, 41)

### Open Questions

- [ ] Should correlation_id be generated once per CLI invocation or once per request? (Spec says "per command invocation")
- [ ] Should existing `logger.warning()` calls in GraphClient be converted to the new JSON format, or kept as-is for stderr?
- [ ] Should verbose mode (`--verbose`) also enable console logging in addition to debug log file?

---

## Implementation Recommendations

### Suggested Approach

1. **Add debug_log_path to CLIContext** (`context.py`)
   - Add `debug_log_path: Path | None = None` field
   - No lazy initialization needed - just store the path

2. **Add --debug-log option to main callback** (`main.py`)
   - Add `debug_log: Optional[Path] = typer.Option(None, "--debug-log", ...)` parameter
   - Store in CLIContext
   - Configure logging when specified

3. **Create debug logging module** (`src/teams_phone/infrastructure/debug_logger.py`)
   - Create `DebugLogger` class that writes JSON Lines
   - Implement secret masking functions
   - Implement UPN truncation function
   - Generate correlation_id per session

4. **Integrate with GraphClient** (`graph_client.py`)
   - Add optional DebugLogger parameter
   - Log request method, URL (sanitized), timing
   - Log response status, duration_ms
   - Log retry attempts

5. **Write comprehensive tests**
   - Test log file creation
   - Test JSON Lines format
   - Test secret masking (critical)
   - Test timing information

### Files to Create/Modify

| Action | File | Reason |
|--------|------|--------|
| Modify | `src/teams_phone/cli/context.py` | Add debug_log_path field to CLIContext |
| Modify | `src/teams_phone/cli/main.py` | Add --debug-log option, configure logging |
| Create | `src/teams_phone/infrastructure/debug_logger.py` | New module for JSON Lines logging with secret masking |
| Modify | `src/teams_phone/infrastructure/__init__.py` | Export DebugLogger |
| Modify | `src/teams_phone/infrastructure/graph_client.py` | Add timing and request/response logging |
| Create | `tests/unit/test_debug_logging.py` | Tests for debug logging functionality |
| Modify | `tests/unit/test_cli_main.py` | Tests for --debug-log option |

### Quick Wins

- Start by adding the `--debug-log` option and CLIContext field (simple, validates approach)
- Create the secret masking function early (critical for security, testable in isolation)
- Use `time.perf_counter()` for high-resolution timing

---

## Cheat Sheet

### Key Imports

```python
# For main.py
from pathlib import Path
from typing import Optional
import logging
import uuid

# For debug_logger.py (new file)
import json
import logging
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
import uuid

# For graph_client.py additions
import time
```

### Key Functions to Use

- `logging.basicConfig(filename=..., level=logging.DEBUG, format=...)` - Basic setup
- `logging.getLogger(__name__)` - Get module-specific logger (already used)
- `time.perf_counter()` - High-resolution timing for duration_ms
- `uuid.uuid4()` - Generate correlation_id
- `datetime.now(timezone.utc).isoformat()` - ISO 8601 timestamp
- `json.dumps(log_entry)` - Serialize log entry to JSON

### Common Patterns

- **Secret masking**: `re.sub(r'Bearer [A-Za-z0-9._-]+', 'Bearer [REDACTED]', header_str)`
- **UPN truncation**: `f"{upn[0]}***@{upn.split('@')[1]}"` (e.g., `j***@contoso.com`)
- **JSON Lines**: `f.write(json.dumps(entry) + '\n')` (each line is valid JSON)
- **UTF-8 file**: `open(path, 'a', encoding='utf-8')`

### Secrets to NEVER Log

From codebase analysis:
- Access tokens (`Bearer ...` in Authorization header)
- Client secrets (environment variable `TEAMS_PHONE_CLIENT_SECRET`)
- Certificate contents (PEM data from `cert_path`)
- Certificate thumbprints (could aid attacks)
- Full tokens from `auth_service.get_token()` return value

---

*Generated by task-recon (claude-opus-4-5-20251101) | Ready for implementation*
