# Task Recon: Create contract tests workflow (.github/workflows/contract-tests.yml)

**Task ID:** 37.3
**Status:** pending
**Generated:** 2026-01-30T19:30:00Z
**Agent:** claude-opus-4-5-20251101

---

## Task Summary

Configure a scheduled daily GitHub Actions workflow that runs contract tests against the live Microsoft Graph API, with failure notifications and artifact handling for debugging.

### Parent Task Context

**Parent Task:** Set Up CI/CD Pipeline (ID: 37)
**Parent Goal:** Create GitHub Actions workflows for CI (lint, test), releases, and daily contract tests to ensure code quality and detect API contract drift.
**This Subtask's Role:** Provides the final piece of the CI/CD pipeline by implementing scheduled contract tests that validate Pydantic models against live Graph API responses. This catches API contract drift early, before it breaks production code.

### Dependencies
- Depends on: 37.1 (CI workflow - done)

### Acceptance Criteria
From task details:
1. Schedule: cron '0 6 * * *' (daily at 6 AM UTC) + workflow_dispatch for manual triggering
2. Contract test job: checkout, uv setup, credentials from secrets, run `pytest -m contract -v --tb=long`
3. Failure notification: Create GitHub issue on failure using github-script
4. Artifact handling: Upload test results/logs as artifacts on failure, retain for 7 days
5. Security: Secrets for AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_CERTIFICATE (base64-encoded)

---

## Insights from Previous Tasks

### Dependency Task Notes
| Task ID | Title | Key Insight |
|---------|-------|-------------|
| 37.1 | Create main CI workflow | Used actions/checkout@v5 and astral-sh/setup-uv@v7 (latest stable versions) |
| 37.1 | Create main CI workflow | Python versions must be quoted strings in YAML matrix ("3.10" not 3.10) |
| 37.2 | Create release workflow | Consistent action versions: checkout@v5, setup-uv@v7 across workflows |

### Reusable Patterns from Dependencies
- **UV setup with caching**: Implemented in `.github/workflows/ci.yml:15-18` with `enable-cache: true`
- **Checkout pattern**: `actions/checkout@v5` used consistently in both ci.yml and release.yml
- **Test execution**: `uv run pytest -m unit --cov=teams_phone` pattern from ci.yml:65

### Warnings and Gotchas
- **From Task 37.1**: Python versions must be quoted strings in YAML matrix
- **From Task 37.1**: asyncio_mode = "strict" in pytest config requires explicit `@pytest.mark.asyncio`
- **Contract tests use environment variables**: TEST_TENANT_ID, TEST_CLIENT_ID, TEST_CERT_PATH (see conftest.py)

### Open Items Inherited
- [ ] CODECOV_TOKEN secret must be configured in GitHub repository settings (from 37.1)
- [ ] PyPI trusted publisher must be configured before first release (from 37.2)

---

## Architecture Context

### Relevant Design Decisions
- Contract tests use `extra='forbid'` on Pydantic models to detect API contract drift
- Contract tests require live Graph API credentials (certificate-based auth)
- Tests are marked with `@pytest.mark.contract` for selective execution
- Environment variables (TEST_TENANT_ID, TEST_CLIENT_ID, TEST_CERT_PATH) configure contract test credentials

### Module Structure
```
.github/workflows/
├── ci.yml              # Lint, typecheck, unit tests (on push/PR)
├── release.yml         # Tag-triggered PyPI publishing
└── contract-tests.yml  # NEW: Daily scheduled contract tests
```

### Conventions to Follow
- Use consistent action versions: actions/checkout@v5, astral-sh/setup-uv@v7
- Use `enable-cache: true` for uv setup
- Follow existing workflow file structure from ci.yml and release.yml
- Document prerequisites in YAML comments (as done in release.yml:1-16)

---

## Related Code References

### Primary References (Must Read)
| File | Lines | Purpose |
|------|-------|---------|
| `.github/workflows/ci.yml` | 1-73 | Reference for workflow structure, uv setup, pytest execution |
| `tests/contract/conftest.py` | 36-48 | How contract tests read environment variable credentials |
| `tests/contract/conftest.py` | 55-91 | TestConfigManager that bypasses config.toml for env-based auth |
| `tests/contract/conftest.py` | 158-184 | live_graph_client fixture (primary entry point for contract tests) |

### Secondary References (Helpful Context)
| File | Lines | Purpose |
|------|-------|---------|
| `tests/contract/test_user_contracts.py` | 36-65 | Example contract test structure |
| `tests/contract/test_number_contracts.py` | 36-69 | Another contract test example |
| `.github/workflows/release.yml` | 1-16 | Documentation comment pattern for prerequisites |
| `pyproject.toml` | 54-61 | pytest markers and asyncio configuration |

### Code Patterns to Follow

```yaml
# Example workflow structure from .github/workflows/ci.yml:10-27
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5

      - name: Install uv
        uses: astral-sh/setup-uv@v7
        with:
          enable-cache: true

      - name: Install dependencies
        run: uv sync
```

```python
# Contract test credential pattern from tests/contract/conftest.py:36-48
TEST_TENANT_ID = os.environ.get("TEST_TENANT_ID")
TEST_CLIENT_ID = os.environ.get("TEST_CLIENT_ID")
TEST_CERT_PATH = os.environ.get("TEST_CERT_PATH")

# Check if all required credentials are present
CREDENTIALS_CONFIGURED = bool(TEST_TENANT_ID and TEST_CLIENT_ID and TEST_CERT_PATH)
```

---

## Dependencies

### Internal Modules
- `tests/contract/conftest.py` - Contract test fixtures using environment variables
- `tests/contract/test_*.py` - Contract test files marked with `@pytest.mark.contract`

### External Packages
- `pytest>=7.4.0` - Test framework
- `pytest-asyncio>=0.21.0` - Async test support
- `actions/checkout@v5` - GitHub Action for checkout
- `astral-sh/setup-uv@v7` - GitHub Action for uv setup
- `actions/upload-artifact@v4` - GitHub Action for artifact upload
- `actions/github-script@v7` - GitHub Action for creating issues on failure

### New Dependencies Needed
- None required - all needed GitHub Actions are standard/available

---

## Testing Guidance

### Test File Location
`tests/contract/` directory - all files with `test_*.py` pattern

### Test Patterns in Use
```python
# Example from tests/contract/test_user_contracts.py:36-65
@pytest.mark.contract
@pytest.mark.asyncio
async def test_user_configuration_schema(live_graph_client: GraphClient) -> None:
    """Validate UserConfiguration model against live /admin/teams/userConfigurations endpoint."""
    data = await live_graph_client.get("/admin/teams/userConfigurations?$top=5")
    users = data.get("value", [])

    if not users:
        pytest.skip("No user configurations available in test tenant")

    # Validate each user configuration
    for user_data in users:
        clean_data = _strip_odata_metadata(user_data)
        user = UserConfiguration.model_validate(clean_data)
```

### Fixtures Available
- `live_graph_client` - Fully authenticated GraphClient for live API calls
- `test_tenant_profile` - TenantProfile created from environment variables
- `test_auth_service` - AuthService using test credentials

### Test Utilities
- `tests/contract/conftest.py` - TestConfigManager that bypasses config.toml
- `_strip_odata_metadata()` helper function in test files

---

## Risks and Blockers

### Technical Risks
- [ ] **Certificate handling in CI**: Base64-encoded certificate secret must be decoded and written to temp file. The workflow must clean up the certificate file after tests complete.
- [ ] **pytest.skip() on missing data**: Contract tests may skip if test tenant lacks data (users, phone numbers). This is expected behavior but should be reflected in test results.
- [ ] **Async event loop issues on Ubuntu**: Previous Windows-specific asyncio fixes (cf83a03) may not apply, but async tests should work on Linux.

### Dependency Risks
- [ ] **GitHub secrets not configured**: AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_CERTIFICATE must be set in repository secrets before workflow runs
- [ ] **Test tenant access**: A real Azure tenant with Graph API access and test data is required

### Open Questions
- [ ] Should the workflow create a GitHub issue for EVERY failure, or only on first failure after success?
- [ ] What label should be applied to auto-created issues (e.g., "contract-failure", "api-drift")?
- [ ] Should there be a cooldown period to prevent duplicate issues on repeated failures?

---

## Implementation Recommendations

### Suggested Approach
1. Create `.github/workflows/contract-tests.yml` with schedule and workflow_dispatch triggers
2. Add contract-tests job with checkout, uv setup, and dependency sync
3. Configure environment variables from GitHub secrets (decode base64 certificate to temp file)
4. Run `uv run pytest -m contract -v --tb=long` with test environment variables
5. Add failure notification step using `actions/github-script@v7` with `if: failure()`
6. Add artifact upload step for test results with `if: failure()` condition
7. Add cleanup step to remove temporary certificate file

### Files to Create/Modify
| Action | File | Reason |
|--------|------|--------|
| Create | `.github/workflows/contract-tests.yml` | Main workflow file |

### Quick Wins
- Copy structure from ci.yml for checkout and uv setup steps
- Use `if: failure()` condition for notification and artifact steps (standard pattern)

---

## Cheat Sheet

### Key Environment Variables for Contract Tests
```bash
# Required environment variables (from tests/contract/conftest.py)
TEST_TENANT_ID=<azure-tenant-guid>
TEST_CLIENT_ID=<app-registration-client-id>
TEST_CERT_PATH=<path-to-pem-certificate>
```

### Key GitHub Secrets to Configure
```yaml
# Repository secrets needed
AZURE_TENANT_ID: Azure AD tenant GUID
AZURE_CLIENT_ID: App registration client ID
AZURE_CLIENT_CERTIFICATE: Base64-encoded PEM certificate content
```

### Contract Test Execution Command
```bash
# Run all contract tests with verbose output and full tracebacks
uv run pytest -m contract -v --tb=long
```

### Creating Issue on Failure Pattern
```yaml
- name: Create issue on failure
  if: failure()
  uses: actions/github-script@v7
  with:
    script: |
      await github.rest.issues.create({
        owner: context.repo.owner,
        repo: context.repo.repo,
        title: 'Contract Tests Failed',
        body: `Contract tests failed. See [workflow run](${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId})`,
        labels: ['contract-failure']
      })
```

### Artifact Upload Pattern
```yaml
- name: Upload test artifacts
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: contract-test-results
    path: |
      .pytest_cache/
      *.log
    retention-days: 7
```

### Certificate Decoding Pattern
```yaml
- name: Setup certificate
  run: |
    echo "${{ secrets.AZURE_CLIENT_CERTIFICATE }}" | base64 -d > /tmp/cert.pem
    chmod 600 /tmp/cert.pem
  env:
    AZURE_CLIENT_CERTIFICATE: ${{ secrets.AZURE_CLIENT_CERTIFICATE }}
```

---

*Generated by task-recon (claude-opus-4-5-20251101) | Ready for implementation*
