# Final Test Strategy: Unit + Integration, No Recording

## Core Principle
**Unit tests** test OUR code. **Integration tests** test against REAL systems.

## 1. Unit Tests (Our Code Logic)

### What to Test
- Response parsing logic
- Error handling and retry logic  
- Request formatting
- State management
- Business logic

### How to Test
```python
class TestFCPProviderUnit:
    """Test our provider implementation."""
    
    def test_parse_error_response(self):
        """Test we parse error responses correctly."""
        provider = FCPProvider(api_key="test")
        
        # Test OUR parsing logic
        error_response = {
            "error": "QUOTA_EXCEEDED",
            "message": "GPU quota exceeded",
            "details": {"current": 10, "limit": 10}
        }
        
        error = provider._parse_error(error_response)
        assert isinstance(error, QuotaExceededError)
        assert "GPU quota exceeded" in str(error)
    
    def test_instance_type_normalization(self):
        """Test we normalize instance types correctly."""
        provider = FCPProvider(api_key="test")
        
        # Test OUR normalization logic
        assert provider._normalize_instance_type("a100") == "a100-80gb"
        assert provider._normalize_instance_type("A100-80GB") == "a100-80gb"
        assert provider._normalize_instance_type("a100.80gb") == "a100-80gb"
    
    def test_retry_with_backoff(self):
        """Test our exponential backoff implementation."""
        provider = FCPProvider(api_key="test")
        
        # Mock ONLY at HTTP boundary
        with patch.object(provider._http_client, 'post') as mock_post:
            mock_post.side_effect = [
                HTTPError(503),  # Service unavailable
                HTTPError(503),  # Service unavailable
                {"success": True}  # Success
            ]
            
            start = time.time()
            result = provider._post_with_retry("/endpoint", {})
            duration = time.time() - start
            
            # Should have exponential backoff
            assert duration > 1.5  # 0.5 + 1.0 backoff
            assert result["success"] is True
```

### What NOT to Do
- Don't mock the entire provider
- Don't test external API behavior
- Don't use recorded responses

## 2. Integration Tests (Real API)

### What to Test
- Full workflows against real API
- API contract compliance
- Performance characteristics
- Error scenarios with real API

### How to Test
```python
@pytest.mark.integration
class TestFCPIntegration:
    """Test against real FCP API."""
    
    @pytest.fixture(scope="class")
    def provider(self):
        """Real provider for integration tests."""
        if not os.environ.get("FCP_TEST_API_KEY"):
            pytest.skip("Integration tests require FCP_TEST_API_KEY")
        return create_test_provider()
    
    def test_gpu_allocation_and_release(self, provider):
        """Test real GPU allocation and cleanup."""
        # Allocate real GPU
        config = TaskConfigBuilder().with_gpu("a100-40gb").build()
        task = provider.submit_task(config)
        
        # Verify real allocation
        assert task.instance_id is not None
        instance = provider.get_instance(task.instance_id)
        assert instance.gpu_type == "a100-40gb"
        
        # Real cleanup
        provider.cancel_task(task.task_id)
        
        # Verify release
        with pytest.raises(ResourceNotFoundError):
            provider.get_instance(task.instance_id)
    
    def test_concurrent_submissions(self, provider):
        """Test real concurrent API usage."""
        configs = [
            TaskConfigBuilder().with_name(f"concurrent-{i}").build()
            for i in range(5)
        ]
        
        # Submit concurrently to real API
        with ThreadPoolExecutor(max_workers=5) as executor:
            futures = [executor.submit(provider.submit_task, c) for c in configs]
            tasks = [f.result() for f in futures]
        
        # All should succeed with unique IDs
        task_ids = [t.task_id for t in tasks]
        assert len(set(task_ids)) == 5
```

### Test Environment Setup
```bash
# .env.test
FCP_TEST_API_KEY=your-test-key
FCP_TEST_PROJECT=flow-sdk-tests
FCP_TEST_REGION=us-east-1-sandbox

# Run integration tests
pytest -m integration --env-file .env.test
```

## 3. No VCR/Recording Because:

1. **Stale Data**: Recorded responses become outdated
2. **False Confidence**: Tests pass but real integration is broken
3. **Hidden Changes**: API changes aren't detected
4. **Complexity**: Another layer to maintain

## 4. Contract Tests (API Assumptions)

```python
@pytest.mark.integration
class TestFCPContract:
    """Verify our assumptions about the API."""
    
    def test_instance_type_format(self, provider):
        """Verify instance types follow expected format."""
        instances = provider.list_instance_types()
        
        for instance in instances:
            # Our assumption: type contains GPU model and memory
            assert re.match(r'^[a-z0-9]+-\d+gb$', instance.type)
    
    def test_task_state_machine(self, provider):
        """Verify task states follow expected transitions."""
        config = TaskConfigBuilder().build()
        task = provider.submit_task(config)
        
        states_seen = [task.status]
        for _ in range(30):
            task = provider.get_task(task.task_id)
            if task.status != states_seen[-1]:
                states_seen.append(task.status)
            if task.status in TERMINAL_STATES:
                break
            time.sleep(1)
        
        # Verify valid transitions
        validate_state_transitions(states_seen)
```

## Summary

### Do:
- Unit test YOUR code's logic (with minimal HTTP mocking)
- Integration test against REAL API
- Separate test environments (sandbox/prod)
- Contract tests to verify assumptions

### Don't:
- Mock entire providers
- Record API responses
- Test fake implementations
- Mix unit and integration concerns

This is what Dean, Carmack, and the others would do: Test your code's logic in isolation, then test that it actually works with the real system.