# Playbook: Managing Infrastructure as Code

## Overview

A comprehensive workflow for managing cloud infrastructure using Pulumi, from planning through production deployment, with integrated cost controls and financial governance.

## Prerequisites

- Pulumi CLI installed
- Cloud provider credentials configured
- Virtual environment set up
- Familiarity with target cloud platform (Vercel, Netlify, Cloudflare, Supabase, AWS, GCP, or Azure)

## Steps

### 1. Plan Infrastructure

**Goal**: Define what infrastructure you need and estimate costs before deployment

**Action**:
- Document requirements clearly
- Identify all required resources
- Estimate costs for dev/staging/production
- Get approval for estimated costs

**Cost Estimation Template**:
```
Infrastructure Requirements:

Development:
- Supabase Free tier
- Vercel Hobby tier
- Cloudflare Free tier
- Estimated cost: $0/month

Production:
- Supabase Pro ($25/month)
- Vercel Pro ($20/month base + $30 overage est.)
- Cloudflare Workers Paid ($5/month base + $10 overage est.)
- Cloudflare R2 ($5/month est.)
- Estimated cost: ~$95/month

Approval: [  ] Approved by: _______
```

**AI Prompt**:
```
I need to deploy [describe application] to production.

Requirements:
- [List features and constraints]

Tech stack:
- [Languages, frameworks, databases]

Help me:
1. Identify required infrastructure components
2. Estimate monthly costs for dev and production
3. Suggest cost optimization strategies
```

### 2. Set Up Project Structure

**Goal**: Organize infrastructure code following best practices

**Action**: Create well-structured Pulumi project

**Directory Structure**:
```
infrastructure/
├── Pulumi.yaml                 # Project config
├── Pulumi.dev.yaml             # Dev stack config
├── Pulumi.staging.yaml         # Staging stack config
├── Pulumi.production.yaml      # Production stack config
├── __main__.py                 # Entry point
├── components/                 # Reusable components
│   ├── __init__.py
│   ├── database.py
│   ├── web_service.py
│   └── storage.py
├── policies/                   # Cost governance policies
│   ├── __init__.py
│   └── cost_controls.py
├── scripts/                    # Helper scripts
│   ├── check_cost_threshold.py
│   └── check_usage.py
├── tests/                      # Infrastructure tests
│   ├── test_components.py
│   └── test_cost_estimates.py
├── requirements.txt            # Python dependencies
└── .gitignore                  # Ignore secrets and state
```

**Commands**:
```bash
# Create project
mkdir infrastructure && cd infrastructure
pulumi new python --name my-infrastructure

# Set up virtual environment
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Create stacks
pulumi stack init dev
pulumi stack init staging
pulumi stack init production
```

**Tips**:
- Keep infrastructure code separate from application code
- Use components for reusable patterns
- Version control everything except secrets

### 3. Configure Cost Controls

**Goal**: Prevent unexpected spending before any deployment

**Action**: Set up cost estimation and budget policies

**Install Infracost**:
```bash
brew install infracost  # macOS
# Or: curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh
```

**Configure pre-deployment cost check**:
```yaml
# Pulumi.yaml
scripts:
  pre-up:
    - infracost breakdown --path . --format json --out-file infracost.json
    - python scripts/check_cost_threshold.py
```

**Set budget threshold**:
```bash
pulumi config set monthlyBudget 100  # $100/month limit for this stack
```

For the cost threshold script implementation, see `.claude/knowledge/infrastructure/cost-tracking.md`

### 4. Write Infrastructure Code

**Goal**: Define infrastructure resources with cost awareness

**Action**: Implement infrastructure with environment-specific sizing

**Example __main__.py**:
```python
import pulumi
from dataclasses import dataclass

@dataclass
class AppConfig:
    stack: str
    monthly_budget: int
    cost_center: str

    @classmethod
    def from_pulumi_config(cls) -> "AppConfig":
        config = pulumi.Config()
        return cls(
            stack=pulumi.get_stack(),
            monthly_budget=config.require_int("monthlyBudget"),
            cost_center=config.get("costCenter") or "engineering",
        )

# Load config
app_config = AppConfig.from_pulumi_config()

# Environment-specific sizing
is_prod = app_config.stack == "production"

# Create infrastructure
# (See provider-specific guides for examples)

# Export outputs
pulumi.export("environment", app_config.stack)
pulumi.export("monthly_budget", app_config.monthly_budget)
```

**Tips**:
- Use stack name to determine environment
- Size resources appropriately (dev < staging < production)
- Always tag resources for cost tracking
- Document cost estimates in code comments

### 5. Test Infrastructure

**Goal**: Validate infrastructure code before deployment

**Action**: Write and run tests

**Example tests/test_components.py**:
```python
import pytest
import pulumi

@pulumi.runtime.test
def test_dev_uses_cheap_instances():
    """Ensure dev environment uses cost-effective resources."""
    import os
    os.environ["PULUMI_STACK"] = "dev"

    from components.database import create_database

    db = create_database(name="test-db")

    # Dev should use small instances
    assert db.instance_class.startswith("db.t3.micro")

@pulumi.runtime.test
def test_production_has_backups():
    """Ensure production has adequate backup retention."""
    import os
    os.environ["PULUMI_STACK"] = "production"

    from components.database import create_database

    db = create_database(name="test-db")

    # Production should have 30-day backups
    assert db.backup_retention_days >= 30
```

**Run tests**:
```bash
pytest tests/ -v
```

### 6. Preview Changes

**Goal**: Review what will be deployed before committing

**Action**: Use `pulumi preview` to see planned changes

**Commands**:
```bash
# Select stack
pulumi stack select dev

# Preview changes
pulumi preview

# Review output
# - Which resources will be created/updated/deleted
# - Estimated costs (if Infracost configured)
# - Any policy violations
```

**Checklist**:
- [ ] No unexpected resources
- [ ] Cost estimate within budget
- [ ] All required tags present
- [ ] No secrets exposed in outputs
- [ ] Policies passing

### 7. Deploy Infrastructure

**Goal**: Deploy infrastructure to cloud provider

**Action**: Execute `pulumi up` with approval

**Commands**:
```bash
# Deploy to dev
pulumi stack select dev
pulumi up --yes  # Auto-approve for dev

# Deploy to staging (manual approval)
pulumi stack select staging
pulumi up  # Review and approve

# Deploy to production (manual approval + confirmation)
pulumi stack select production
pulumi up  # Review, double-check, approve
```

**Deployment Checklist**:
- [ ] Cost check passed
- [ ] Tests passed
- [ ] Preview reviewed
- [ ] Stakeholders notified (production only)
- [ ] Monitoring configured
- [ ] Budget alerts set up

### 8. Configure Monitoring and Alerts

**Goal**: Monitor costs and usage after deployment

**Action**: Set up budget alerts and usage monitoring

**AWS Example**:
```python
import pulumi_aws as aws

# Create billing alarm
billing_alarm = aws.cloudwatch.MetricAlarm(
    "billing-alarm",
    alarm_description=f"Alert when monthly cost exceeds ${app_config.monthly_budget * 0.8}",
    comparison_operator="GreaterThanThreshold",
    evaluation_periods=1,
    metric_name="EstimatedCharges",
    namespace="AWS/Billing",
    period=21600,  # 6 hours
    statistic="Maximum",
    threshold=app_config.monthly_budget * 0.8,  # Alert at 80%
    alarm_actions=[sns_topic.arn],
)
```

**Set up regular cost reviews**:
```bash
# Add to crontab or CI/CD
0 9 * * * cd infrastructure && python scripts/check_usage.py
```

### 9. Maintain Infrastructure

**Goal**: Keep infrastructure updated and optimized

**Action**: Regular maintenance tasks

**Weekly Tasks**:
- [ ] Review cost reports
- [ ] Check for failed deployments
- [ ] Review logs for errors
- [ ] Update dependencies

**Monthly Tasks**:
- [ ] Review and optimize costs
- [ ] Update Pulumi and provider versions
- [ ] Review and update policies
- [ ] Clean up unused resources

**Quarterly Tasks**:
- [ ] Evaluate new services/features
- [ ] Review architecture for optimization
- [ ] Update disaster recovery procedures
- [ ] Security audit

**Cost Optimization Script**:
```python
# scripts/check_usage.py
import boto3
from datetime import datetime, timedelta

def check_aws_costs():
    ce = boto3.client('ce')

    # Get last 7 days
    end = datetime.now().strftime('%Y-%m-%d')
    start = (datetime.now() - timedelta(days=7)).strftime('%Y-%m-%d')

    response = ce.get_cost_and_usage(
        TimePeriod={'Start': start, 'End': end},
        Granularity='DAILY',
        Metrics=['UnblendedCost'],
        GroupBy=[{'Type': 'TAG', 'Key': 'Environment'}],
    )

    for result in response['ResultsByTime']:
        date = result['TimePeriod']['Start']
        cost = result['Total']['UnblendedCost']['Amount']
        print(f"{date}: ${float(cost):.2f}")

if __name__ == "__main__":
    check_aws_costs()
```

### 10. Handle Issues

**Goal**: Quickly resolve infrastructure problems

**Common Issues**:

**1. Deployment Failures**
```bash
# Check Pulumi logs
pulumi stack select production
pulumi logs

# Rollback if needed
pulumi stack select production
pulumi up --target urn:pulumi:production::my-infrastructure::aws:s3/bucket:Bucket::my-bucket
# Fix the specific resource

# Or destroy and recreate
pulumi destroy --target urn:pulumi:production::my-infrastructure::aws:s3/bucket:Bucket::my-bucket
pulumi up
```

**2. Cost Overruns**
```bash
# Identify expensive resources
aws ce get-cost-and-usage \
  --time-period Start=2025-01-01,End=2025-01-07 \
  --granularity DAILY \
  --metrics UnblendedCost \
  --group-by Type=DIMENSION,Key=SERVICE

# Downsize or remove resources
pulumi destroy --target urn:pulumi:production::my-infrastructure::aws:ec2/instance:Instance::expensive-instance
```

**3. State Conflicts**
```bash
# Refresh state
pulumi refresh

# Export and inspect state
pulumi stack export > state.json

# Import corrected state
pulumi stack import < corrected-state.json
```

## Best Practices

See detailed best practices in:
- **Financial Governance**: `.claude/knowledge/infrastructure/financial-governance.md` - Cost estimation, budget alerts, tagging
- **Pulumi Testing**: `.claude/knowledge/infrastructure/pulumi-testing.md` - Testing infrastructure code
- **Pulumi Cost Controls**: `.claude/knowledge/infrastructure/pulumi-cost-controls.md` - Policy as code, pre-deployment checks
- **Secrets Management**: `.claude/knowledge/infrastructure/secrets-management.md` - Never commit secrets, use encrypted config
- **Provider Guides**: `.claude/knowledge/infrastructure/` - Platform-specific optimization (Vercel, Netlify, Cloudflare, Supabase)

**Key principles:**
- Estimate costs before deploying (use Infracost)
- Set budget alerts at 50%, 80%, 100% of budget
- Tag all resources (Environment, CostCenter, Owner, ManagedBy)
- Right-size for environment (dev < staging < production)
- Never commit secrets (use `pulumi config set --secret`)
- Test in dev/staging before production
- Monitor usage and optimize regularly
