# Database Migrations Guide

## When to Use This Guide

**Use when:**
- Adding, modifying, or removing database tables/columns
- Changing indexes or constraints
- Seeding reference data
- Managing schema versions across environments
- Rolling back database changes

## Core Principles

### 1. Migrations Are Code

Database migrations should be:
- **Version controlled** - Committed alongside application code
- **Immutable** - Never modify a migration after it's deployed
- **Idempotent** - Safe to run multiple times (handled by migration tools)
- **Reversible** - Include rollback logic when possible

### 2. Forward-Only in Production

```
Development: Test migrations, iterate freely
Staging: Apply migrations, test rollbacks
Production: Forward-only, rollback is a new migration
```

### 3. Atomic Changes

Each migration should do ONE thing:

```sql
-- BAD: Multiple unrelated changes
-- 001_big_migration.sql
ALTER TABLE users ADD COLUMN email VARCHAR(255);
CREATE TABLE orders (...);
DROP TABLE legacy_data;

-- GOOD: Atomic migrations
-- 001_add_user_email.sql
ALTER TABLE users ADD COLUMN email VARCHAR(255);

-- 002_create_orders_table.sql
CREATE TABLE orders (...);

-- 003_remove_legacy_data.sql
DROP TABLE legacy_data;
```

## Platform-Specific Guides

### Supabase Migrations

**Setup:**

```bash
# Initialize Supabase in your project
supabase init

# Link to your project
supabase link --project-ref your-project-ref
```

**Creating Migrations:**

```bash
# Create new migration
supabase migration new create_users_table

# This creates: supabase/migrations/TIMESTAMP_create_users_table.sql
```

**Migration File Structure:**

```sql
-- supabase/migrations/20250115000000_create_users_table.sql

-- Create users table
CREATE TABLE users (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    email TEXT UNIQUE NOT NULL,
    name TEXT,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW()
);

-- Enable Row Level Security
ALTER TABLE users ENABLE ROW LEVEL SECURITY;

-- Create policies
CREATE POLICY "Users can view own data"
    ON users FOR SELECT
    USING (auth.uid() = id);

CREATE POLICY "Users can update own data"
    ON users FOR UPDATE
    USING (auth.uid() = id);

-- Create indexes
CREATE INDEX idx_users_email ON users(email);

-- Create trigger for updated_at
CREATE TRIGGER set_updated_at
    BEFORE UPDATE ON users
    FOR EACH ROW
    EXECUTE FUNCTION moddatetime(updated_at);
```

**Applying Migrations:**

```bash
# Push migrations to remote database
supabase db push

# Or apply to local development
supabase db reset  # Resets and applies all migrations
```

**Checking Status:**

```bash
# List migrations
supabase migration list

# Check which migrations are applied
supabase db diff  # Shows pending changes
```

### Alembic (Python/SQLAlchemy)

**Setup:**

```bash
# Install
pip install alembic

# Initialize
alembic init alembic
```

**Configuration:**

```python
# alembic/env.py
from myapp.models import Base
from myapp.config import DATABASE_URL

config.set_main_option("sqlalchemy.url", DATABASE_URL)
target_metadata = Base.metadata
```

**Creating Migrations:**

```bash
# Auto-generate from model changes
alembic revision --autogenerate -m "add users table"

# Create empty migration
alembic revision -m "add custom index"
```

**Migration File:**

```python
# alembic/versions/001_add_users_table.py
"""add users table

Revision ID: 001
Revises:
Create Date: 2025-01-15
"""

from alembic import op
import sqlalchemy as sa


revision = '001'
down_revision = None
branch_labels = None
depends_on = None


def upgrade():
    op.create_table(
        'users',
        sa.Column('id', sa.UUID(), primary_key=True),
        sa.Column('email', sa.String(255), nullable=False, unique=True),
        sa.Column('name', sa.String(255)),
        sa.Column('created_at', sa.DateTime(), server_default=sa.func.now()),
    )
    op.create_index('idx_users_email', 'users', ['email'])


def downgrade():
    op.drop_index('idx_users_email')
    op.drop_table('users')
```

**Running Migrations:**

```bash
# Apply all pending migrations
alembic upgrade head

# Apply specific migration
alembic upgrade 001

# Rollback one migration
alembic downgrade -1

# Rollback to specific version
alembic downgrade 001

# Check current version
alembic current

# Show migration history
alembic history
```

### Prisma (TypeScript/Node.js)

**Setup:**

```bash
# Install
npm install prisma --save-dev
npm install @prisma/client

# Initialize
npx prisma init
```

**Schema Definition:**

```prisma
// prisma/schema.prisma
generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

model User {
  id        String   @id @default(uuid())
  email     String   @unique
  name      String?
  posts     Post[]
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt

  @@index([email])
}

model Post {
  id        String   @id @default(uuid())
  title     String
  content   String?
  author    User     @relation(fields: [authorId], references: [id])
  authorId  String
  createdAt DateTime @default(now())

  @@index([authorId])
}
```

**Creating and Applying Migrations:**

```bash
# Create migration from schema changes
npx prisma migrate dev --name add_users_table

# Apply migrations in production
npx prisma migrate deploy

# Reset database (development only!)
npx prisma migrate reset
```

## Migration Best Practices

### Safe Schema Changes

**Adding Columns:**

```sql
-- SAFE: Add nullable column
ALTER TABLE users ADD COLUMN phone VARCHAR(20);

-- SAFE: Add column with default
ALTER TABLE users ADD COLUMN status VARCHAR(20) DEFAULT 'active';

-- DANGEROUS: Add NOT NULL without default
-- This fails if table has existing rows!
ALTER TABLE users ADD COLUMN required_field TEXT NOT NULL;

-- SAFE: Add NOT NULL with default, then remove default
ALTER TABLE users ADD COLUMN required_field TEXT NOT NULL DEFAULT '';
-- Later, in application code, ensure all new rows have a value
-- Then optionally: ALTER TABLE users ALTER COLUMN required_field DROP DEFAULT;
```

**Removing Columns:**

```sql
-- Step 1: Stop using column in application code
-- Step 2: Deploy application changes
-- Step 3: Drop column in migration
ALTER TABLE users DROP COLUMN legacy_field;
```

**Renaming Columns:**

```sql
-- DANGEROUS: Direct rename breaks running application
ALTER TABLE users RENAME COLUMN old_name TO new_name;

-- SAFE: Multi-step process
-- Migration 1: Add new column
ALTER TABLE users ADD COLUMN new_name TEXT;
UPDATE users SET new_name = old_name;

-- Deploy: Application writes to both, reads from new
-- Migration 2: Make new column NOT NULL (if needed)
-- Migration 3: Drop old column
ALTER TABLE users DROP COLUMN old_name;
```

### Index Management

```sql
-- SAFE: Create index concurrently (PostgreSQL)
-- Doesn't lock the table
CREATE INDEX CONCURRENTLY idx_users_email ON users(email);

-- DANGEROUS: Regular index creation locks table
CREATE INDEX idx_users_email ON users(email);
```

### Data Migrations

```sql
-- Migration with data transformation
-- 001_split_name_field.sql

-- Add new columns
ALTER TABLE users ADD COLUMN first_name TEXT;
ALTER TABLE users ADD COLUMN last_name TEXT;

-- Migrate data
UPDATE users SET
    first_name = split_part(name, ' ', 1),
    last_name = split_part(name, ' ', 2)
WHERE name IS NOT NULL;

-- Note: Don't drop 'name' column yet!
-- Wait for application to stop using it
```

## CI/CD Integration

### GitHub Actions Workflow

```yaml
# .github/workflows/migrate.yml
name: Database Migrations

on:
  push:
    branches: [main]
    paths:
      - 'supabase/migrations/**'
      - 'prisma/migrations/**'

jobs:
  migrate-staging:
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - uses: actions/checkout@v4

      - name: Setup Supabase CLI
        uses: supabase/setup-cli@v1

      - name: Apply Migrations to Staging
        run: |
          supabase link --project-ref ${{ secrets.SUPABASE_PROJECT_REF }}
          supabase db push
        env:
          SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}

  migrate-production:
    needs: migrate-staging
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4

      - name: Setup Supabase CLI
        uses: supabase/setup-cli@v1

      - name: Apply Migrations to Production
        run: |
          supabase link --project-ref ${{ secrets.SUPABASE_PROJECT_REF }}
          supabase db push
        env:
          SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
```

### Pre-Deployment Checks

```python
"""Pre-deployment migration checks."""

import subprocess
import sys


def check_pending_migrations() -> bool:
    """Check if there are pending migrations."""
    result = subprocess.run(
        ["alembic", "current"],
        capture_output=True,
        text=True
    )
    current = result.stdout.strip()

    result = subprocess.run(
        ["alembic", "heads"],
        capture_output=True,
        text=True
    )
    head = result.stdout.strip()

    return current != head


def main():
    """Verify migrations before deployment."""
    if check_pending_migrations():
        print("ERROR: Pending migrations detected!")
        print("Run migrations before deploying.")
        sys.exit(1)

    print("All migrations applied.")


if __name__ == "__main__":
    main()
```

## Rollback Strategies

### When Rollback is Needed

- Migration introduced a bug
- Performance issue discovered
- Data corruption detected

### Rollback Options

**Option 1: Reverse Migration (Preferred for simple changes)**

```bash
# Alembic
alembic downgrade -1

# Supabase: Create new migration that reverses changes
supabase migration new revert_bad_migration
```

**Option 2: Point-in-Time Recovery (For data issues)**

```bash
# Restore from backup to specific point
# This is platform-specific (RDS, Supabase, etc.)
```

**Option 3: Forward Fix (For complex situations)**

Create a new migration that fixes the issue rather than rolling back.

```sql
-- 003_fix_bad_migration.sql
-- Instead of rolling back 002, fix the issue

-- Restore accidentally deleted data from backup
INSERT INTO users (id, email, name)
SELECT id, email, name FROM users_backup
WHERE NOT EXISTS (SELECT 1 FROM users WHERE users.id = users_backup.id);
```

## Testing Migrations

### Local Testing Workflow

```bash
# 1. Start fresh database
docker-compose up -d db

# 2. Apply all migrations
alembic upgrade head  # or supabase db reset

# 3. Run application tests
pytest tests/

# 4. Test rollback
alembic downgrade -1
alembic upgrade head

# 5. Verify data integrity
python scripts/verify_schema.py
```

### Migration Test Script

```python
"""Test migration rollback safety."""

import subprocess


def test_migration_rollback():
    """Test that migrations can be rolled back safely."""
    # Apply all migrations
    result = subprocess.run(["alembic", "upgrade", "head"], check=True)

    # Get current revision
    result = subprocess.run(
        ["alembic", "current"],
        capture_output=True,
        text=True
    )
    head = result.stdout.strip()

    # Rollback one migration
    subprocess.run(["alembic", "downgrade", "-1"], check=True)

    # Re-apply
    subprocess.run(["alembic", "upgrade", "head"], check=True)

    # Verify we're back at head
    result = subprocess.run(
        ["alembic", "current"],
        capture_output=True,
        text=True
    )
    assert result.stdout.strip() == head


if __name__ == "__main__":
    test_migration_rollback()
    print("Migration rollback test passed!")
```

## Related Documentation

- **Supabase**: [supabase.md](./supabase.md) - Supabase-specific patterns
- **Environment Isolation**: [environment-isolation.md](./environment-isolation.md) - Per-env databases
- **Secrets Management**: [secrets-management.md](./secrets-management.md) - Database credentials
- **Configuration as Code**: [configuration-as-code.md](./configuration-as-code.md) - Schema as code

---

**Key Takeaway**: Database migrations are code—version control them, review them, test them. Use atomic migrations that do one thing each. Always test rollbacks in staging before production. Never modify a deployed migration; create a new one instead.
