# Engineering Standards

> **Philosophy**: Write code that is a joy to read, trivial to test, and easy to extend.

This document defines organizational engineering standards. These principles apply to all projects and guide every technical decision.

---

## Core Principles

### Simple > Complex
Choose straightforward solutions over clever ones. Code is read far more often than it's written.

```python
# GOOD: Explicit and clear
numbers = [int(x) for x in data.split()]
odd_numbers = [n for n in numbers if n % 2 == 1]
result = sum(odd_numbers)
```

### Explicit > Implicit
Make intentions clear. Use enums, type hints, clear names. No magic, no surprises.

```python
# GOOD: Explicit intent
class ProcessingMode(Enum):
    NORMALIZE = "normalize"
    VALIDATE = "validate"

def process_data(df: pd.DataFrame, mode: ProcessingMode) -> pd.DataFrame:
    match mode:
        case ProcessingMode.NORMALIZE:
            return normalize_columns(df)
```

### DRY (Don't Repeat Yourself)
Every piece of knowledge should have a single, authoritative representation. Extract shared logic immediately.

### Fail Fast
Detect errors early with clear messages. Validate inputs at boundaries.

```python
# GOOD: Explicit validation, clear errors
def process_file(path: Path) -> ProcessedData:
    if not path.exists():
        raise FileNotFoundError(f"File not found: {path}")
    if not path.suffix == '.xlsx':
        raise ValueError(f"Expected .xlsx file, got: {path.suffix}")
    return process_data(load_file(path))
```

### YAGNI (You Aren't Gonna Need It)
Build what's needed now. Add features when actually required, not for imaginary futures.

### File Size Limits
- **Python/Code files**: 800 lines max
- **Markdown documentation**: 600 lines max

If exceeded: extract helpers, create submodules, or split into focused docs.

---

## CRITICAL: Python Requirements

### Virtual Environments - MANDATORY

**ALWAYS use virtual environments. NEVER modify system Python.**

```bash
# Required workflow
python3 -m venv .venv
source .venv/bin/activate  # macOS/Linux
pip install -r requirements.txt

# Verify venv is active
which python  # Must show .venv/bin/python
```

**PROHIBITED:**
```bash
# NEVER DO THIS
pip install click                    # No venv
python3 -m pip install --user rich   # --user flag
sudo pip install anything            # sudo
```

**AI Assistant Checklist:**
- [ ] Before `pip install`: Verify `which python` shows `.venv/bin/python`
- [ ] NEVER use: `pip install` (without venv), `--user`, `sudo pip`
- [ ] If no venv exists: Create it first, then activate, then install

### Type Hints - REQUIRED

All functions MUST have type hints.

```python
from typing import Protocol
from pathlib import Path
from dataclasses import dataclass
from datetime import date
import pandas as pd

def process_data(
    data: pd.DataFrame,
    threshold: float = 0.8
) -> tuple[pd.DataFrame, list[str]]:
    """Process dataframe and return (result, errors)."""
    ...

@dataclass
class User:
    id: str
    name: str
    email: str
    created_at: date
    is_active: bool = True
```

### Error Handling - Explicit Only

**Only catch exceptions you expect and can handle.**

```python
# GOOD: Specific, expected exceptions
def parse_file(path: Path) -> pd.DataFrame:
    if not path.exists():
        raise FileNotFoundError(f"File not found: {path}")

    try:
        df = pd.read_excel(path)  # BadZipFile expected for corrupt files
    except BadZipFile as e:
        raise ValueError(f"Corrupt Excel file: {path}") from e

    return df

# BAD: Generic exception catching
try:
    return process_dataframe(df)
except Exception:  # Don't do this - hides bugs
    return None
```

### Logging - Structured Only

**Always use structured logging. NEVER use `print()`.**

```python
import structlog
logger = structlog.get_logger()

def process_batch(batch_id: str, items: list[Item]) -> BatchResult:
    logger.info("batch_processing_started", batch_id=batch_id, item_count=len(items))
    # ... processing ...
    logger.info("batch_processing_completed", batch_id=batch_id, success_count=len(results))
    return BatchResult(results, errors)
```

---

## Code Style

- **Line length**: 100 characters max
- **Formatter**: Ruff
- **Type checker**: ty (Astral's type checker)
- **Docstrings**: Google-style for public functions/classes

### Naming Conventions

```python
MAX_RETRY_COUNT = 3                    # Constants: UPPER_SNAKE_CASE
def calculate_total(items): ...        # Functions: snake_case
class DataProcessor: ...               # Classes: PascalCase
def _internal_helper(self): ...        # Private: leading underscore
```

---

## Architecture Patterns

### Separation of Concerns
Each module/class/function has ONE clear responsibility.

```
src/myproject/
├── parsers/        # Read and parse files
├── validators/     # Validate data quality
├── processors/     # Transform data
├── models/         # Data models (Pydantic)
├── services/       # Business logic orchestration
└── utils/          # Shared utilities
```

### Dependency Injection
Don't create dependencies inside; inject from outside.

```python
# GOOD: Dependencies injected
class ProcessingService:
    def __init__(self, parser: DataParser, validator: DataValidator):
        self.parser = parser
        self.validator = validator
```

### Composition Over Inheritance
Build complex behavior by combining simple objects, not deep inheritance hierarchies.

### Immutability Where Possible
```python
@dataclass(frozen=True)  # Immutable
class User:
    id: str
    name: str

def normalize_name(name: str) -> str:  # Pure function
    return name.strip().upper()
```

---

## Documentation Requirements

Documentation includes code comments, docstrings, README files—**and commit messages**.

### Docstrings (Google-style)

```python
def process_file(path: Path, threshold: float = 0.8) -> tuple[pd.DataFrame, list[str]]:
    """Process file and return results with validation errors.

    Args:
        path: Path to file to process
        threshold: Minimum quality threshold (0.0 to 1.0)

    Returns:
        Tuple of (processed_dataframe, error_messages)

    Raises:
        FileNotFoundError: If file doesn't exist
        ValueError: If threshold not in valid range
    """
    ...
```

### Commit Messages as Documentation

Commit messages explain *why* changes were made. Discoverable via `git log`, `git blame`, `git bisect`.

```bash
# BAD: Only describes what
git commit -m "fix: change timeout to 30"

# GOOD: Explains why
git commit -m "fix: increase API timeout to prevent false failures

Default 10s timeout caused intermittent failures when API slow."
```

See `.claude/knowledge/git/commit-messages.md` for format details.

---

## Testing Requirements

### Structure
```
tests/
├── unit/           # Fast, isolated, mocked dependencies
├── integration/    # Multi-component workflows
└── fixtures/       # Test data
```

### Running Tests
```bash
pytest tests/ -v                              # All tests
pytest tests/ --cov=src --cov-report=term-missing  # With coverage
pytest -m "not e2e"                           # Skip slow tests
```

### Test Quality
- Test behavior, not implementation
- One assertion per test (generally)
- Descriptive names: `test_<what>_<when>_<expected>`
- Mock external dependencies
- Aim for >80% coverage

---

## CRITICAL: Development Workflow

### Pull Request Requirement

**ALWAYS use pull requests. NEVER push directly to main.**

```bash
git checkout -b feature/my-feature
# Make changes
pytest tests/ -v
git add .
git commit -m "feat: add feature"
git push -u origin feature/my-feature
gh pr create
```

**AI Assistant Checklist:**
- [ ] Before pushing: Verify current branch is NOT main/master
- [ ] If on main: Immediately create feature branch
- [ ] Always create PR, never merge directly

### Atomic Commits - Required

**Each commit = ONE logical, complete change.**

#### The "AND" Test
```bash
# BAD: Multiple changes
git commit -m "feat: add auth and fix bug and update docs"

# GOOD: Separate logical changes
git commit -m "feat: add user authentication"
git commit -m "fix: resolve login redirect loop"
git commit -m "docs: update API authentication guide"
```

#### Commit Sequences

**Feature:**
```bash
git commit -m "feat: add UserService skeleton"
git commit -m "feat: implement validation logic"
git commit -m "test: add UserService tests"
```

**Bug Fix:**
```bash
git commit -m "test: add failing test for race condition"
git commit -m "fix: resolve cache invalidation race"
```

#### When to Commit
- Added a function/class
- Fixed a single bug
- Added tests for specific functionality
- Made one refactoring change
- ❌ Broken/incomplete changes
- ❌ Multiple unrelated changes

See `.claude/knowledge/git/atomic-commits.md` for detailed guidance.

### Pre-Commit Checks

```bash
uvx ruff check --fix src/ tests/   # Lint and fix
uvx ruff format src/ tests/        # Format
uvx ty check src/                  # Type check
uv run pytest tests/ -v            # Run tests
```

---

## CRITICAL: Configuration as Code Standards

### Why Configuration as Code Matters

**Code is context. Code is text. LLMs understand text.**

In the AI age, expressing configuration and infrastructure as code enables:
- AI assistants can read, understand, and modify configurations
- Version control captures the history of why changes were made
- Code review catches issues before deployment
- Reproducible environments across dev/staging/production

### Configuration as Code Requirements

**Everything expressible should be code:**

```python
# Environment variables - validated at startup
@dataclass
class AppConfig:
    database_url: str
    log_level: str = "INFO"
    debug: bool = False

    @classmethod
    def from_environment(cls) -> "AppConfig":
        database_url = os.getenv("DATABASE_URL")
        if not database_url:
            raise ValueError("DATABASE_URL is required")
        return cls(database_url=database_url, ...)

# Infrastructure - defined in Pulumi/code
bucket = aws.s3.Bucket("uploads", versioning=True, encryption="AES256")

# Feature flags - code, not UI toggles
FEATURE_FLAGS = {
    "new_checkout": os.getenv("FEATURE_NEW_CHECKOUT", "false") == "true",
}
```

**AI Assistant Checklist:**
- [ ] Environment variables documented in `.env.example`
- [ ] Configuration validated at startup (fail fast)
- [ ] Secrets referenced, never hardcoded
- [ ] Infrastructure defined in Pulumi/IaC
- [ ] Same code runs in all environments (config varies)

**Related Documentation:**
- **Configuration as Code:** `.claude/knowledge/infrastructure/configuration-as-code.md`
- **IaC Overview:** `.claude/knowledge/infrastructure/infrastructure-as-code-overview.md`
- **Environment Variables:** `.claude/knowledge/infrastructure/environment-variables.md`
- **Environment Isolation:** `.claude/knowledge/infrastructure/environment-isolation.md`

---

## CRITICAL: Infrastructure Cost Standards

**ALWAYS estimate costs before deploying infrastructure.**

```bash
# Install Infracost
brew install infracost

# Generate cost estimate
infracost breakdown --path .

# Set budget threshold
pulumi config set monthlyBudget 100  # $100/month
```

### Required Tags for All Cloud Resources

```python
{
    "Environment": pulumi.get_stack(),  # dev/staging/production
    "CostCenter": "engineering",
    "Owner": "platform-team",
    "ManagedBy": "pulumi",
    "Project": pulumi.get_project(),
}
```

### Environment-Specific Sizing

| Environment | Instance Size | Availability | Backup Retention |
|-------------|---------------|--------------|------------------|
| Development | t3.micro (free tier) | Single-AZ | 7 days |
| Staging | t3.medium | Single-AZ | 7 days |
| Production | Right-sized | Multi-AZ (HA) | 30+ days |

**AI Assistant Checklist:**
- [ ] Before infrastructure deployment: Run cost estimation
- [ ] Verify estimated cost within budget threshold
- [ ] Tag all resources with required tags
- [ ] Set up budget alerts at 50%, 80%, 100%

**Related Documentation:**
- **Financial Governance:** `.claude/knowledge/infrastructure/financial-governance.md`
- **Pulumi Guide:** `.claude/knowledge/infrastructure/pulumi-guide.md`
- **Database Migrations:** `.claude/knowledge/infrastructure/database-migrations.md`
- **Container Compute:** `.claude/knowledge/infrastructure/container-compute.md`
- **Storage Infrastructure:** `.claude/knowledge/infrastructure/storage-infrastructure.md`
- **Hybrid IaC Approaches:** `.claude/knowledge/infrastructure/hybrid-iac-approaches.md`

---

## Code Quality Tools

### Ruff Configuration
```toml
[tool.ruff]
line-length = 100
target-version = "py311"
select = ["E", "W", "F", "I", "N", "UP", "B", "C4", "SIM"]

[tool.ruff.format]
quote-style = "double"
```

### ty Configuration
```toml
[tool.ty.environment]
python-version = "3.12"

[tool.ty.analysis]
allowed-unresolved-imports = ["package.**"]

[tool.ty.rules]
"unused-ignore-comment" = "ignore"
```

### pytest Configuration
```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = ["-v", "--strict-markers", "--tb=short"]
```

---

## AI Assistant Quick Reference

### Before Any Operation
1. Check virtual environment active: `which python`
2. Verify not on main branch: `git branch --show-current`
3. Read project CLAUDE.md for project-specific rules
4. Before infrastructure changes: Estimate costs with Infracost

### Common Operations

**Install packages:**
```bash
which python  # Must show .venv/bin/python
pip install package-name
```

**Create PR:**
```bash
git checkout -b feature/name
pytest tests/ -v
git add . && git commit -m "feat: description"
git push -u origin feature/name && gh pr create
```

**Run quality checks:**
```bash
uvx ruff check --fix . && uvx ruff format . && uvx ty check src/ && uv run pytest tests/ -v
```

### Common Mistakes to Avoid
- ❌ `pip install` without checking venv
- ❌ `git push` when on main branch
- ❌ Using `print()` instead of logging
- ❌ Catching generic `Exception`
- ❌ Files over 800 lines (code) or 600 lines (markdown)
- ❌ Functions without type hints
- ❌ Committing without running tests
- ❌ Multiple unrelated changes in one commit
- ❌ Commit messages that don't explain why

---

## Related Documentation

- **Critical Rules**: `.claude/standards/CLAUDE_CODE_RULES.md`
- **Knowledge Base**: `.claude/knowledge/` (tactical tips and patterns)
- **Git Workflow**: `.claude/knowledge/git/`
- **Infrastructure**: `.claude/knowledge/infrastructure/`

---

**Status**: These standards are synced to all projects via `persuade sync`.
