# Engineering Standards (AI-Optimized)

> **For AI Assistants**: This is a concise, actionable version of organizational standards.
> **Human developers**: See `/docs/ENGINEERING_STANDARDS.md` for comprehensive version.

---

## Core Principles

### Simple > Complex
Choose straightforward solutions. Avoid clever code.

```python
# GOOD
numbers = [int(x) for x in data.split()]
odd_numbers = [n for n in numbers if n % 2 == 1]
result = sum(odd_numbers)
```

### Explicit > Implicit
Use enums, type hints, clear names. No magic.

```python
# GOOD
class ProcessingMode(Enum):
    NORMALIZE = "normalize"
    VALIDATE = "validate"

def process_data(df: pd.DataFrame, mode: ProcessingMode) -> pd.DataFrame:
    match mode:
        case ProcessingMode.NORMALIZE:
            return normalize_columns(df)
```

### DRY (Don't Repeat Yourself)
Extract shared logic immediately.

### Fail Fast
Validate inputs early. Raise specific exceptions with clear messages.

### YAGNI (You Aren't Gonna Need It)
Build what's needed now. Add features when actually required.

### File Size Limit: 500 Lines
If file exceeds 500 lines, split it. Extract helpers, create submodules.

---

## ⚠️ CRITICAL: Python Requirements

### Virtual Environments - MANDATORY

**ALWAYS use virtual environments. NEVER modify system Python.**

```bash
# Required workflow
python3 -m venv .venv
source .venv/bin/activate  # macOS/Linux
pip install -r requirements.txt
```

**AI Assistant Checklist:**
- [ ] Before `pip install`: Verify `which python` shows `.venv/bin/python`
- [ ] NEVER use: `pip install` (without venv), `--user`, `sudo pip`
- [ ] If no venv exists: Create it first, then activate, then install

### Type Hints - REQUIRED

All functions MUST have type hints.

```python
from typing import Protocol
from pathlib import Path
import pandas as pd

# Required
def process_data(
    data: pd.DataFrame,
    threshold: float = 0.8
) -> tuple[pd.DataFrame, list[str]]:
    """Process dataframe and return (result, errors)."""
    ...

# Data classes with types
@dataclass
class User:
    id: str
    name: str
    email: str
    created_at: date
    is_active: bool = True
```

### Error Handling - Explicit Only

**Only catch exceptions you expect and can handle.**

```python
# GOOD: Explicit validation
def parse_file(path: Path) -> pd.DataFrame:
    if not path.exists():
        raise FileNotFoundError(f"File not found: {path}")

    if path.stat().st_size > 100_000_000:
        raise ValueError(f"File too large (>100MB): {path}")

    try:
        df = pd.read_excel(path)  # Expected: BadZipFile for corrupt files
    except BadZipFile as e:
        raise ValueError(f"Corrupt Excel file: {path}") from e

    return df

# BAD: Catching everything
try:
    return process_dataframe(df)
except Exception:  # Don't do this
    return None
```

**When to use try/except:**
- ✅ File operations (FileNotFoundError expected)
- ✅ External APIs (network errors expected)
- ✅ Parsing user input (ValueError expected)
- ❌ Regular Python logic (bugs should crash)

### Logging - Structured Only

**Always use structured logging. NEVER use `print()`.**

```python
import structlog

logger = structlog.get_logger()

def process_batch(batch_id: str, items: list[Item]) -> BatchResult:
    logger.info(
        "batch_processing_started",
        batch_id=batch_id,
        item_count=len(items)
    )

    # ... processing ...

    logger.info(
        "batch_processing_completed",
        batch_id=batch_id,
        success_count=len(results),
        error_count=len(errors)
    )

    return BatchResult(results, errors)
```

---

## Code Style

- **Line length**: 100 characters max
- **Formatter**: Ruff
- **Type checker**: mypy with --strict
- **Docstrings**: Google-style for public functions/classes

### Naming Conventions

```python
# Constants
MAX_RETRY_COUNT = 3

# Functions and variables
def calculate_total_price(items: list[Item]) -> Decimal:
    total_price = sum(item.price for item in items)
    return total_price

# Classes
class DataProcessor:
    pass

# Private methods
class MyClass:
    def _internal_helper(self):
        pass
```

---

## Architecture Patterns

### Separation of Concerns

Each module/class/function has ONE clear responsibility.

```
src/myproject/
├── parsers/        # Read and parse files
├── validators/     # Validate data quality
├── processors/     # Transform data
├── exporters/      # Export formats
├── models/         # Data models (Pydantic)
├── services/       # Business logic orchestration
└── utils/          # Shared utilities
```

### Dependency Injection

Don't create dependencies inside; inject from outside.

```python
# GOOD
class ProcessingService:
    def __init__(
        self,
        parser: DataParser,
        validator: DataValidator,
    ):
        self.parser = parser
        self.validator = validator
```

### Composition Over Inheritance

Build complex behavior by combining simple objects.

### Immutability Where Possible

```python
# Immutable data classes
@dataclass(frozen=True)
class User:
    id: str
    name: str

# Pure functions
def normalize_name(name: str) -> str:
    return name.strip().upper()
```

---

## Testing Requirements

### Structure

```
tests/
├── unit/           # Fast, isolated, mocked dependencies
├── integration/    # Multi-component workflows
└── fixtures/       # Test data
```

### Running Tests

```bash
# All tests
pytest tests/ -v

# With coverage
pytest tests/ --cov=src --cov-report=term-missing

# Skip slow tests
pytest -m "not e2e"
```

### Test Quality

- Test behavior, not implementation
- One assertion per test
- Descriptive names: `test_<what>_<when>_<expected>`
- Mock external dependencies
- Aim for >80% coverage

---

## ⚠️ CRITICAL: Development Workflow

### Pull Request Requirement

**ALWAYS use pull requests. NEVER push to main.**

```bash
# Required workflow
git checkout -b feature/my-feature
# Make changes
pytest tests/ -v
git add .
git commit -m "feat: add feature"
git push -u origin feature/my-feature
gh pr create
```

**AI Assistant Checklist:**
- [ ] Before pushing: Verify current branch is NOT main/master
- [ ] If on main: Immediately create feature branch
- [ ] Always create PR, never merge directly

### Pre-Commit Checks

Run before committing:

```bash
# Lint and fix
ruff check --fix src/ tests/

# Format
ruff format src/ tests/

# Type check
mypy src/

# Run tests
pytest tests/ -v
```

### Commit Messages

```bash
# Format: <type>: <description>
feat: add user authentication
fix: resolve database connection issue
docs: update API documentation
refactor: simplify data processing logic
test: add unit tests for parser

# Include co-authors
Co-Authored-By: Name <email@example.com>
```

---

## Code Quality Tools

### Ruff Configuration

```toml
[tool.ruff]
line-length = 100
target-version = "py311"

select = [
    "E",   # pycodestyle errors
    "W",   # pycodestyle warnings
    "F",   # pyflakes
    "I",   # isort
    "N",   # pep8-naming
    "UP",  # pyupgrade
    "B",   # bugbear
    "C4",  # comprehensions
    "SIM", # simplify
]

[tool.ruff.format]
quote-style = "double"
```

### mypy Configuration

```toml
[tool.mypy]
python_version = "3.11"
strict = true
warn_return_any = true
disallow_untyped_defs = true
```

### pytest Configuration

```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = ["-v", "--strict-markers", "--tb=short"]
markers = [
    "e2e: end-to-end tests (may be slow)",
    "unit: fast unit tests",
]
```

---

## AI Assistant Quick Reference

### Before Any Operation

1. Check virtual environment active: `which python`
2. Verify not on main branch: `git branch --show-current`
3. Read project CLAUDE.md for project-specific rules

### Common Operations

**Install packages:**
```bash
# Verify venv first
which python  # Must show .venv/bin/python
pip install package-name
```

**Create PR:**
```bash
git checkout -b feature/name
# Make changes
pytest tests/ -v
git add .
git commit -m "feat: description"
git push -u origin feature/name
gh pr create
```

**Run quality checks:**
```bash
ruff check --fix .
ruff format .
mypy src/
pytest tests/ -v
```

### Common Mistakes to Avoid

❌ `pip install` without checking venv
❌ `git push` when on main branch
❌ Using `print()` instead of logging
❌ Catching generic `Exception`
❌ Files over 500 lines
❌ Functions without type hints
❌ Committing without running tests

---

## Related Documentation

- **Full Standards**: `/docs/ENGINEERING_STANDARDS.md` (human-facing, comprehensive)
- **Critical Rules**: `.claude/docs/CLAUDE_CODE_RULES.md` (non-negotiable requirements)
- **Knowledge Base**: `.claude/knowledge/` (tactical tips and patterns)

---

**Status**: These standards apply to all organizational projects. Updated: 2025-12-04
