# v0.5.0 Roadmap

This document tracks planned features, improvements, and implementation details
for v0.5.0.

## Theme: Stability, Squawk-Inspired Rules & Integrations

Building on v0.4.0's plugin system and 27 rules, v0.5.0 prioritizes fixing false
positives and bugs in existing rules, improving test quality, then adding rules
inspired by [Squawk](https://squawkhq.com/) and expanding integrations.

______________________________________________________________________

## Bug Fixes & Quality (High Priority — Do First)

These must be addressed before adding new rules. False positives in existing
rules will drive users away before any new feature matters.

### Critical Bugs

| Issue                   | Rule         | Description                                                                                                                                   | Status |
| ----------------------- | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
| SM001 UUIDField bypass  | SM001        | `UUIDField` unconditionally whitelisted even without a default — should only skip if `default` is set (e.g., `uuid.uuid4`)                    | Done   |
| SM009/SM011 duplication | SM009, SM011 | Both fire on `AddConstraint(UniqueConstraint(...))` producing duplicate issues — deduplicate or merge                                         | Done   |
| SM004 dead code         | SM004        | `METADATA_ONLY_ATTRIBUTES` and `SAFE_DIRECTION_ATTRIBUTES` defined but never referenced in check logic                                        | Done   |
| SM010 docstring error   | SM010        | Claims `CREATE INDEX` takes ACCESS EXCLUSIVE lock — actually takes SHARE lock (blocking writes, not reads). Advice is correct, docs are wrong | Done   |

### AlterField Before-State Gap

All AlterField rules only see the _new_ field definition. They cannot compare
with the previous state, causing both false positives and false negatives. This
is the most systemic issue in the codebase.

| Issue                 | Rule  | Description                                                                                                                        | Status |
| --------------------- | ----- | ---------------------------------------------------------------------------------------------------------------------------------- | ------ |
| SM020 false positives | SM020 | Fires on every `AlterField` where `null=False` (the default for most fields). Changing `help_text` on a `CharField` triggers SM020 | Done   |
| SM021 false positives | SM021 | Fires on every `AlterField` where `unique=True`, even if field was already unique                                                  | Done   |
| SM013 false positives | SM013 | Cannot distinguish increasing vs decreasing `max_length` — fires on all `CharField` `AlterField` ops                               | Done   |
| SM004 false negatives | SM004 | Silently passes any `AlterField` where `null=True`, even dangerous type changes like `CharField` to `IntegerField`                 | Done   |

**Fix approach:** Integrate with Django's migration state resolver to obtain the
old field definition, or use heuristics comparing the operation's field with the
model state at that point in the migration graph. If full state resolution is too
complex for v0.5.0, downgrade affected rules to INFO severity and document them
as "best-effort heuristics."

```python
# Ideal fix: resolve old field from migration state
from django.db.migrations.state import ProjectState

def get_old_field(migration, operation):
    """Resolve the field state before this operation."""
    state = ProjectState()
    loader = MigrationLoader(None)
    # Build state up to (but not including) this migration
    state = loader.project_state((app_label, migration_name), at_end=False)
    model_state = state.models.get((app_label, operation.model_name))
    if model_state:
        return model_state.fields.get(operation.name)
    return None
```

### False Positive Reduction

| Issue                  | Rule  | Description                                                                                                                                                                  | Status |
| ---------------------- | ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
| SM019 keyword list     | SM019 | Flags `name`, `status`, `date`, `value`, `type`, `admin`, `level`, `role` — none are SQL reserved keywords. Trim list to actual reserved words per SQL standard and database | Done   |
| SM024 JSON/array match | SM024 | Pattern `\{[^}]*\}` matches valid PostgreSQL JSON/array syntax `'{}'`. Pattern `%s` matches `LIKE '%something%'`                                                             | Done   |
| SM012 broad regex      | SM012 | Second pattern `add\s+value` matches any SQL containing those words, not just `ALTER TYPE ... ADD VALUE`                                                                     | Done   |
| SM022 substring match  | SM022 | `"now" in full_name` matches `"renow_something"` — use word boundary or exact match                                                                                          | Done   |

### Reporter Fixes

| Issue                 | Reporter | Description                                                                           | Status |
| --------------------- | -------- | ------------------------------------------------------------------------------------- | ------ |
| SARIF helpUri 404s    | SARIF    | `helpUri` points to `/rules/SM001/` on readthedocs — pages likely don't exist         | Done   |
| SARIF absolute paths  | SARIF    | `file_path` is absolute; GitHub Code Scanning expects relative paths from `%SRCROOT%` | Done   |
| Console Unicode crash | Console  | `"✖"`, `"⚠"`, `"💡"` symbols crash on non-UTF-8 terminals (e.g., Windows cp1252)      | Done   |
| GitHub title escaping | GitHub   | `issue.operation` used in title without escaping `%`, `\n`, `\r`                      | Done   |
| JSON summary DRY      | JSON     | `_get_summary()` duplicates logic from `MigrationAnalyzer.get_summary()`              | Done   |

### Test Quality

| Issue                     | Description                                                                                                                           | Status |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ------ |
| Tests that cannot fail    | `TestErrorRecovery` uses `try/except/pass` pattern — tests always pass regardless of behavior. Delete or rewrite with real assertions | Done   |
| Tautological assertions   | `assert result is None or result is not None` appears in multiple test files — remove                                                 | Done   |
| Missing unit tests        | `analyze_all()`, `analyze_new_migrations()`, severity overrides are untested                                                          | Done   |
| Integration test weakness | Several tests catch `SystemExit` then assert `True` or `output is not None` — add meaningful assertions                               | Done   |
| SM026 silent skip         | Every SM026 test wraps in `try/except OSError: pytest.skip()` — can silently pass entire test class on some CI environments           | Done   |

### Other Fixes

| Issue                           | Location                 | Description                                                                                                                                               | Status |
| ------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
| SM027 architecture              | `graph.py`               | Standalone function, not a `BaseRule` subclass — breaks suppression/config pipeline, `operation` field is a string, `file_path`/`line_number` always None | Done   |
| SM023 dead branch               | `relations.py`           | Checks `through != "auto"` but Django never sets `through` to the string `"auto"` — dead code                                                             | Done   |
| SM001 db_default                | `add_field.py`           | Does not recognize `db_default` (Django 5.0+) as "having a default" — false positive on fields using `db_default`                                         | Done   |
| Suppression validation          | `suppression.py`         | Typos like `# safe-migrations: ignore SM999` silently accepted — no warning for non-existent rule IDs                                                     | Done   |
| `get_db_vendor` swallows errors | `utils.py`               | Bare `except Exception` returns `"unknown"`, silently disabling all vendor-specific rules with no warning                                                 | Done   |
| PostGIS vendor detection        | `utils.py`/`analyzer.py` | PostGIS reports `database_vendor = 'postgis'` — all PostgreSQL rules silently skipped                                                                     | Done   |

______________________________________________________________________

## New Rules (Inspired by Squawk)

### High Priority

| Rule  | Name                              | Severity | Status |
| ----- | --------------------------------- | -------- | ------ |
| SM028 | `prefer_bigint_over_int`          | WARNING  | Done   |
| SM029 | `drop_not_null`                   | WARNING  | Done   |
| SM030 | `require_concurrent_index_delete` | ERROR    | Done   |

### Medium Priority

| Rule  | Name                        | Severity | Status |
| ----- | --------------------------- | -------- | ------ |
| SM031 | `prefer_text_over_varchar`  | INFO     | Done   |
| SM032 | `prefer_timestamptz`        | INFO     | Done   |
| SM033 | `adding_field_with_default` | WARNING  | Done   |
| SM034 | `prefer_identity`           | INFO     | Done   |

### Low Priority

| Rule  | Name                   | Severity | Status |
| ----- | ---------------------- | -------- | ------ |
| SM035 | `require_lock_timeout` | INFO     | Done   |
| SM036 | `prefer_if_exists`     | INFO     | Done   |

______________________________________________________________________

## Rule Implementation Details

### SM028: prefer_bigint_over_int

**Problem:** Django's default `AutoField` uses 32-bit integers, which max out at
~2.1 billion. High-volume tables can hit this limit.

**Detection:**

```python
# Detect AutoField or IntegerField as primary key
if isinstance(field, (models.AutoField, models.IntegerField)):
    if field.primary_key:
        # Check if BigAutoField would be better
        return self.create_issue(...)
```

**Trigger conditions:**

- `AutoField` used as primary key (suggest `BigAutoField`)
- `IntegerField` used as primary key (suggest `BigIntegerField`)
- New models with default ID field

**Safe pattern:**

```python
class HighVolumeModel(models.Model):
    id = models.BigAutoField(primary_key=True)
```

**Implementation file:** `django_safe_migrations/rules/add_field.py`

______________________________________________________________________

### SM029: drop_not_null

**Problem:** Removing `NOT NULL` constraint via `AlterField(null=True)` can lead
to data integrity issues if application code doesn't handle NULLs.

**Detection:**

```python
# Detect AlterField changing null=False to null=True
if isinstance(operation, migrations.AlterField):
    if getattr(operation.field, 'null', False) is True:
        # This might be removing NOT NULL
        return self.create_issue(...)
```

**Trigger conditions:**

- `AlterField` with `null=True` on a previously `null=False` field
- Requires state tracking or heuristic detection

**Safe pattern:**

```python
# Ensure application code handles NULL before migration
# Add default value handling in model
class MyModel(models.Model):
    field = models.CharField(null=True, default='')

    def get_field_display(self):
        return self.field or 'N/A'
```

**Implementation file:** `django_safe_migrations/rules/alter_field.py`

______________________________________________________________________

### SM030: require_concurrent_index_delete

**Problem:** `RemoveIndex` on large PostgreSQL tables blocks writes. Should use
`RemoveIndexConcurrently`.

**Detection:**

```python
# Detect RemoveIndex without CONCURRENTLY
if isinstance(operation, migrations.RemoveIndex):
    # PostgreSQL only
    if db_vendor == 'postgresql':
        return self.create_issue(...)
```

**Trigger conditions:**

- `migrations.RemoveIndex` operation
- PostgreSQL database
- Not using `RemoveIndexConcurrently`

**Safe pattern:**

```python
from django.contrib.postgres.operations import RemoveIndexConcurrently

class Migration(migrations.Migration):
    atomic = False  # Required for concurrent operations

    operations = [
        RemoveIndexConcurrently(
            model_name='order',
            name='order_status_idx',
        ),
    ]
```

**Implementation file:** `django_safe_migrations/rules/add_index.py`

______________________________________________________________________

### SM031: prefer_text_over_varchar

**Problem:** `CharField` with `max_length` is often unnecessary. PostgreSQL's
`TEXT` type has no performance difference from `VARCHAR`.

**Detection:**

```python
# Detect CharField in PostgreSQL
if isinstance(field, models.CharField):
    if db_vendor == 'postgresql':
        return self.create_issue(
            severity=Severity.INFO,
            message="Consider TextField instead of CharField for flexibility"
        )
```

**Notes:**

- INFO severity (suggestion, not error)
- PostgreSQL-specific (other DBs have VARCHAR performance differences)
- Skip if `max_length` is for validation purposes

**Implementation file:** `django_safe_migrations/rules/add_field.py`

______________________________________________________________________

### SM032: prefer_timestamptz

**Problem:** `DateTimeField` without timezone awareness can cause issues in
multi-timezone applications.

**Detection:**

```python
# Check Django settings for USE_TZ
# Warn if DateTimeField added when USE_TZ=False
if isinstance(field, models.DateTimeField):
    if not getattr(settings, 'USE_TZ', False):
        return self.create_issue(...)
```

**Safe pattern:**

```python
# settings.py
USE_TZ = True

# models.py
class Event(models.Model):
    created_at = models.DateTimeField(auto_now_add=True)  # Will use TIMESTAMPTZ
```

**Implementation file:** `django_safe_migrations/rules/add_field.py`

______________________________________________________________________

### SM033: adding_field_with_default

**Problem:** Adding a field with ANY default value on large tables is slow
because it must update every existing row.

**Detection:**

```python
# Detect AddField with default (not just callable)
if isinstance(operation, migrations.AddField):
    if hasattr(operation.field, 'default'):
        if operation.field.default is not models.NOT_PROVIDED:
            return self.create_issue(...)
```

**Difference from SM022:**

- SM022: Warns about expensive _callable_ defaults (e.g., `datetime.now`)
- SM033: Warns about ANY default on large tables

**Safe pattern:**

```python
# Migration 1: Add nullable field
migrations.AddField(
    model_name='user',
    name='status',
    field=models.CharField(max_length=20, null=True),
)

# Migration 2: Backfill in batches
def backfill_status(apps, schema_editor):
    User = apps.get_model('myapp', 'User')
    User.objects.filter(status__isnull=True).update(status='active')

# Migration 3: Add default and remove null
migrations.AlterField(
    model_name='user',
    name='status',
    field=models.CharField(max_length=20, default='active'),
)
```

**Implementation file:** `django_safe_migrations/rules/add_field.py`

______________________________________________________________________

### SM034: prefer_identity

**Problem:** PostgreSQL 10+ supports `IDENTITY` columns which are preferred over
`SERIAL` for auto-incrementing fields.

**Detection:**

```python
# Detect AutoField/BigAutoField on PostgreSQL < 10 style
# Django 4.0+ uses IDENTITY by default
if isinstance(field, (models.AutoField, models.BigAutoField)):
    if db_vendor == 'postgresql':
        # Check PostgreSQL version if possible
        return self.create_issue(severity=Severity.INFO, ...)
```

**Notes:**

- INFO severity (modern best practice)
- Only relevant for older Django versions or manual serial definitions
- Django 3.2+ defaults to IDENTITY

**Implementation file:** `django_safe_migrations/rules/add_field.py`

______________________________________________________________________

### SM035: require_lock_timeout

**Problem:** Migrations without lock timeouts can hang indefinitely waiting for
locks on busy tables.

**Detection:**

```python
# Check if migration sets lock_timeout via RunSQL or connection
# This is an advanced check that may require parsing RunSQL statements
```

**Safe pattern:**

```python
class Migration(migrations.Migration):
    operations = [
        migrations.RunSQL(
            sql="SET lock_timeout = '5s';",
            reverse_sql="SET lock_timeout = '0';",
        ),
        # ... other operations
    ]
```

**Implementation file:** `django_safe_migrations/rules/run_sql.py`

______________________________________________________________________

### SM036: prefer_if_exists

**Problem:** Migrations without `IF EXISTS` / `IF NOT EXISTS` can fail on
partial rollbacks or re-runs.

**Detection:**

```python
# Check RunSQL for CREATE/DROP without IF EXISTS
if isinstance(operation, migrations.RunSQL):
    sql = operation.sql.upper()
    if 'CREATE TABLE' in sql and 'IF NOT EXISTS' not in sql:
        return self.create_issue(...)
    if 'DROP TABLE' in sql and 'IF EXISTS' not in sql:
        return self.create_issue(...)
```

**Safe pattern:**

```python
migrations.RunSQL(
    sql="CREATE TABLE IF NOT EXISTS temp_data (...)",
    reverse_sql="DROP TABLE IF EXISTS temp_data",
)
```

**Implementation file:** `django_safe_migrations/rules/run_sql.py`

______________________________________________________________________

## Features

### High Priority (Deferred from v0.4.0)

#### Interactive Mode (`--interactive`)

**Description:** Guide users through handling detected issues interactively.

**Implementation:**

```python
# cli.py additions
@click.option('--interactive', '-i', is_flag=True,
              help='Interactive mode for handling issues')
def main(interactive, ...):
    if interactive:
        for issue in issues:
            display_issue_detail(issue)
            action = prompt_action(issue)  # suppress, fix, skip
            if action == 'suppress':
                generate_suppression_comment(issue)
```

**Features:**

- Display detailed explanation for each rule
- Show safe alternatives with code examples
- Generate suppression comments
- Allow bulk actions (suppress all SM001, etc.)

**Files to modify:**

- `django_safe_migrations/cli.py`
- `django_safe_migrations/management/commands/check_migrations.py`
- New: `django_safe_migrations/interactive.py`

______________________________________________________________________

#### Progress Reporting (`--verbose`)

**Description:** Show detailed progress for large codebases.

**Implementation:**

```python
# analyzer.py additions
def analyze_app(self, app_label, verbose=False):
    if verbose:
        print(f"Analyzing {app_label}...")
    for migration in migrations:
        if verbose:
            print(f"  {migration.name}")
        # ... analysis
```

**Features:**

- Per-app progress
- Per-migration progress
- Timing information
- Summary statistics (issues by severity, by rule)

**Files to modify:**

- `django_safe_migrations/analyzer.py`
- `django_safe_migrations/cli.py`

______________________________________________________________________

### Medium Priority

#### Watch Mode (`--watch`)

**Description:** Re-run analysis when migration files change.

**Implementation:**

```python
# New file: django_safe_migrations/watch.py
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class MigrationWatcher(FileSystemEventHandler):
    def on_modified(self, event):
        if event.src_path.endswith('.py'):
            if '/migrations/' in event.src_path:
                run_analysis()
```

**Dependencies:**

- `watchdog` (optional dependency)

______________________________________________________________________

#### Baseline Support

**Description:** Ignore existing issues, only flag new ones.

**Implementation:**

```python
# New file: django_safe_migrations/baseline.py

def generate_baseline(issues):
    """Generate baseline file from current issues."""
    baseline = {
        'generated_at': datetime.now().isoformat(),
        'issues': [
            {
                'rule_id': issue.rule_id,
                'file': issue.file_path,
                'line': issue.line_number,
                'hash': compute_issue_hash(issue),
            }
            for issue in issues
        ]
    }
    return baseline

def filter_baseline_issues(issues, baseline):
    """Remove issues that are in the baseline."""
    baseline_hashes = {i['hash'] for i in baseline['issues']}
    return [i for i in issues if compute_issue_hash(i) not in baseline_hashes]
```

**CLI:**

```bash
# Generate baseline
python manage.py check_migrations --generate-baseline > .migration-baseline.json

# Use baseline
python manage.py check_migrations --baseline=.migration-baseline.json
```

______________________________________________________________________

#### Diff Mode

**Description:** Only analyze changed migrations.

**Implementation:**

```python
# New file: django_safe_migrations/diff.py
import subprocess

def get_changed_migrations(base_ref='origin/main'):
    """Get list of changed migration files from git."""
    result = subprocess.run(
        ['git', 'diff', '--name-only', base_ref, '--', '*/migrations/*.py'],
        capture_output=True, text=True
    )
    return result.stdout.strip().split('\n')
```

**CLI:**

```bash
# Analyze only changed migrations
python manage.py check_migrations --diff
python manage.py check_migrations --diff=origin/main
```

______________________________________________________________________

## Integrations

### VS Code Extension

**Description:** Real-time linting in VS Code.

**Architecture:**

```text
┌─────────────────┐     ┌──────────────────┐
│  VS Code        │────▶│  Language Server │
│  Extension      │◀────│  (Python)        │
└─────────────────┘     └──────────────────┘
                               │
                               ▼
                        ┌──────────────────┐
                        │  django-safe-    │
                        │  migrations      │
                        └──────────────────┘
```

**Features:**

- Diagnostics (squiggles under problematic code)
- Quick fixes (suppress, show safe pattern)
- Hover information (rule explanation)
- Code actions (generate migration fix)

**Implementation:**

- Separate repository: `django-safe-migrations-vscode`
- Use `pygls` for Language Server Protocol
- Bundle as VS Code extension

______________________________________________________________________

### MkDocs Deployment

**Description:** Deploy documentation to GitHub Pages.

**Implementation:**

```yaml
# .github/workflows/docs.yml
name: Deploy Docs

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install mkdocs-material mkdocstrings[python]
      - run: mkdocs gh-deploy --force
```

______________________________________________________________________

### GitLab Code Quality Report

**Description:** Output in GitLab Code Quality format for MR annotations.

**Implementation:**

```python
# New file: django_safe_migrations/reporters/gitlab.py

class GitLabCodeQualityReporter(BaseReporter):
    def report(self, issues):
        output = []
        for issue in issues:
            output.append({
                'description': issue.message,
                'fingerprint': compute_fingerprint(issue),
                'severity': map_severity(issue.severity),
                'location': {
                    'path': issue.file_path,
                    'lines': {'begin': issue.line_number}
                }
            })
        return json.dumps(output)
```

**CLI:**

```bash
python manage.py check_migrations --format=gitlab > gl-code-quality-report.json
```

______________________________________________________________________

## CI/CD

### Release Automation

**Description:** Automated version bumping and PyPI publishing.

**Implementation:**

```yaml
# .github/workflows/release.yml
name: Release

on:
  push:
    tags: ["v*"]

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
      - run: pip install build twine
      - run: python -m build
      - run: twine upload dist/*
        env:
          TWINE_USERNAME: __token__
          TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
```

______________________________________________________________________

## Testing Requirements

### Fix Existing Tests (Do First)

- Remove all `try/except/pass` tests in `TestErrorRecovery` — replace with real
  assertions or delete
- Remove tautological `assert result is None or result is not None` assertions
- Fix integration tests that catch `SystemExit` and assert `True`
- Add unit tests for `analyze_all()`, `analyze_new_migrations()`, and severity
  override behavior
- Fix SM026 `OSError` skip pattern — use class-level `pytest.mark.skipif`
  instead of per-test try/except

### New Test Files

- `tests/unit/rules/test_squawk_rules.py` - Tests for SM028-SM036
- `tests/unit/test_interactive.py` - Interactive mode tests
- `tests/unit/test_baseline.py` - Baseline functionality tests
- `tests/unit/test_diff.py` - Diff mode tests
- `tests/unit/reporters/test_gitlab.py` - GitLab reporter tests

### Integration Tests

- Test with real Django projects (django-Oscar, Wagtail)
- Cross-database tests (PostgreSQL, MySQL, SQLite)

______________________________________________________________________

## Breaking Changes

None planned.

______________________________________________________________________

## Migration Guide

N/A - no breaking changes.

______________________________________________________________________

## Priority Legend

- **High Priority**: Core functionality, high user impact
- **Medium Priority**: Nice-to-have, moderate impact
- **Low Priority**: Future enhancements, can defer to v0.6.0+

______________________________________________________________________

## References

- [Squawk PostgreSQL Linter](https://squawkhq.com/)
- [PostgreSQL IDENTITY columns](https://www.postgresql.org/docs/current/sql-createtable.html)
- [Django BigAutoField](https://docs.djangoproject.com/en/stable/ref/models/fields/#bigautofield)
- [VS Code Language Server Protocol](https://microsoft.github.io/language-server-protocol/)
