# Fake Data Generator Design (FraiseQL Pattern)

**Pattern**: Integer PK/FK, UUID as stable `id` field
**Date**: 2025-10-17

---

## Trinity Pattern (FraiseQL)

```sql
CREATE TABLE catalog.tb_language (
    -- Stable UUID (public identifier, exposed in APIs/views)
    id UUID DEFAULT gen_random_uuid() NOT NULL,

    -- Integer PK (internal optimization, used in FKs)
    pk_language INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,

    -- Human-readable slug
    identifier VARCHAR(255) NOT NULL,

    name VARCHAR(20),
    iso_code VARCHAR(10),

    CONSTRAINT tb_language_id_key UNIQUE (id)
);

CREATE TABLE catalog.tb_country (
    id UUID DEFAULT gen_random_uuid() NOT NULL,
    pk_country INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,

    -- Foreign keys use INTEGER (reference pk_* of parent)
    fk_continent INTEGER REFERENCES tb_continent(pk_continent),

    identifier TEXT NOT NULL
);
```

### Read Side (Views)

```sql
CREATE OR REPLACE VIEW public.tv_locale AS
SELECT
    id,  -- UUID exposed directly (stable across environments)
    code,
    name,
    created_at
FROM tb_locale
WHERE deleted_at IS NULL;
```

**Key difference**: Views expose UUID `id` directly - no need for `pk_locale AS id` aliasing.

---

## Simplified Architecture

### Core Changes from PrintOptim Pattern

| Aspect | PrintOptim Pattern | FraiseQL Pattern |
|--------|-------------------|------------------|
| **UUID field** | `pk_entity` | `id` |
| **Integer PK** | `id` | `pk_entity` |
| **FK type** | UUID | INTEGER |
| **FK target** | `pk_parent` | `pk_parent` |
| **UUID mapping** | Required | **Not needed** ✨ |
| **FK resolution** | UUID lookup | Direct integer |
| **View exposure** | Alias `pk_* AS id` | Use `id` directly |

### Simplified Generator Flow

```python
def _generate_single_row(self, metadata: TableMetadata, overrides: dict[str, Any] | None) -> dict[str, Any]:
    """Generate single row - SIMPLIFIED for FraiseQL pattern"""

    row = {}

    # 1. Generate stable UUID for 'id' field (not pk_entity!)
    row['id'] = self.uuid_gen.generate(metadata.table_code)

    # 2. Skip pk_entity - DB auto-generates
    # 3. Skip fk_* initially - resolve after parent insertion

    for col_name, col_meta in metadata.columns.items():
        if col_name == 'id':  # Already set
            continue
        if col_name == metadata.pk_column:  # pk_entity - DB handles
            continue

        if col_meta.is_fk:
            # FKs are integers - resolve from parent table
            row[col_name] = self._resolve_fk_integer(col_meta)
        elif col_meta.is_identifier:
            row[col_name] = self._generate_identifier(metadata.name)
        elif col_meta.is_audit:
            row[col_name] = self._generate_audit_value(col_name)
        else:
            row[col_name] = self._generate_fake_value(metadata.name, col_name, col_meta.type)

    return row
```

### FK Resolution - Direct Integer Lookup

```python
def _resolve_fk_integer(self, col_meta: ColumnMetadata) -> int:
    """Resolve FK by getting integer PK from parent table"""

    fk_table = col_meta.fk_table
    pk_col = col_meta.fk_column  # e.g., "pk_continent"

    # Simple query - no UUID mapping needed!
    result = self.db.execute(f"""
        SELECT {pk_col}
        FROM {fk_table}
        WHERE deleted_at IS NULL
        ORDER BY random()  -- Or LIMIT 1 for deterministic
        LIMIT 1
    """).fetchone()

    if not result:
        raise ValueError(f"No data in {fk_table} for FK {col_meta.name}")

    return result[0]  # Return integer directly
```

**Benefits**:
- ✅ No UUID→Integer mapping dict needed
- ✅ No memory overhead for large datasets
- ✅ Can query parent table directly during generation
- ✅ Simpler code, fewer state variables

---

## Updated Component Designs

### 1. UUID Generator (Unchanged)

The UUID generator is identical - we still encode metadata:

```python
class SemanticUUIDGenerator:
    """Generate deterministic, metadata-encoded UUIDs"""

    def generate(self, table_code: int, sequence: int | None = None) -> uuid.UUID:
        """Generate UUID for table"""
        if sequence is None:
            sequence = self._next_sequence(table_code)

        # Encode: table_code (32) | scenario (16) | version (16) | sequence (64)
        uuid_bytes = (
            table_code.to_bytes(4, 'big') +
            self.scenario_id.to_bytes(2, 'big') +
            self.version.to_bytes(2, 'big') +
            sequence.to_bytes(8, 'big')
        )
        return uuid.UUID(bytes=uuid_bytes)
```

**Usage**: Same encoding, just stored in `id` field instead of `pk_*` field.

### 2. Schema Introspection (Simplified Metadata)

```python
@dataclass
class ColumnMetadata:
    name: str
    type: str
    is_pk: bool  # True for pk_entity
    is_uuid_id: bool  # True for 'id' column (NEW)
    is_fk: bool
    fk_table: str | None
    fk_column: str | None  # e.g., "pk_continent" (INTEGER!)
    is_nullable: bool
    is_identifier: bool
    is_audit: bool

@dataclass
class TableMetadata:
    schema: str
    name: str
    table_code: int
    columns: dict[str, ColumnMetadata]

    pk_column: str  # "pk_language" (INTEGER)
    uuid_id_column: str  # "id" (UUID) - NEW!
    identifier_column: str | None  # "identifier" (slug)
```

### 3. Simplified FakeDataGenerator

```python
class FakeDataGenerator:
    """Generate fake data for FraiseQL trinity pattern"""

    def __init__(
        self,
        db_connection,
        scenario_id: int,
        locale: str = 'en_US',
        seed: int | None = None
    ):
        self.db = db_connection
        self.introspector = SchemaIntrospector(db_connection)
        self.uuid_gen = SemanticUUIDGenerator(scenario_id)
        self.faker_provider = FakerProvider(locale, seed)

        # NO UUID MAPPING NEEDED! ✨
        # FKs use integers directly from pk_* columns

    def insert_generated_data(
        self,
        table: str,
        rows: list[dict[str, Any]]
    ) -> list[int]:
        """Insert rows and return generated integer PKs"""

        metadata = self.introspector.get_table_metadata(table)
        pk_col = metadata.pk_column  # "pk_language"

        pks = []

        for row in rows:
            cols = ', '.join(row.keys())
            placeholders = ', '.join(['%s'] * len(row))

            query = f"""
                INSERT INTO {table} ({cols})
                VALUES ({placeholders})
                RETURNING {pk_col}
            """

            result = self.db.execute(query, list(row.values())).fetchone()
            pk_int = result[0]
            pks.append(pk_int)

        return pks  # Return integers for child FK references
```

### 4. Parent-Child Insertion Pattern

```python
# Generate parent table
continent_rows = generator.generate_rows('catalog.tb_continent', count=7)
continent_pks = generator.insert_generated_data('catalog.tb_continent', continent_rows)

# Generate child table - FKs automatically resolve to parent integers
country_rows = generator.generate_rows('catalog.tb_country', count=50)
# Each row's fk_continent will be an integer from continent_pks
country_pks = generator.insert_generated_data('catalog.tb_country', country_rows)
```

**Behind the scenes**:
```python
# In _resolve_fk_integer()
SELECT pk_continent FROM tb_continent WHERE deleted_at IS NULL LIMIT 1
# Returns: 42 (an integer)

# Row generated:
{
    'id': UUID('01020304-5001-0001-0000-000000000015'),  # Encoded UUID
    # 'pk_country': <skipped - DB generates>
    'fk_continent': 42,  # Integer FK - direct reference
    'identifier': 'france-15',
    'name': 'France',
    'iso_code': 'FR'
}
```

---

## FK Resolution Strategies

### Strategy 1: Random Parent (Non-Deterministic)

```python
def _resolve_fk_integer(self, col_meta: ColumnMetadata) -> int:
    """Pick random parent row"""
    result = self.db.execute(f"""
        SELECT {col_meta.fk_column}
        FROM {col_meta.fk_table}
        WHERE deleted_at IS NULL
        ORDER BY random()
        LIMIT 1
    """).fetchone()
    return result[0]
```

### Strategy 2: Round-Robin (Deterministic Distribution)

```python
def _resolve_fk_integer(self, col_meta: ColumnMetadata) -> int:
    """Distribute children evenly across parents"""

    # Cache parent PKs
    cache_key = col_meta.fk_table
    if cache_key not in self._parent_pk_cache:
        results = self.db.execute(f"""
            SELECT {col_meta.fk_column}
            FROM {col_meta.fk_table}
            WHERE deleted_at IS NULL
            ORDER BY {col_meta.fk_column}
        """).fetchall()
        self._parent_pk_cache[cache_key] = [r[0] for r in results]

    parent_pks = self._parent_pk_cache[cache_key]

    # Round-robin selection
    idx = self._fk_counter.get(cache_key, 0) % len(parent_pks)
    self._fk_counter[cache_key] = idx + 1

    return parent_pks[idx]
```

### Strategy 3: Explicit Override (Scenario-Specific)

```yaml
# In scenario YAML
tables:
  - name: catalog.tb_country
    count: 50
    overrides:
      fk_continent: 3  # All countries in continent pk=3
```

Or for distribution:

```yaml
tables:
  - name: catalog.tb_country
    count: 50
    providers:
      fk_continent: |
        lambda f: f.random_element([1, 2, 3, 4, 5, 6, 7])  # Distribute across 7 continents
```

---

## Comparison: What Gets Simpler

### Memory Usage

**PrintOptim Pattern**:
```python
# Need to track UUID→Integer mapping for ALL entities
self._uuid_to_pk: dict[uuid.UUID, int] = {}
# For 100K rows → 100K dict entries → ~3-4MB memory
```

**FraiseQL Pattern**:
```python
# Optional: Cache parent PKs for round-robin (only parent tables)
self._parent_pk_cache: dict[str, list[int]] = {}
# For 7 continents → 1 list with 7 integers → ~100 bytes
```

**Savings**: ~99% reduction in memory for child entity generation

### Code Complexity

**PrintOptim Pattern**:
```python
def insert_generated_data(self, table, rows):
    for row in rows:
        pk_int = db.insert(...).returning('id')

        # Extract UUID from row to store mapping
        uuid_col = metadata.uuid_pk_column  # "pk_language"
        if uuid_col in row:
            self._uuid_to_pk[row[uuid_col]] = pk_int  # Track mapping

    return pks

def _resolve_foreign_key(self, col_meta):
    # Need to maintain/lookup UUID mapping
    parent_uuid = self._get_parent_uuid(...)
    return parent_uuid  # Return UUID for FK
```

**FraiseQL Pattern**:
```python
def insert_generated_data(self, table, rows):
    for row in rows:
        pk_int = db.insert(...).returning('pk_language')
        # No mapping needed!

    return pks

def _resolve_fk_integer(self, col_meta):
    # Direct query - no mapping
    pk = db.execute(f"SELECT {col_meta.fk_column} FROM {col_meta.fk_table} LIMIT 1")
    return pk  # Return integer for FK
```

**Savings**: ~30% less code, no state management

---

## UUID Encoding Strategy (Unchanged)

The UUID encoding remains identical - we're just storing it in `id` instead of `pk_*`:

```
Example UUID: 01020304-5001-0001-0000-000000000042
              ^^^^^^^^ ^^^^ ^^^^ ^^^^^^^^^^^^^^^^
              Table    Scen Ver  Sequence

Decoding:
- Table code: 0x01020304 → tb_language
- Scenario:   5001 → "minimal_seed"
- Version:    1
- Sequence:   66
```

**Storage**:
```sql
INSERT INTO catalog.tb_language (
    id,  -- Store encoded UUID here (not pk_language!)
    identifier,
    name,
    iso_code
) VALUES (
    '01020304-5001-0001-0000-000000000042',
    'en',
    'English',
    'en'
)
RETURNING pk_language;  -- DB generates: 1, 2, 3, ...
```

---

## Read Side: Cleaner Views

### PrintOptim Pattern (Aliasing Required)

```sql
CREATE OR REPLACE VIEW public.v_locale AS
SELECT
    pk_locale AS id,  -- Must alias UUID to 'id'
    pk_locale,        -- Also expose original name
    code,
    name
FROM tb_locale;
```

### FraiseQL Pattern (Direct Exposure)

```sql
CREATE OR REPLACE VIEW public.tv_locale AS
SELECT
    id,    -- Already a UUID, just expose directly!
    code,
    name
FROM tb_locale;
```

**Benefits**:
- ✅ Less aliasing confusion
- ✅ Consistent naming (always `id` for UUID)
- ✅ Clearer API contracts (GraphQL, REST always use `id`)

---

## Migration from PrintOptim to FraiseQL Pattern

If converting an existing generator:

### Changes Required

1. **Column Mapping**:
   ```python
   # OLD
   uuid_pk_column = "pk_language"  # UUID field
   pk_column = "id"                # Integer field

   # NEW
   uuid_id_column = "id"           # UUID field
   pk_column = "pk_language"       # Integer field
   ```

2. **FK Type**:
   ```python
   # OLD
   fk_type = 'UUID'

   # NEW
   fk_type = 'INTEGER'
   ```

3. **Remove UUID Mapping**:
   ```python
   # OLD
   self._uuid_to_pk: dict[uuid.UUID, int] = {}

   # NEW
   # Delete this entirely!
   ```

4. **FK Resolution**:
   ```python
   # OLD
   def _resolve_foreign_key(self, col_meta) -> uuid.UUID:
       parent_uuid = ...
       return parent_uuid

   # NEW
   def _resolve_fk_integer(self, col_meta) -> int:
       parent_pk = db.execute(f"SELECT {col_meta.fk_column} FROM {col_meta.fk_table} LIMIT 1")
       return parent_pk[0]
   ```

---

## Summary: What's Better in FraiseQL Pattern

| Aspect | Improvement |
|--------|-------------|
| **Memory** | 99% reduction (no UUID→int mapping) |
| **Code complexity** | 30% less code |
| **FK resolution** | Direct integer lookup (simpler) |
| **View definitions** | No aliasing needed |
| **API consistency** | Always use `id` for UUID |
| **Performance** | Slightly faster (integers vs UUIDs for joins) |
| **Debugging** | Same UUID encoding benefits |
| **State management** | No mapping dict to maintain |

**The only tradeoff**: Views don't expose the `pk_*` integer (but they shouldn't - that's internal).

---

## Recommended Implementation Order

1. **UUID Generator** (identical to PrintOptim)
2. **Schema Introspector** (detect `id` as UUID, `pk_*` as integer)
3. **FK Resolution** (direct integer queries)
4. **Generator** (skip UUID mapping entirely)
5. **Scenario Manager** (identical to PrintOptim)

---

**This pattern is strictly simpler to implement and maintain than the PrintOptim pattern.**

The core insight: **UUIDs are for stability, integers are for performance. Don't mix them in FK relationships.**
