# Spec 007: Many-to-Many Relationships

Phase 4 of Hybrid Storage — complete relational support with junction tables.

## Problem

Currently, only many-to-one relationships are fully supported. Many-to-many relationships (e.g., Product ↔ Tag, Student ↔ Course) require junction tables which aren't implemented yet.

## Goals

1. Auto-generate junction tables for many-to-many relationships
2. Provide clean APIs to manage relationship links
3. Support both shared and dedicated storage modes
4. Integrate with existing cascade behavior (Spec 006)
5. Update schema context for LLM-generated JOINs

## Design Principles

1. **SQL is the interface** — no ORM-style eager loading; agents use SQL
2. **Storage-aware** — junction tables adapt to storage modes
3. **Minimal API surface** — simple link/unlink operations
4. **Cascade-aware** — respect on_delete from both sides

## Junction Table Architecture

### Naming Convention

```
kdb_{source_entity}_{target_entity}
```

Example: `Product` ↔ `Tag` → `kdb_product_tag`

### Schema

```sql
CREATE TABLE kdb_product_tag (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    product_id UUID NOT NULL,      -- FK to source
    tag_id UUID NOT NULL,          -- FK to target
    created_at TIMESTAMPTZ DEFAULT NOW(),
    created_by VARCHAR(255),
    
    UNIQUE(product_id, tag_id)     -- Prevent duplicates
);
```

### Storage Mode Behavior

| Source | Target | Junction Storage |
|--------|--------|------------------|
| shared | shared | Shared (kdb_records with special entity) |
| shared | dedicated | Dedicated junction table |
| dedicated | shared | Dedicated junction table |
| dedicated | dedicated | Dedicated junction table with FKs |

**Rationale**: If either side is dedicated, use a real table for better JOIN performance. Only use shared storage when both sides are shared.

## API Design

### Creating Many-to-Many Relationships

```python
db.create_entity("Product", fields=[...])
db.create_entity("Tag", fields=[...])

# Create the relationship
db._schema_engine.add_relationship(
    source_entity_name="Product",
    name="tags",
    target_entity_name="Tag",
    relationship_type="many_to_many",
    inverse_name="products",  # Optional: creates inverse on Tag
    on_delete="CASCADE",      # When Product deleted, remove links
)
```

This auto-generates:
- Junction table `kdb_product_tag`
- `JunctionTable` metadata record
- Inverse relationship (if `inverse_name` provided)

### Managing Links

```python
product = db.entity("Product")
product_id = product.insert({"name": "Widget"})

# Link operations
product.link("tags", product_id, tag_id)
product.unlink("tags", product_id, tag_id)
product.unlink_all("tags", product_id)  # Remove all tags

# Bulk operations
product.link_many("tags", product_id, [tag_id_1, tag_id_2, tag_id_3])
product.unlink_many("tags", product_id, [tag_id_1, tag_id_2])

# Query links (returns list of IDs)
tag_ids = product.get_linked("tags", product_id)
```

### Querying via SQL

Agents query many-to-many via SQL with schema context hints:

```python
context = db.get_schema_context(entities=["Product", "Tag"])
# Context includes junction table info and JOIN patterns

# Example query (generated by agent)
sql = """
SELECT p.id, p.data->>'name' as product_name, 
       t.data->>'name' as tag_name
FROM kdb_records p
JOIN kdb_product_tag pt ON pt.product_id = p.id::text
JOIN kdb_records t ON t.id::text = pt.tag_id
WHERE p.entity_id = :product_entity_id
  AND t.entity_id = :tag_entity_id
  AND p.is_deleted = false
  AND t.is_deleted = false
"""
```

## Implementation

### 1. Junction Table Creation

When `add_relationship()` is called with `relationship_type="many_to_many"`:

```python
def _create_junction_table(
    self,
    source_entity: EntityDefinition,
    target_entity: EntityDefinition,
    relationship: RelationshipDefinition,
) -> JunctionTable:
    """Create junction table for many-to-many relationship."""
    
    # Determine table name
    table_name = f"kdb_{source_entity.name.lower()}_{target_entity.name.lower()}"
    
    # Determine if we need a real table or shared storage
    use_dedicated = (
        source_entity.storage_mode == "dedicated" or
        target_entity.storage_mode == "dedicated"
    )
    
    if use_dedicated:
        # Create actual table
        self._create_dedicated_junction_table(
            table_name=table_name,
            source_entity=source_entity,
            target_entity=target_entity,
        )
    else:
        # Use shared storage - create as pseudo-entity
        self._create_shared_junction_entity(
            table_name=table_name,
            source_entity=source_entity,
            target_entity=target_entity,
        )
    
    # Record junction metadata
    junction = JunctionTable(
        relationship_id=relationship.id,
        table_name=table_name,
        source_fk_column=f"{source_entity.name.lower()}_id",
        target_fk_column=f"{target_entity.name.lower()}_id",
    )
    # ... save junction
```

### 2. Link Operations (Entity class)

```python
class Entity:
    def link(
        self,
        relationship_name: str,
        record_id: str,
        target_id: str,
        created_by: str | None = None,
    ) -> bool:
        """Add a link in a many-to-many relationship."""
        rel = self._get_relationship(relationship_name)
        if rel.relationship_type != "many_to_many":
            raise ValueError(f"'{relationship_name}' is not a many-to-many relationship")
        
        junction = rel.junction_table
        # Insert into junction table (or shared entity)
        ...
    
    def unlink(
        self,
        relationship_name: str,
        record_id: str,
        target_id: str,
    ) -> bool:
        """Remove a link in a many-to-many relationship."""
        ...
    
    def get_linked(
        self,
        relationship_name: str,
        record_id: str,
    ) -> list[str]:
        """Get all linked record IDs."""
        ...
```

### 3. Cascade Integration

When a record is deleted (via Spec 006 cascade logic):

```python
# In Entity.delete() cascade handling:
for rel in incoming_rels:
    if rel.relationship_type == "many_to_many":
        # Delete junction entries, not the related records
        self._delete_junction_entries(rel, record_id)
```

Many-to-many CASCADE means:
- Delete junction entries (the links)
- NOT delete the records on the other side

### 4. Schema Context Updates

Add junction table info to context:

```python
{
    "relationships": [
        {
            "name": "tags",
            "source_entity": "Product",
            "target_entity": "Tag",
            "relationship_type": "many_to_many",
            "junction_table": "kdb_product_tag",
            "join_hint": """
                JOIN kdb_product_tag pt ON pt.product_id = p.id::text
                JOIN kdb_records t ON t.id::text = pt.tag_id
            """,
        }
    ]
}
```

## Edge Cases

1. **Self-referential many-to-many**: User ↔ User (followers)
   - Junction: `kdb_user_user` with `user_id` and `follows_id`
   
2. **Duplicate links**: Prevented by UNIQUE constraint on junction

3. **Orphaned links**: When one side is deleted, junction entries are cleaned up

4. **Materialization**: When source or target is materialized, migrate junction to dedicated

5. **Link metadata**: Junction tables could have extra columns (e.g., `role` in User ↔ Project)
   - Future enhancement: `link(... metadata={...})`

## Testing

1. Create many-to-many relationship (both shared)
2. Create many-to-many relationship (one dedicated)
3. Create many-to-many relationship (both dedicated)
4. link() adds junction entry
5. unlink() removes junction entry
6. unlink_all() clears all links for record
7. link_many() bulk add
8. unlink_many() bulk remove
9. get_linked() returns correct IDs
10. Duplicate link is idempotent (or raises)
11. Delete source → cascade removes junction entries
12. Delete target → cascade removes junction entries
13. Self-referential relationship works
14. Schema context includes junction info
15. SQL JOINs work correctly

## Migration Path

1. Update `add_relationship()` to handle `many_to_many` type
2. Implement `_create_junction_table()` in SchemaEngine
3. Add `link()`, `unlink()`, `get_linked()` to Entity
4. Add bulk operations `link_many()`, `unlink_many()`
5. Update cascade logic for many-to-many
6. Update `SchemaContextBuilder` with junction patterns
7. Add tests
8. Update BACKLOG.md → DONE.md

## Out of Scope

- Link metadata (extra columns on junction)
- Ordered relationships (position column)
- Polymorphic relationships
- Graph traversal queries
