# Module Wrapper

A modular system for Python module introspection with Qdrant vector database integration. ModuleWrapper extracts components from Python modules, generates embeddings, and enables semantic search with DSL notation support.

## Features

- **Module Introspection** - Automatically extract classes, functions, and relationships from Python modules
- **Multi-Vector Embeddings** - ColBERT (128D) + MiniLM (384D) for precise semantic search
- **DSL Notation** - Unicode symbols for compact structure representation (`§[δ, Ƀ[ᵬ×2]]`)
- **DAG Relationships** - NetworkX-based containment graph with traversal queries
- **Tiered Caching** - L1 (memory) → L2 (pickle) → L3 (resolve) for fast retrieval
- **Instance Patterns** - Store and vary successful component usage patterns

## Quick Start

```python
from adapters.module_wrapper import ModuleWrapper

# Initialize with a module
wrapper = ModuleWrapper("card_framework.v2", auto_initialize=True)

# Search for components
results = wrapper.search("button with click action")

# Get component by path
Button = wrapper.get_component_by_path("card_framework.v2.widgets.button.Button")

# Use DSL-aware search
results = wrapper.search_by_dsl("§[δ, Ƀ[ᵬ×2]] Build a status card")
```

## Architecture

ModuleWrapper uses a mixin composition pattern for modularity:

```mermaid
classDiagram
    class ModuleWrapper {
        +search(query)
        +get_component_by_path(path)
        +search_hybrid(description)
    }

    class ModuleWrapperBase {
        +module_name: str
        +components: Dict
        +_initialized: bool
    }

    class QdrantMixin {
        +client: QdrantClient
        +collection_name: str
        +_initialize_qdrant()
    }

    class EmbeddingMixin {
        +embedder: TextEmbedding
        +colbert_embedder: LateInteraction
        +_embed_with_colbert()
        +_embed_with_minilm()
    }

    class IndexingMixin {
        +_index_module_components()
        +_extract_component_info()
    }

    class SearchMixin {
        +search()
        +search_named_vector()
        +search_hybrid()
        +search_by_dsl()
    }

    class RelationshipsMixin {
        +relationships: Dict
        +extract_relationships()
    }

    class SymbolsMixin {
        +symbol_mapping: Dict
        +parse_dsl_to_components()
        +build_dsl_from_paths()
    }

    class PipelineMixin {
        +run_ingestion_pipeline()
        +_build_payload()
    }

    class GraphMixin {
        +build_relationship_graph()
        +get_descendants()
        +get_all_paths()
    }

    class CacheMixin {
        +cache_pattern()
        +get_cached_class()
        +get_cached_entry()
    }

    class InstancePatternMixin {
        +store_instance_pattern()
        +generate_variations()
        +get_cached_variation()
    }

    ModuleWrapper --|> QdrantMixin
    ModuleWrapper --|> EmbeddingMixin
    ModuleWrapper --|> IndexingMixin
    ModuleWrapper --|> SearchMixin
    ModuleWrapper --|> RelationshipsMixin
    ModuleWrapper --|> SymbolsMixin
    ModuleWrapper --|> PipelineMixin
    ModuleWrapper --|> GraphMixin
    ModuleWrapper --|> CacheMixin
    ModuleWrapper --|> InstancePatternMixin
    ModuleWrapper --|> ModuleWrapperBase
```

## Multi-Vector Embedding Schema

The collection uses three named vectors for different search strategies:

```mermaid
graph TB
    subgraph "Collection Schema (Named Vectors)"
        subgraph "Named Vectors"
            C["components<br/>ColBERT 128D"]
            I["inputs<br/>ColBERT 128D"]
            R["relationships<br/>MiniLM 384D"]
        end

        subgraph "Payload Fields"
            P1["name, type, full_path"]
            P2["symbol, docstring"]
            P3["parent_paths, instance_params"]
            P4["dsl_notation, feedback"]
        end
    end

    subgraph "Search Flow"
        Q["Query: '§[δ, Ƀ] status card'"]
        Q --> DSL["DSL Extraction"]
        DSL --> CE["ColBERT Embed"]
        DSL --> ME["MiniLM Embed"]
        CE --> C
        CE --> I
        ME --> R
    end

    subgraph "Result Fusion"
        C --> RRF["RRF Fusion"]
        I --> RRF
        R --> RRF
        RRF --> Results["Ranked Results"]
    end
```

| Vector | Model | Dimension | Purpose |
|--------|-------|-----------|---------|
| `components` | ColBERT | 128D | Component identity matching |
| `inputs` | ColBERT | 128D | Parameter/content matching |
| `relationships` | MiniLM | 384D | Structural pattern matching |

## DSL Notation

Compact Unicode symbols represent component structures:

| Symbol | Component | Example |
|--------|-----------|---------|
| `§` | Section | `§[...]` |
| `δ` | DecoratedText | `§[δ]` |
| `Ƀ` | ButtonList | `§[Ƀ[ᵬ]]` |
| `ᵬ` | Button | `Ƀ[ᵬ×2]` |
| `ℊ` | Grid | `§[ℊ[ǵ×4]]` |
| `ǵ` | GridItem | `ℊ[ǵ×4]` |
| `Ɨ` | Icon | `ᵬ[Ɨ]` |
| `ɪ` | Image | `§[ɪ]` |

**Syntax:** `SYMBOL[child, child×N, ...]`

**Examples:**
- `§[δ]` - Section with DecoratedText
- `§[δ, Ƀ[ᵬ×2]]` - Section with text and 2 buttons
- `§[ℊ[ǵ×4]]` - Section with 4-item grid

```python
# Parse DSL to components
parsed = wrapper.parse_dsl_to_components("§[δ, Ƀ[ᵬ×2]]")
# {'components': [{'name': 'Section', 'symbol': '§'}, ...], 'is_valid': True}

# Build DSL from paths
dsl = wrapper.build_dsl_from_paths(["Section", "DecoratedText", "Button"])
# "§[δ, ᵬ] | Section DecoratedText Button"
```

## Search Flow

```mermaid
sequenceDiagram
    participant User
    participant SearchMixin
    participant DSLParser
    participant EmbeddingMixin
    participant Qdrant

    User->>SearchMixin: search_hybrid("§[δ] status card")
    SearchMixin->>DSLParser: extract_dsl_from_text()
    DSLParser-->>SearchMixin: {dsl: "§[δ]", description: "status card"}

    par Parallel Embedding
        SearchMixin->>EmbeddingMixin: _embed_with_colbert(query)
        SearchMixin->>EmbeddingMixin: _embed_with_minilm(query)
    end

    EmbeddingMixin-->>SearchMixin: ColBERT vectors (128D)
    EmbeddingMixin-->>SearchMixin: MiniLM vector (384D)

    SearchMixin->>Qdrant: Prefetch queries (3 vectors)
    Qdrant-->>SearchMixin: Raw results

    SearchMixin->>SearchMixin: RRF Fusion
    SearchMixin-->>User: (classes, patterns, relationships)
```

## DAG Relationships

Component containment forms a directed acyclic graph:

```mermaid
graph TD
    Card["Card"]
    Section["Section §"]
    DT["DecoratedText δ"]
    TP["TextParagraph"]
    BL["ButtonList Ƀ"]
    Button["Button ᵬ"]
    Icon["Icon Ɨ"]
    Image["Image ɪ"]
    Grid["Grid ℊ"]
    GI["GridItem ǵ"]

    Card --> Section
    Section --> DT
    Section --> TP
    Section --> BL
    Section --> Image
    Section --> Grid

    DT --> Icon
    DT --> Button

    BL --> Button

    Button --> Icon

    Grid --> GI
    GI --> Image
    GI --> Icon
```

```python
# Build the relationship graph
wrapper.build_relationship_graph()

# Get all descendants
widgets = wrapper.get_descendants("Section", depth=2)
# ['DecoratedText', 'ButtonList', 'Image', 'Grid', 'Button', 'Icon', ...]

# Find paths between components
paths = wrapper.get_all_paths("Section", "Icon")
# [['Section', 'DecoratedText', 'Icon'], ['Section', 'ButtonList', 'Button', 'Icon']]

# Check containment
wrapper.can_contain("Section", "Icon")  # True (via DecoratedText or Button)
```

## Caching Architecture

Three-tier cache for fast component retrieval:

```mermaid
flowchart LR
    subgraph "Cache Tiers"
        L1["L1: Memory<br/>(LRU, 100 entries)"]
        L2["L2: Pickle<br/>(.component_cache/)"]
        L3["L3: Resolve<br/>(ModuleWrapper)"]
    end

    Request["get_cached_class('Button')"]

    Request --> L1
    L1 -->|miss| L2
    L2 -->|miss| L3
    L3 -->|resolve| Classes["Component Classes"]
    Classes -->|populate| L2
    L2 -->|populate| L1
    L1 -->|hit| Response["Return Class"]
```

```python
# Cache a pattern
entry = wrapper.cache_pattern(
    key="status_card_v1",
    component_paths=["Section", "DecoratedText", "ButtonList"],
    instance_params={"text": "Status: Online"},
)

# Fast class retrieval (uses L1/L2/L3)
Section = wrapper.get_cached_class("Section")
Button = wrapper.get_cached_class("Button")

# Get cached entry
entry = wrapper.get_cached_entry("status_card_v1")
```

## Quick Reference

| Mixin | Purpose | Key Methods |
|-------|---------|-------------|
| `QdrantMixin` | Qdrant client management | `_initialize_qdrant()`, `_ensure_collection()` |
| `EmbeddingMixin` | MiniLM + ColBERT embeddings | `_embed_with_colbert()`, `_embed_with_minilm()` |
| `IndexingMixin` | Component extraction & indexing | `_index_module_components()` |
| `SearchMixin` | Multi-vector semantic search | `search()`, `search_hybrid()`, `search_by_dsl()` |
| `RelationshipsMixin` | Type-hint relationship extraction | `extract_relationships()` |
| `SymbolsMixin` | DSL symbol generation & parsing | `parse_dsl_to_components()`, `build_dsl_from_paths()` |
| `PipelineMixin` | Collection ingestion pipeline | `run_ingestion_pipeline()` |
| `GraphMixin` | NetworkX DAG operations | `get_descendants()`, `get_all_paths()` |
| `CacheMixin` | Tiered component caching | `cache_pattern()`, `get_cached_class()` |
| `InstancePatternMixin` | Pattern storage & variation | `store_instance_pattern()`, `generate_variations()` |

## Detailed Documentation

- [Embeddings](docs/embeddings.md) - Multi-vector embedding strategies
- [DSL Notation](docs/dsl-notation.md) - Symbol syntax and parsing
- [Search Strategies](docs/search-strategies.md) - Search methods and vector selection
- [Relationships & DAG](docs/relationships-dag.md) - Graph structure and traversal
- [Caching](docs/caching.md) - Tiered cache system
- [Instance Patterns](docs/instance-patterns.md) - Pattern storage and variations
- [Types Reference](docs/types-reference.md) - Type aliases, protocols, and dataclasses

## Type System

The package includes a centralized type system (`types.py`) for consistency:

```python
from adapters.module_wrapper.types import (
    # Type Aliases
    Payload, SymbolMapping, ComponentPath, RelationshipDict,
    EmbeddingVector, MultiVector, DSLNotation,
    # Protocols
    Serializable, Timestamped, Validatable, HasSymbol, Embeddable,
    # Dataclasses
    SearchResult, ComponentInfo, RelationshipInfo,
    QdrantConfig, EmbeddingConfig, IndexingStats,
    # Constants
    COLBERT_DIM, MINILM_DIM, RELATIONSHIPS_DIM,
)

# Example: Use SearchResult for consistent API
results: List[SearchResult] = wrapper.search_hybrid("status card")
for r in results:
    print(f"{r.name} ({r.symbol}): {r.score:.3f}")
```

See [Types Reference](docs/types-reference.md) for complete documentation.

## Version

`2.0.0`
