# Clew: High-Level Specification

**Version:** 0.1 (Indie V1)
**Architecture:** Docker Native (Single Node)
**Goal:** Provide AI Agents with infinite, relationship-aware context via a unified graph+vector engine.

## 1. Core Philosophy

> "Vectors find similarity; Graphs find structure."

The Clew (Clew) is a self-hostable service that fuses these two paradigms to allow agents to answer complex questions like *"Why did the login API break after the last commit?"*—a query impossible with vector search alone.

We prioritize **Simplicity of Operations** (Docker Compose) over **Scale at all costs** (Kubernetes). Agentic orchestration handles the complexity of coding, permitting us to build sophisticated internal logic without over-engineering the deployment.

## 2. System Architecture

The entire system runs as a set of containers defined in a single `docker-compose.yml`.

```mermaid
graph TD
    User[User / AI Agent] -->|MCP Protocol| MCP[MCP Server]
    
    subgraph "Clew Engine (Data Plane)"
        MCP -->|REST| API[Context API (FastAPI)]
        
        API -->|Read| Qdrant[(Qdrant: Vectors)]
        API -->|Read| Neo4j[(Neo4j: Graph)]
        
        Ingest[clew-ingestion Library] -->|Write| Qdrant
        Ingest -->|Write| Neo4j
    end
    
    subgraph "Clew Platform (Control Plane)"
        Worker[Platform Worker] -->|Executes| Ingest
        API_Platform[Platform API] -.->|Manages| Worker
    end
    
    Source[GitHub / Jira / Docs] -.->|Poll/Webhook| Worker
```

### 2.1 The Data Plane (The "Brain")
*   **Vector Store: Qdrant**
    *   *Role:* Semantic search ("Find code about authorization").
    *   *Why:* Native filtering, high performance, single binary.
*   **Graph Store: Neo4j**
    *   *Role:* Structural relationships (`:PR` -> `:LINKS_TO` -> `:ISSUE`).
    *   *Why:* Cypher query language offers the most expressive graph traversal for RAG expansion.

### 2.2 The Compute Plane
*   **`context-api` (FastAPI)**: The central nervous system.
    *   Exposes endpoints for `search` (Hybrid), `graph`, and `ingest`.
    *   Handles "Graph Expansion": Query -> Vector Search -> Node IDs -> Graph Traversal -> Rerank.
*   **`context-ingest` (Python)**: The background worker.
    *   Uses *Agentic Orchestration* patterns to intelligently crawl repos.
    *   Example: Doesn't just "index files"; parses ASTs to link `Function` nodes to `File` nodes.
*   **`mcp-server`**: The Agent Interface.
    *   Implements the Model Context Protocol.
    *   Tools: `search_context`, `browse_graph`, `read_file`.

## 3. Data Model Strategy

We use a **Unified Graph-Vector Model**. Every entity exists in both databases, linked by a generic `id`.

*   **In Qdrant**: `Point { id: "pr-123", vector: [...], payload: { type: "pr", title: "Fix login" } }`
*   **In Neo4j**: `Node(:PR { id: "pr-123" }) -[:LINKS_TO]-> Node(:Issue { id: "issue-456" })`

### Key Entities
1.  **Code**: `Repository`, `Directory`, `File`, `Function`, `Class`.
2.  **Work**: `Issue`, `PullRequest`, `Comment`, `Commit`.
3.  **Knowledge**: `Document` (Markdown/Wiki).

## 4. Security (Indie V1)

We assume a **Single-Tenant / Personal Instance** model for V1 to reduce scope.
*   **Auth**: Simple Bearer Token (API Key) defined in `.env`.
*   **Isolation**: The instance indexes *your* tokens.
*   **Future Proofing**: The internal data model supports `acl_groups` from day one, so we can turn on multi-tenancy later without a rewrite.

## 5. Development Roadmap

1.  **Infrastructure**: `docker-compose up` brings up the "Brain" (Qdrant + Neo4j).
2.  **Ingestion Skeleton**: A script that takes a GitHub Repo URL and populates the graph.
3.  **API Core**: Endpoint `/search` that hits Qdrant.
4.  **Graph Logic**: Implement "Expansion" (The secret sauce).
5.  **MCP Layer**: Expose it all to Claude.
