# Embeddings and Thinking Tools

This document describes the specialized tools available in `chroma-mcp-server` for managing embeddings and implementing sequential thinking patterns using ChromaDB as the underlying vector store.

## Embedding Function Management

The server utilizes embedding functions provided by ChromaDB (or compatible custom functions) to convert text into vector embeddings. The choice of embedding function impacts performance, cost (for API-based functions), and the quality of semantic search results.

**Key Concepts:**

* **Default Function:** The server starts with a default embedding function (e.g., `default` or `fast` using ONNX models, or another specified via `--embedding-function` at startup). See [Getting Started](./getting_started.md#choosing-an-embedding-function) for available options.
* **Runtime Selection:** While not directly supported by a dedicated tool in this reference, the underlying ChromaDB client can be configured with different embedding functions at startup (e.g., `openai`, `cohere`, `google`, `bedrock`, `ollama`). The server primarily uses the function it was initialized with.

**Related Tools:**

* `test_chroma_get_embedding_function`: Retrieves the name of the currently configured embedding function for the server.

## Sequential Thinking Tools

These tools leverage ChromaDB to store and retrieve sequences of thoughts, enabling structured reasoning and context management across multiple interactions. This is particularly useful for complex problem-solving or maintaining state in multi-turn conversations.

**Core Idea:**

The central concept is the **Sequential Thought**. This represents a single, discrete step or piece of information captured during a development session. Examples include:

* A specific user instruction (`Implement function X`)
* An AI's proposed plan (`1. Read file Y. 2. Modify class Z.`)
* A code snippet generated by the AI
* An error message encountered during testing
* A summary of a successful refactoring
* A key decision made (`Decided to use library A instead of B`)

These thoughts are stored chronologically within a **Thinking Session**, identified by a unique `session_id`.

## How Embeddings Power the Working Memory

Each recorded thought is automatically embedded into a vector representation by the configured embedding function (e.g., `default`, `accurate`, `openai`). These embeddings capture the semantic meaning of the thought's content.

* **Storage:** Thoughts and their embeddings are stored in a dedicated ChromaDB collection (`sequential_thoughts_v1`).
* **Retrieval:** When an AI assistant needs context from the past, it doesn't rely on simple keyword matching. Instead, it uses the `find_similar_thoughts` tool. This tool takes a query (e.g., "What was the plan for refactoring the database module?") and performs a **semantic search** against the stored thought embeddings.
* **Semantic Search:** ChromaDB finds thoughts whose embeddings are *semantically closest* to the query's embedding, even if the exact wording differs. This allows retrieving relevant past thoughts, plans, or code snippets based on their meaning, not just keywords.

## Integration with Enhanced Context Capture and Bidirectional Linking

The Thinking Utilities complement the enhanced context capture system and bidirectional linking:

* **Connection to Chat History:** Thinking sessions can reference discussions recorded in `chat_history_v1`, creating connections between explicit reasoning steps and AI-developer interactions.
* **Connection to Code Changes:** Thoughts can link to specific code chunks in `codebase_v1`, allowing AI to connect conceptual reasoning with concrete implementations.
* **Context-Rich Reasoning:** Recorded thoughts can utilize the enhanced context metadata (code diffs, tool sequences, confidence scores) from chat history entries to inform their reasoning.
* **Multi-Collection Queries:** When searching for context, AI can leverage both working memory (`sequential_thoughts_v1`) and the rich contextual information stored in `chat_history_v1` and `codebase_v1`.

This integration enables a more comprehensive understanding of the development process, where explicit reasoning steps can be linked to actual code changes and conversations:

```mermaid
%%{init: {'theme': 'dark'}}%%
flowchart TD
    A[Thinking Session] -- references --> B[Chat History Entry]
    B -- modifies --> C[Code Chunk]
    A -- reasons about --> C
    D[AI Assistant] -- queries --> A
    D -- queries --> B
    D -- queries --> C
    
    subgraph "ChromaDB Collections"
        A["sequential_thoughts_v1"]
        B["chat_history_v1"]
        C["codebase_v1"]
    end
    
    style A fill:#66BB6A,stroke:#E6E6E6,stroke-width:1px
    style B fill:#66BB6A,stroke:#E6E6E6,stroke-width:1px
    style C fill:#66BB6A,stroke:#E6E6E6,stroke-width:1px
    style D fill:#FFCA28,stroke:#E6E6E6,stroke-width:1px,color:#333333
```

## Embedding Models

The Chroma MCP Server relies on embedding models to represent text data (documents, thoughts) as numerical vectors. These vectors capture the semantic meaning of the text, enabling similarity searches.

### Selecting an Embedding Model

You can choose the embedding model the server uses via:

1. **Command-Line Argument:** Use the `--embedding-function` argument when starting the server. For example:

    ```bash
    uvx chroma-mcp-server --embedding-function=accurate
    ```

2. **Environment Variable:** Set the `CHROMA_EMBEDDING_FUNCTION` environment variable.

    ```bash
    export CHROMA_EMBEDDING_FUNCTION=openai
    uvx chroma-mcp-server
    ```

If both are set, the command-line argument takes precedence.

### Available Models

* `default`: Uses a Sentence Transformers model (`all-MiniLM-L6-v2`) locally. Good balance of speed and accuracy, runs entirely on your machine.
* `accurate`: Uses a larger Sentence Transformers model (`all-mpnet-base-v2`) locally. Potentially more accurate but slower and requires more resources.
* `openai`: Uses OpenAI's `text-embedding-ada-002` model (requires an `OPENAI_API_KEY` environment variable).
* `cohere`: Uses Cohere's embedding models (requires a `COHERE_API_KEY` environment variable).

**Important:** Using non-local models like OpenAI or Cohere requires setting the corresponding API key environment variable.

## Sequential Thinking

```mermaid
%%{init: {'theme': 'dark'}}%%
flowchart TD
    A[User/AI Interaction] --> B["Record Thought via MCP"]
    B --> C{"Embedding Function"}
    C --> D[(ChromaDB Collection:\nsequential_thoughts_v1)]
    B --> D

    E[AI Needs Context] --> F["Query via MCP for\nrelevant past info"]
    F --> C
    F --> G["Query ChromaDB"]
    C --> G
    G --> D
    D --> G
    G --> F
    F --> E
    
    style A fill:#42A5F5,stroke:#E6E6E6,stroke-width:1px
    style B fill:#7E57C2,stroke:#E6E6E6,stroke-width:1px
    style C fill:#26A69A,stroke:#E6E6E6,stroke-width:1px
    style D fill:#66BB6A,stroke:#E6E6E6,stroke-width:1px
    style E fill:#FFCA28,stroke:#E6E6E6,stroke-width:1px,color:#333333
    style F fill:#26A69A,stroke:#E6E6E6,stroke-width:1px
    style G fill:#AB47BC,stroke:#E6E6E6,stroke-width:1px
```

## Memory Integration for AI Assistants

To use these thinking tools effectively, AI assistants should follow the memory integration guidelines defined in the `memory-integration-rule`. This rule helps AI assistants to:

1. Check for previous thought threads and ensure continuity
2. Structure complex reasoning across multiple messages
3. Find and reference semantically similar past thoughts
4. Synthesize knowledge across multiple thinking sessions
5. Connect conceptual discussions with concrete code implementations
6. Document conceptual evolution with clear markers

For more details on how AI assistants should use these tools, see the [Memory Integration Rules](../rules/memory-integration-rules.md).

## The Thinking Tools (`thinking_tools.py`)

These tools provide the interface to manage the working memory.

### 1. Recording Thoughts

* **`#chroma_sequential_thinking`**: Records a single thought.
  * **Key Params:** `thought` (content), `thought_number`, `total_thoughts`, `session_id` (optional, generated if empty).
  * **Functionality:** Embeds the `thought` content and stores it along with metadata (session, sequence number, timestamp) in the `sequential_thoughts_v1` collection. Returns the `session_id` and generated `thought_id`.
* **Branches:** Allows creating alternative thought sequences within a session using `branch_id` and `branch_from_thought`.

### 2. Retrieving Thoughts (Semantic Search)

* **`#chroma_find_similar_thoughts`**: Finds thoughts semantically similar to a query.
  * **Key Params:** `query` (text to search for), `session_id` (optional filter), `n_results`, `threshold` (similarity cutoff), `include_branches`.
  * **Functionality:** Embeds the `query`, searches the `sequential_thoughts_v1` collection for thoughts with similar embeddings (within the optional session and branch constraints), filters by the similarity `threshold`, and returns the matching thoughts (content, metadata, similarity score).

### 3. Retrieving Sessions

* **`#chroma_get_session_summary`**: Retrieves all thoughts recorded for a specific `session_id`.
  * **Key Params:** `session_id`, `include_branches`.
  * **Functionality:** Fetches all thoughts matching the `session_id` (optionally filtering branches), sorts them by `thought_number`, and returns the complete sequence.
* **`#chroma_find_similar_sessions`**: Finds sessions whose overall content is semantically similar to a query.
  * **Key Params:** `query`, `n_results`, `threshold`.
  * **Functionality:** (Requires pre-computation or a separate `thinking_sessions` collection). Embeds the `query` and searches for sessions (represented by aggregated embeddings or summaries stored separately) that are semantically similar. Returns matching `session_id`s and similarity scores.

```mermaid
%%{init: {'theme': 'dark'}}%%
sequenceDiagram
    participant Dev as Developer/AI Assistant
    participant MCP as Chroma MCP Server
    participant Chroma as ChromaDB

    Dev->>+MCP: Record Thought 1 (Session A, #1)
    MCP->>+Chroma: Embed "Thought 1"
    Chroma-->>-MCP: Embedding Vector 1
    MCP->>+Chroma: Add Document (ID: A_1, Embed: Vec 1, Meta: {session:A, #:1})
    Chroma-->>-MCP: OK
    MCP-->>-Dev: Success (ID: A_1)

    Dev->>+MCP: Record Thought 2 (Session A, #2)
    MCP->>+Chroma: Embed "Thought 2"
    Chroma-->>-MCP: Embedding Vector 2
    MCP->>+Chroma: Add Document (ID: A_2, Embed: Vec 2, Meta: {session:A, #:2})
    Chroma-->>-MCP: OK
    MCP-->>-Dev: Success (ID: A_2)

    Dev->>+MCP: Find Similar Thoughts (Query: "similar to thought 1?")
    MCP->>+Chroma: Embed "similar to thought 1?"
    Chroma-->>-MCP: Query Embedding Vector Q
    MCP->>+Chroma: Query Collection (Embed: Vec Q, n_results=5)
    Chroma-->>-MCP: Results (Thought 1: dist 0.1, Thought 2: dist 0.8)
    MCP-->>-Dev: Similar Thoughts (Thought 1, score 0.9)

    Dev->>+MCP: Get Session Summary (Session A)
    MCP->>+Chroma: Get Documents (Where: {session_id: A})
    Chroma-->>-MCP: Documents (Thought 1, Thought 2)
    MCP-->>-Dev: Session Summary ([Thought 1, Thought 2])
    
    style Dev fill:#42A5F5,stroke:#E6E6E6,stroke-width:1px
    style MCP fill:#26A69A,stroke:#E6E6E6,stroke-width:1px
    style Chroma fill:#66BB6A,stroke:#E6E6E6,stroke-width:1px
```

## Use Cases

* **Context Recall:** "What did we decide about the error handling for function X yesterday?" -> `find_similar_thoughts`
* **Plan Retrieval:** "Show me the plan we made for implementing feature Y." -> `find_similar_thoughts` or `get_session_summary`
* **Code Snippet Search:** "Find the code snippet where we handled database connections." -> `find_similar_thoughts`
* **Session Review:** "Summarize the key steps taken in session Z." -> `get_session_summary`
* **Cross-Session Learning:** "Find other sessions where we discussed authentication." -> `find_similar_sessions` (if implemented)
* **Reasoning with Enhanced Context:** "Analyze this code pattern, considering the context from our previous discussions and code changes." -> Combined approach using `find_similar_thoughts` and queries across `chat_history_v1` and `codebase_v1`

By using these tools, AI assistants can maintain a more continuous understanding of the development process, leading to more relevant suggestions, fewer repeated questions, and a smoother workflow for the developer.
