# Story 1.8: `search-ids` Tool Implementation

Status: drafted

## Story

As an AI assistant,
I want to search for RabbitMQ operations using natural language queries,
so that I can discover relevant operations without knowing exact API endpoint names.

## Acceptance Criteria

1. **Tool Registration in MCP Server**
   - Tool `search-ids` registered in MCP server tools list
   - Tool definition includes: name="search-ids", description, input_schema (JSON Schema format)
   - Tool accessible via `tools/call` JSON-RPC method
   - Tool returns results via MCP content response format

2. **Query Processing**
   - Accepts parameter `query` (string, required) - natural language search query
   - Accepts parameter `threshold` (float, optional, default 0.7) - minimum similarity score
   - Accepts parameter `max_results` (int, optional, default 10) - maximum results to return
   - Query is validated: non-empty string, length ≥2 characters
   - Invalid queries return clear error: "Query must be at least 2 characters"

3. **Semantic Search Implementation**
   - Loads pre-computed embeddings from `data/embeddings.json` at server startup (cached in memory)
   - Generates query embedding using same model: `all-MiniLM-L6-v2` (sentence-transformers)
   - Computes cosine similarity between query vector and all operation vectors
   - Cosine similarity formula: dot(query_vec, op_vec) / (norm(query_vec) * norm(op_vec))
   - Uses numpy for vectorized similarity calculations (fast)
   - Filters results where similarity_score ≥ threshold
   - Orders results by similarity_score descending (highest first)
   - Returns top N results (limited by max_results parameter)

4. **Result Format**
   - Each result contains: operation_id, description, similarity_score, namespace
   - Results ordered by similarity_score descending
   - Example result: `{"operation_id": "queues.list", "description": "List all queues in vhost", "similarity_score": 0.89, "namespace": "queues"}`
   - Empty results (no scores ≥threshold) return empty list with suggestion: "Try broader terms or check spelling"
   - Results formatted as JSON array in MCP content response

5. **Performance Requirements**
   - Search completes in <100ms at p95 latency
   - Query embedding generation cached with LRU cache (max 100 queries)
   - Embeddings loaded into memory <500ms on server startup
   - Similarity calculations use numpy vectorized operations for speed
   - No disk I/O during search (all data in memory)

6. **Example Queries Work Correctly**
   - Query "list all queues" returns operations including: "queues.list", "queues.list_by_vhost" with scores ≥0.7
   - Query "delete exchange" returns "exchanges.delete" with score ≥0.7
   - Query "create binding" returns "bindings.create" with score ≥0.7
   - Query "gibberish xyz123" returns empty list with helpful suggestion

7. **Analytics Logging (Foundation for Innovation Validation)**
   - Log each search query with: query text, result count, top similarity score, timestamp, correlation_id
   - Log format: structured JSON for analysis
   - Logging enables future experiments validating semantic discovery innovation
   - No PII logged (queries are operational terms only)

## Tasks / Subtasks

- [ ] **Task 1: Create SemanticSearch Class** (AC: #3, #5)
  - [ ] Create module `src/mcp_server/semantic_search.py`
  - [ ] Define `SemanticSearch` class with `__init__(embeddings_path: Path, model_name: str)`
  - [ ] Load embeddings.json in __init__, store in memory (dict: operation_id → vector)
  - [ ] Load sentence-transformers model in __init__, cache model instance
  - [ ] Add method `search(query: str, threshold: float = 0.7, max_results: int = 10) -> List[SearchResult]`
  - [ ] Implement query embedding generation using model.encode()
  - [ ] Implement cosine similarity calculation with numpy
  - [ ] Filter and rank results by similarity score
  - [ ] Return list of SearchResult dataclass instances

- [ ] **Task 2: Implement Result Models** (AC: #4)
  - [ ] Create `src/mcp_server/models.py` (if not exists)
  - [ ] Define `SearchResult` dataclass with fields: operation_id, description, similarity_score, namespace
  - [ ] Add type hints for all fields (str, float)
  - [ ] Add JSON serialization method: `to_dict() -> Dict[str, Any]`
  - [ ] Add validation: similarity_score must be 0.0-1.0

- [ ] **Task 3: Implement Query Caching** (AC: #5)
  - [ ] Import functools.lru_cache
  - [ ] Create cached method `_encode_query_cached(query: str) -> np.ndarray`
  - [ ] Set cache size: maxsize=100 (100 most recent queries)
  - [ ] Log cache hits/misses for monitoring
  - [ ] Ensure cache key is normalized (lowercase, stripped)

- [ ] **Task 4: Create search-ids Tool Handler** (AC: #1, #2)
  - [ ] Create module `src/mcp_server/tools/search_ids.py`
  - [ ] Define async function `handle_search_ids(arguments: Dict[str, Any]) -> Dict[str, Any]`
  - [ ] Extract and validate parameters: query (required), threshold (default 0.7), max_results (default 10)
  - [ ] Validate query: non-empty, length ≥2 characters
  - [ ] Call SemanticSearch.search() with parameters
  - [ ] Format results as list of dicts
  - [ ] Handle empty results with suggestion message
  - [ ] Return MCP-compatible response format

- [ ] **Task 5: Register Tool in MCP Server** (AC: #1)
  - [ ] Open `src/mcp_server/server.py`
  - [ ] Import search_ids tool handler
  - [ ] Add tool definition to tools registry:
    - name: "search-ids"
    - description: "Search for RabbitMQ operations using natural language queries"
    - inputSchema: JSON Schema with query (string, required), threshold (number, optional), max_results (integer, optional)
  - [ ] Register handler in tool dispatcher: route "search-ids" → handle_search_ids()
  - [ ] Ensure tools/list returns search-ids in tool list

- [ ] **Task 6: Add Analytics Logging** (AC: #7)
  - [ ] Import structlog logger in search_ids.py
  - [ ] Log search query event: query_text, result_count, top_score, timestamp, correlation_id
  - [ ] Use structured logging format: logger.info("semantic_search", query=..., results=...)
  - [ ] Add correlation_id from request context
  - [ ] Ensure no PII logged (queries are operational terms)

- [ ] **Task 7: Add Unit Tests** (AC: #3, #4, #5, #6)
  - [ ] Create `tests/unit/test_semantic_search.py`
  - [ ] Test SemanticSearch initialization: loads embeddings, loads model
  - [ ] Test search with known queries: "list queues" → queues.list with score ≥0.7
  - [ ] Test threshold filtering: results all have score ≥threshold
  - [ ] Test result ordering: results sorted by score descending
  - [ ] Test max_results limit: returns at most N results
  - [ ] Test empty results: query with no matches returns empty list
  - [ ] Test query caching: cache hit reduces model.encode() calls
  - [ ] Mock sentence-transformers model for fast tests

- [ ] **Task 8: Add Integration Tests** (AC: #1, #2, #6)
  - [ ] Create `tests/integration/test_search_ids_tool.py`
  - [ ] Test tool registration: search-ids in tools/list
  - [ ] Test tool execution via MCP protocol: tools/call → search-ids
  - [ ] Test example queries: "list all queues", "delete exchange", "create binding"
  - [ ] Test invalid queries: empty string, single character
  - [ ] Test threshold parameter: different thresholds return different result counts
  - [ ] Test max_results parameter: limit respected
  - [ ] Verify MCP response format correctness

- [ ] **Task 9: Add Performance Tests** (AC: #5)
  - [ ] Create `tests/performance/test_search_ids_performance.py`
  - [ ] Benchmark search latency: repeat search 100 times, measure p50/p95/p99
  - [ ] Validate p95 latency <100ms
  - [ ] Benchmark embedding load time: <500ms
  - [ ] Benchmark with/without cache: cache improves performance
  - [ ] Use pytest-benchmark for statistical analysis
  - [ ] Test on reference hardware (GitHub Actions runner)

- [ ] **Task 10: Documentation and Examples** (AC: #1, #2, #6)
  - [ ] Update README.md with search-ids tool description
  - [ ] Add usage examples: basic search, threshold tuning, max_results
  - [ ] Document query best practices: use action verbs, specific terms
  - [ ] Add troubleshooting: zero results → try broader terms
  - [ ] Document analytics logging for innovation validation
  - [ ] Add code examples for Python MCP client usage

## Dev Notes

### Architecture Patterns and Constraints

**Semantic Search Architecture (ADR-002, ADR-004):**
- Implements 3-tool semantic discovery pattern that solves MCP tool explosion problem [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#System-Architecture-Alignment]
- Pre-computed embeddings loaded at server startup from data/embeddings.json (132 operations, 384 dimensions) [Source: docs-bmad/sprint-artifacts/1-6-semantic-embeddings-generation.md#Completion-Notes]
- Cosine similarity with threshold=0.7 filters relevant results [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#AC-8]
- Query embeddings cached (LRU maxsize=100) to avoid redundant model inference [Source: docs-bmad/epics/epic-1-foundation-mcp-protocol.md - Story 1.8 AC]
- Numpy vectorized operations ensure <100ms p95 latency [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#Performance]

**MCP Protocol Integration (ADR-001):**
- Tool registered with JSON-RPC 2.0 method: tools/call → search-ids [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#AC-7]
- Input schema follows JSON Schema format for parameter validation [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#APIs-and-Interfaces]
- Response format: MCP content with text/json containing results array [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#MCP-Protocol-Interface]
- Error responses use JSON-RPC error codes: -32602 for invalid params [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#Non-Functional-Requirements]

**Type Safety and Performance Requirements:**
- Type hints required for all function signatures (mypy --strict compliance) [Source: docs-bmad/architecture/implementation-patterns.md#Type-Safety]
- Use numpy.typing.NDArray[np.float32] for embedding vectors [Source: docs-bmad/sprint-artifacts/1-6-semantic-embeddings-generation.md#Senior-Developer-Review]
- Constants: DEFAULT_THRESHOLD = 0.7, DEFAULT_MAX_RESULTS = 10, CACHE_SIZE = 100 [Source: docs-bmad/architecture/implementation-patterns.md#Constants]
- Async/await pattern for all tool handlers [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#Architectural-Constraints-Followed]
- Import order: stdlib → third-party → local modules [Source: docs-bmad/architecture/implementation-patterns.md#Import-Order]
- Use Pydantic for all input validation (JSON Schema compliance) [Source: docs-bmad/architecture/implementation-patterns.md#Validation]
- Error messages must include actionable remediation steps [Source: docs-bmad/architecture/implementation-patterns.md#Error-Messages]

**Code Quality Standards:**
- Follow PEP 8 style guide (enforced by black formatter)
- Maximum line length: 100 characters (configured in pyproject.toml)
- Docstrings required for all public classes and methods (Google style)
- No unused imports (enforced by ruff linter)
- No bare except clauses (must catch specific exceptions)
- Use f-strings for string formatting (not % or .format())
- Avoid mutable default arguments (use None and initialize in function body)

### Project Structure Notes

[Source: docs-bmad/architecture/project-structure.md]

**Tool Implementation Location:**
- Tool handlers in `src/mcp_server/tools/` directory (NOT `src/rabbitmq_mcp_server/mcp_server/tools/`)
- Semantic search module in `src/mcp_server/semantic_search.py` (shared utility)
- Models in `src/mcp_server/models.py` for data structures (SearchResult)
- Tests in `tests/unit/test_semantic_search.py` and `tests/integration/test_search_ids_tool.py`
- Performance tests in `tests/performance/test_search_ids_performance.py`

**Actual Project Structure (from project-structure.md):**
```
src/rabbitmq_mcp_server/
├── mcp_server/
│   ├── __init__.py
│   ├── server.py              # MCPServer with stdio_server() transport (Story 1.7)
│   ├── tools.py               # Tool implementations (search-ids will go here)
│   ├── search.py              # Semantic search implementation (NEW in this story)
│   ├── operations.py          # Operation registry (get-id, Story 1.9)
│   └── executor.py            # Operation execution (call-id, Story 1.10)
├── config/
│   ├── settings.py            # Extended in Story 1.7 with MCP config
│   └── logging.py             # Structlog setup
├── models/
│   ├── operations.py          # Operation metadata models (Story 1.5)
│   └── mcp_types.py           # MCP-specific types (NEW in this story)
├── schemas/
│   └── generated_schemas.py   # Auto-generated from OpenAPI (Story 1.4)
```

**Runtime Data Dependencies:**
- Embeddings file at `data/embeddings.json` (generated by Story 1.6, 1.36MB, 132 operations)
- Operations registry at `data/operations.json` (generated by Story 1.5, 119KB)
- Model cache at `~/.cache/torch/sentence_transformers/all-MiniLM-L6-v2/` (~90MB)
- All loaded at server startup by Story 1.7's server.py (reuse existing loading code)

**Files to Create:**
- `src/rabbitmq_mcp_server/mcp_server/search.py` - SemanticSearch class with search implementation
- `src/rabbitmq_mcp_server/models/mcp_types.py` - SearchResult dataclass (if not exists)
- `tests/unit/test_semantic_search.py` - Unit tests for semantic search
- `tests/integration/test_search_ids_tool.py` - Integration tests via MCP protocol
- `tests/performance/test_search_ids_performance.py` - Performance benchmarks

**Files to Modify:**
- `src/rabbitmq_mcp_server/mcp_server/tools.py` - Add search-ids tool implementation
- `src/rabbitmq_mcp_server/mcp_server/server.py` - Register search-ids tool (if not auto-registered)
- `README.md` - Add search-ids tool documentation and examples

**Dependencies Already Satisfied:**
- `sentence-transformers>=2.6.0` (installed in Story 1.6)
- `numpy>=1.26.0` (installed in Story 1.6)
- `mcp>=1.0.0` (installed in Story 1.7)
- `structlog>=24.1` (installed in Story 1.7)

### Learnings from Previous Story

**From Story 1-7-mcp-server-foundation-json-rpc-20 (Status: done)**

- **New Files Created**: 
  - `src/mcp_server/server.py` - Main MCP server with stdio transport and JSON-RPC 2.0
  - `src/mcp_server/jsonrpc.py` - JSON-RPC 2.0 message validation and response builders
  - `src/mcp_server/__main__.py` - CLI entry point
  - `tests/unit/test_jsonrpc.py` - 16 unit tests for JSON-RPC message handling
  - `tests/unit/test_mcp_protocol.py` - 3 unit tests for MCP protocol methods
  - `tests/integration/test_server_startup.py` - 7 integration tests for server startup

- **Modified Files**:
  - `src/mcp_server/__init__.py` - Added exports for MCPServer and main()
  - `src/config/settings.py` - Extended with MCP server configuration fields (operations_registry_path, embeddings_path, log_level, log_output_dir)

- **Performance Characteristics**: 
  - Server startup time: <1 second (verified in integration tests) ✅
  - Operations registry loads in <50ms (119KB file) ✅
  - Embeddings load in <500ms (1.4MB file, well under target) ✅
  - MCP protocol methods respond with <10ms overhead ✅

- **MCP Protocol Implementation Patterns**: 
  - Used MCP SDK's Server class with stdio_server() transport for protocol compliance
  - Tool definitions include proper JSON Schema inputSchemas as required by MCP spec
  - Protocol version "2024-11-05" and server version "3.0.0" as specified
  - JSON-RPC 2.0 error codes: PARSE_ERROR=-32700, INVALID_REQUEST=-32600, METHOD_NOT_FOUND=-32601, INVALID_PARAMS=-32602, SERVER_ERROR=-32000
  - Graceful shutdown with SIGTERM/SIGINT handlers and 5-second grace period

- **Testing Patterns Established**: 
  - Unit tests for JSON-RPC protocol: 16 tests covering validation, error codes, response formats
  - Integration tests for server startup: 7 tests covering artifact loading, timing, error handling
  - All 26 new tests passing (100% success rate)
  - Mock sentence-transformers for fast tests (pattern for this story)

- **Configuration and Data Loading Patterns**: 
  - Reused existing Settings class from `src/config/settings.py` (extend, don't duplicate)
  - Load operations.json and embeddings.json at server startup with validation
  - Fail fast if critical files missing with clear error messages
  - Log loading progress with timing information using structlog

- **Known Limitations from Story 1-7**:
  - Some Task 9 stdio transport tests not implemented (low priority for foundation story)
  - Task 10 documentation items deferred (README updates not critical for MVP)
  - Pre-existing test import issue in test_debug_support.py (not related to Story 1-7)

- **Recommendations for This Story**:
  - **INTEGRATE** with existing MCPServer class from src/mcp_server/server.py (don't create new server)
  - **REGISTER** search-ids tool in existing tool registry (follow established pattern)
  - **REUSE** existing embeddings loading code from server.py startup sequence
  - **EXTEND** existing Settings class for any search-specific configuration
  - **FOLLOW** JSON-RPC error response pattern from jsonrpc.py (consistency critical)
  - **APPLY** testing patterns: unit tests with mocks, integration tests end-to-end
  - **VALIDATE** all parameters against JSON Schema before execution (MCP requirement)

[Source: docs-bmad/sprint-artifacts/stories/1-7-mcp-server-foundation-json-rpc-20.md#Dev-Agent-Record]

**From Story 1-6-semantic-embeddings-generation (Status: done)**

- **New Files Created**: 
  - `data/embeddings.json` - 132 operation embeddings (384 dimensions, 1.36MB)
  - `scripts/generate_embeddings.py` - Embedding generation with all-MiniLM-L6-v2
  - `scripts/test_embeddings.py` - Quality validation with semantic queries
  - `scripts/benchmark_embeddings.py` - Performance benchmarking
  - `tests/unit/test_embeddings.py` - 13 comprehensive unit tests

- **Performance Characteristics**: 
  - Embeddings load time: 12ms (far under 500ms target) ✅
  - Query encoding time: 9.72ms (far under 100ms target) ✅
  - Similarity computation: <1ms (numpy vectorized operations)
  - All performance targets exceeded by significant margins

- **Embedding Quality Verified**: 
  - Semantic test cases pass with high similarity scores (0.7-1.0)
  - Portuguese language support excellent (model handles multi-language descriptions)
  - Top-5 result validation shows accurate semantic matching
  - Normalized vectors enable efficient cosine similarity via dot product

- **Technical Decisions**: 
  - `all-MiniLM-L6-v2` model selected for optimal speed/quality/size balance
  - Vector normalization (unit vectors) enables cosine similarity via dot product
  - JSON storage format simple, portable, fast loading (<50MB target)
  - Build-time generation eliminates runtime overhead

- **Patterns Established**: 
  - Comprehensive error handling with troubleshooting messages
  - Separate test and benchmark scripts for validation
  - CI/CD integration with model caching (speeds up builds)
  - Type hints with numpy.typing for precise ndarray types

- **Recommendations for This Story**:
  - **USE** embeddings from data/embeddings.json (already validated, 132 operations)
  - **REUSE** same model (all-MiniLM-L6-v2) for query encoding (consistency critical)
  - **LEVERAGE** 9.72ms query encoding time → easily achieves <100ms search target
  - **APPLY** numpy.typing patterns for type safety (avoid mypy --strict violations)
  - **FOLLOW** LRU caching pattern to avoid redundant model.encode() calls
  - **REFERENCE** test_embeddings.py for semantic query examples and similarity thresholds

[Source: docs-bmad/sprint-artifacts/1-6-semantic-embeddings-generation.md#Dev-Agent-Record]

### Testing Strategy

[Source: docs-bmad/architecture/testing-strategy.md, Story 1.7 patterns]

**Unit Tests (tests/unit/test_semantic_search.py):**
- Test SemanticSearch class initialization (loads embeddings, model)
- Test search method with known queries ("list queues" → queues.list)
- Test threshold filtering (all results have score ≥threshold)
- Test result ordering (descending by similarity_score)
- Test max_results parameter (limit enforced)
- Test empty results (no matches → empty list with suggestion)
- Test query caching (cache hit reduces encode calls)
- Mock sentence-transformers for fast tests (<5 seconds total, pattern from Story 1.7)
- Use pytest fixtures for test data (embeddings, operations)
- Achieve ≥90% code coverage for semantic_search.py module

**Integration Tests (tests/integration/test_search_ids_tool.py):**
- Test tool registration (search-ids in tools/list response)
- Test tool execution via MCP protocol (tools/call → search-ids)
- Test example queries from acceptance criteria with real embeddings
- Test parameter validation (invalid query, invalid threshold, invalid max_results)
- Test MCP response format (content type, structure, JSON-RPC compliance)
- Test error handling (empty query, gibberish query, missing parameters)
- Use real embeddings and model (end-to-end validation, no mocks)
- Verify correlation_id present in all responses and logs
- Test stdio transport integration (input via stdin, output via stdout)

**Performance Tests (tests/performance/test_search_ids_performance.py):**
- Benchmark search latency: p50, p95, p99 percentiles (<100ms p95 target from AC #5)
- Benchmark with cache: cache hit latency vs cache miss latency
- Benchmark embedding load time (<500ms target from AC #6)
- Statistical analysis with pytest-benchmark (n=100 runs for significance)
- Run on GitHub Actions runner (reference hardware for consistency)
- Validate performance degrades gracefully under load (100 concurrent queries)
- Memory profiling: ensure no memory leaks with repeated searches

**Test Organization and Naming:**
- Follow pytest naming convention: test_*.py files, test_* functions
- Descriptive test names: test_search_returns_relevant_results_for_list_queues_query
- Group related tests in classes: TestSemanticSearch, TestSearchIdsTool
- Use @pytest.mark.unit, @pytest.mark.integration, @pytest.mark.performance markers
- Parametrize tests for multiple scenarios: @pytest.mark.parametrize

**Mocking Strategy (Unit Tests):**
- Mock sentence-transformers model to avoid downloading 90MB model in CI
- Mock embeddings.json loading for fast test execution
- Mock operations.json to control test data precisely
- Use unittest.mock.patch for external dependencies
- Real implementations only in integration tests

**Coverage Requirements:**
- Overall test coverage: ≥80% (project standard)
- Critical paths: 100% coverage (search method, similarity calculation, result ranking)
- Error handling: 100% coverage (all error paths tested)
- Use pytest-cov for coverage reporting: pytest --cov=src/rabbitmq_mcp_server/mcp_server

**Analytics Validation:**
- Verify structured logging output format (JSON lines to stderr)
- Verify query, result count, top score logged in each search event
- Verify correlation_id present in logs (from request context)
- Verify no PII in logs (queries are operational terms, no user data)
- Test log sanitization if needed (though operational queries unlikely to contain secrets)

### References

**Epic and Story Context:**
- [Source: docs-bmad/epics/epic-1-foundation-mcp-protocol.md#Story-1.8] - Story definition with acceptance criteria
- [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#AC-8] - search-ids tool requirements and performance targets
- [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#Workflows-and-Sequencing] - Runtime semantic discovery flow
- [Source: docs-bmad/sprint-artifacts/tech-spec-epic-1.md#Semantic-Search-Interface] - SemanticSearch class specification
- [Source: docs-bmad/sprint-artifacts/1-6-semantic-embeddings-generation.md#Dev-Agent-Record] - Previous story patterns and learnings

**Architecture Documents:**
- [Source: docs-bmad/architecture/architecture-decision-records-adrs.md - ADR-002] - 3-tool semantic discovery pattern
- [Source: docs-bmad/architecture/architecture-decision-records-adrs.md - ADR-004] - JSON-based vector storage
- [Source: docs-bmad/architecture/technology-stack-details.md#Semantic-Search] - sentence-transformers integration
- [Source: docs-bmad/architecture/implementation-patterns.md#Type-Safety] - Type hints and validation requirements
- [Source: docs-bmad/architecture/implementation-patterns.md#Error-Handling] - Error handling patterns and user-facing messages
- [Source: docs-bmad/architecture/implementation-patterns.md#Validation] - Pydantic validation patterns
- [Source: docs-bmad/architecture/project-structure.md] - Directory layout and artifact locations
- [Source: docs-bmad/architecture/testing-strategy.md] - Testing standards, coverage requirements, and test organization

## Dev Agent Record

### Context Reference

<!-- Path(s) to story context XML will be added here by context workflow -->

### Agent Model Used

{{agent_model_name_version}}

### Debug Log References

### Completion Notes List

### File List
