# Architecture Design Health Review

**Date**: 2026-02-17
**Reviewer**: Claude (QA Auditor)
**Status**: ✅ COMPLETE — webhook removal actioned same session

---

## Executive Summary

clew-engine shows genuine architectural intent. The Ports & Adapters pattern is real, the registry system is elegant, and the ingestion pipeline is well-decomposed. Several structural issues were identified — one of them (webhooks) was a full platform-concern leak that has been removed as part of this session. Remaining issues are fixable without rearchitecting.

**Overall Grade: B+**

---

## What's Working Well

### Hexagonal Architecture — `adapters/base.py`
The three core protocols (`VectorStore`, `GraphStore`, `EmbeddingProvider`) use Python `runtime_checkable` Protocols. Swapping Qdrant for pgvector or Neo4j for a noop is a one-line env var change. This is clean.

### Registry Pattern (Open/Closed) — `adapters/registry.py`, `processors/registry.py`
Both `AdapterRegistry` and `NodeProcessorRegistry` eliminate if/else chains and allow extension without modification. Thread-safe with `Lock`. Optional protocol validation (`validate_protocol_implementation`) is a nice touch. Consistent with OCP.

### Ingestion Pipeline (Chain of Responsibility + Strategy) — `pipeline/orchestrator.py`
The 6-stage pipeline with a shared `IngestionContext` passed through each stage is textbook CoR. Auto-detection of async stages via `inspect.iscoroutinefunction` avoids forcing all stages into async unnecessarily. Batching in `ParsingStage` (BATCH_SIZE=50) shows performance awareness.

### Thin Controllers — `routes/search.py`, `routes/graph.py`
Routes delegate to injected adapters. No raw DB calls in handlers. FastAPI `Depends()` wiring via `@lru_cache` singletons is correct.

### MCP Layer Separation — `clew-mcp/src/server.py`
Clean separation: MCP tools → HTTP client → API. `handle_api_errors` decorator eliminates repetitive try/except. `Formatter` classes correctly separate presentation from data.

---

## Issues Found

### RESOLVED: Webhooks Were a Platform-Concern Leak [Blocker → Fixed]

`/v1/webhooks` (GitHub + GitLab push events), `/v1/jobs` (Celery status), `celery_app.py`, and `src/tasks/sync_task.py` were all present in the engine. This is **wrong by design**:

- The engine is not a server that GitHub/GitLab can reach — it runs locally or in isolated infra
- Webhook receipt and fan-out is `clew-platform`'s job
- The sync task itself was a stub (`time.sleep(2)`) dispatching to a queue that did nothing
- The deduplication logic depended on a Redis instance that the engine should not require

**Action taken**: Removed all of the following (note: two orphaned graph endpoints remain — see below):
- `packages/clew-api/src/routes/webhooks.py`
- `packages/clew-api/src/routes/jobs.py`
- `packages/clew-api/src/tasks/sync_task.py`
- `packages/clew-api/src/tasks/__init__.py`
- `packages/clew-api/celery_app.py`
- `packages/clew-api/tests/test_webhooks.py`
- `packages/clew-api/tests/test_webhooks_pr.py`
- Router registrations removed from `src/main.py`

**Orphaned graph endpoints** — `routes/graph.py` still exposes:
- `GET /v1/graph/file/{path}/pull_requests`
- `GET /v1/graph/pull_request/{pr_number}/impact`

These read PR nodes that were *written* by the now-removed webhook handler. They will return empty results indefinitely until the platform sync connection populates PR data via the socket channel. Left in place intentionally — they are the right *read* API once the write path is correctly implemented via the platform connection.

**Correct sync model** (to be implemented):
- **Local mode**: git post-commit/post-merge hooks trigger the engine's CLI (`clew sync`) directly
- **Platform mode**: engine opens a persistent socket/WebSocket connection to `clew-platform` IFF a platform account is configured; platform pushes change notifications over that channel

---

### Business Logic Leaking into the Route Handler [Medium]

`routes/search.py` is doing too much. The following are use-case-level concerns, not HTTP concerns:

```python
apply_graph_boost(...)         # scoring strategy
_compute_search_multiplier()   # candidate over-fetch heuristic
is_test_file(...)              # domain classification
_apply_reranking(...)          # result re-ordering
```

A `SearchUseCase` class would make these individually testable and reusable from the MCP layer directly (bypassing HTTP entirely). The MCP `search_codebase` tool currently goes through HTTP to reach logic that could be a direct function call.

**Recommendation**: Extract a `SearchUseCase` from `routes/search.py`. Wire both the route and the MCP server to it.

---

### `IngestionContext` Uses `Any` for Its Core Dependencies [Low-Medium]

`pipeline/context.py` lines 149–151:

```python
vector_store: Any = None  # Will be VectorStore instance
graph_store: Any = None   # Will be GraphStore instance
parser: Any = None        # Will be CodeParser instance
```

These defeat the protocol definitions. Should be typed as `VectorStore | None`, `GraphStore | None`, etc. This would let mypy catch adapter misuse at ingestion time and communicate intent to contributors.

---

### Side Effect in `_create_context` Factory Method [Low-Medium]

`orchestrator.py` line 234:

```python
def _create_context(...) -> IngestionContext:
    ...
    self.graph_store.create_repo_node(repo_id, ...)  # ← side effect in factory
    return context
```

Factory methods shouldn't mutate state. `create_repo_node` belongs in `RepositoryPreparationStage.execute()`, which already exists for exactly this purpose.

---

### Embedding Provider Bypasses the Registry [Low]

`dependencies.py`:

```python
def get_embeddings() -> EmbeddingProvider:
    """This still uses the legacy get_embedding_provider() function.
    Future enhancement: Migrate embeddings to use the registry system as well."""
    return get_embedding_provider()
```

Vector and graph adapters use the registry; embeddings do not. Minor inconsistency but creates a second mental model for contributors. The comment acknowledges the debt — should be tracked.

---

### `AdapterRegistry` Does Not Cache Instances [Low]

The registry stores factories, not instances. `get()` calls `factory()` fresh each time. The `@lru_cache` in `dependencies.py` is the only guard. If anyone calls `vector_store_registry.get("qdrant")` directly (e.g., in tests), they get a new connection object per call.

**Recommendation**: Add optional `singleton=True` parameter to `register()`, or document that registry consumers are responsible for caching.

---

### Sync/Async Type Boundary in Ingestion [Low]

`pipeline/stages/parsing.py` line 293:

```python
await asyncio.to_thread(
    context.graph_store.create_file_nodes_batch,  # sync method
    ...
)
```

The `clew-api` `GraphStore` protocol defines async methods. The ingestion package's `graph.py` is a separate sync implementation. Two different `GraphStore` contracts exist in the same codebase with no shared interface. This will create confusion as the codebase grows — especially if someone tries to swap adapters in ingestion.

**Recommendation**: Establish a shared `clew-core` package for protocols, or explicitly document that ingestion's `graph.py` is a different (sync) abstraction.

---

## Summary Table

| Area | Grade | Notes |
|------|-------|-------|
| Ports & Adapters | A | Protocols well-defined, swappable via env |
| Registry Pattern | A- | Solid, but no instance caching |
| Pipeline / Stages | A- | Good CoR; factory side-effect is the wart |
| Route / Controller | B | Business logic belongs in use cases |
| Type Safety | C+ | `Any` on IngestionContext hurts |
| Async Design | B+ | Batching is good; sync/async boundary is murky |
| Webhook / Sync | — | Removed (platform leak) |
| MCP Layer | A- | Clean delegation and formatting separation |

---

## Recommended Next Actions (Priority Order)

1. **[Done]** ~~Remove webhooks~~ — completed this session
2. **Design the platform sync connection** — socket/WS client that activates IFF `CLEW_PLATFORM_TOKEN` is set; git hook CLI path for local mode
3. **Extract `SearchUseCase`** from `routes/search.py` — also enables MCP to call it directly
4. **Fix `_create_context` side effect** — move `create_repo_node` into `RepositoryPreparationStage`
5. **Type `IngestionContext` properly** — replace `Any` with protocol types
6. **Migrate embeddings to registry** — close the inconsistency with vector/graph adapters
