# ADR-007: Sync Architecture — Webhooks Removed, Platform Pub/Sub + Git Hooks

**Date**: 2026-02-17
**Status**: Accepted
**Supersedes**: ADR-005 (GitHub integration via webhooks — engine-side)

---

## Context

The engine previously exposed `/v1/webhooks/github` and `/v1/webhooks/gitlab` endpoints to receive push and pull_request events directly from GitHub/GitLab. This was identified as a platform-concern leak:

- The engine runs locally or in isolated private infrastructure — it is not a public server that GitHub can reach
- The webhook receipt, fan-out, and job queuing are `clew-platform`'s responsibility
- The `sync_task.py` Celery worker was a stub (`time.sleep(2)`) that dispatched to a queue which did nothing
- Requiring Redis and Celery as engine dependencies for a feature that didn't work added operational complexity with no benefit

The sync task stub also revealed a deeper gap: **incremental ingestion does not exist**. The engine only does full re-ingest from scratch. A correct sync model requires this to be built first.

---

## Decision

### Webhooks move to the platform entirely

`clew-platform` receives all inbound webhooks from GitHub and GitLab. The engine has no inbound HTTP surface for repository events. All webhook-related files have been removed from the engine:

- `src/routes/webhooks.py`
- `src/routes/jobs.py`
- `src/tasks/sync_task.py`
- `celery_app.py`

### Two sync trigger paths, one sync pipeline

The engine supports two trigger paths that converge on the same incremental ingestion pipeline:

**Local mode (no platform account)**

```
git commit / git merge
    → post-commit / post-merge hook
    → clew sync (CLI subcommand)
    → diffs HEAD~1..HEAD
    → IncrementalIngestionPipeline({added, modified, removed})
```

The `clew hooks install` command installs these hooks into the user's `.git/hooks/`. No external services required.

**Platform mode (platform account configured)**

```
git push
    → GitHub/GitLab webhook
    → clew-platform receives, validates, enqueues job
    → engine (persistent outbound WebSocket connection)
    → platform pushes job payload: {repo_id, commit_sha, added, modified, removed}
    → IncrementalIngestionPipeline({added, modified, removed})
    → engine reports completion/failure back over same connection
```

The engine initiates the connection outbound (no inbound ports required). The connection is established only when `CLEW_PLATFORM_TOKEN` is set in the engine's environment.

### Platform is the pub/sub producer

The platform holds the job queue. The engine is a pure consumer. The connection protocol is WebSocket (or SSE if the engine is read-only from the platform's perspective). The platform can buffer jobs when no engine is connected and deliver them on reconnect.

This model also gives the platform visibility into sync status, which is required for the dashboard and billing metering.

---

## Consequences

### Positive

- Engine requires no inbound network access — works behind firewalls, on laptops, in CI
- No Redis or Celery dependency on the engine side
- Local mode works completely offline — no platform account required
- Both trigger paths share the same incremental pipeline, reducing divergence
- Platform gains sync status observability as a side effect of the connection protocol

### Negative

- Incremental ingestion must be built before either path is functional (currently only full re-ingest exists)
- Platform requires a WebSocket endpoint and job queue — new platform work
- Engine needs reconnect logic, auth, and backpressure handling for the WS client

### Orphaned graph endpoints

Two graph read endpoints remain in the engine that were previously populated by the removed webhook handler:

- `GET /v1/graph/file/{path}/pull_requests`
- `GET /v1/graph/pull_request/{pr_number}/impact`

These will return empty results until the platform sync connection populates PR node data. The correct write path is: platform pushes PR metadata over the WS connection when a `pull_request` event is received → engine calls `graph_store.create_pr_node()` and `graph_store.link_pr_to_files()`. The read API shape is correct; only the write path needs wiring.

---

## Build Order

See `docs/design/008-incremental-sync-design.md` for the full phased plan.

1. **Phase 1** — Incremental sync pipeline (prerequisite for everything else)
2. **Phase 2** — Local trigger (git hooks + `clew sync` CLI)
3. **Phase 3** — Platform WebSocket connection
