# Open-Source Strategy for Clew Engine

## Decision Context

Releasing Clew Engine under AGPL-3.0 via an organization account, without exposing the full
private repository (infrastructure code, platform implementation, closed-source cloud components).
This document covers the recommended mechanics, what to include or exclude, licensing implications
for the Clew Platform (proprietary SaaS layer), and the key opportunities and risks.

---

## 1. Shadow Repo Mechanics

### Recommended approach: fresh-history mirror

Create a new GitHub repository under your org account and initialize it with a clean commit
history. Do **not** use `git filter-branch` or `git filter-repo` on the private repo — filtering
is lossy and error-prone, and even a well-filtered history can leak fragment messages, author
emails, or deleted file content through commit metadata.

**Step-by-step:**

```bash
# 1. Create a staging area
mkdir /tmp/clew-engine-public && cd /tmp/clew-engine-public
git init

# 2. Copy only the public artifacts from the private repo
rsync -av --include='src/' --include='packages/' --include='tests/' \
          --include='specs/' --include='docs/' --include='pyproject.toml' \
          --include='docker-compose.yml' --include='Dockerfile' \
          --include='LICENSE' --include='CONTRIBUTING.md' --include='README.md' \
          --exclude='*' /path/to/private/clew-engine/ .

# 3. Write a clean README targeted at public contributors
# (see Section 3 — Public Repo Contents)

# 4. Commit as a single "initial release" commit
git add .
git commit -m "chore: initial public release v0.1.0"

# 5. Push to the org account
git remote add origin git@github.com:<org>/clew-engine.git
git push -u origin main
```

After the initial push, you can synchronize future releases by cherry-picking or by scripting
an rsync from the private repo + `git commit` on the public one. A CI workflow on the private
repo (triggered by a `release/*` tag) can automate this: build a tarball of the public subset,
push a signed commit to the public mirror, and create a GitHub Release.

### What to include in the public repo

| Path | Include? | Notes |
|------|----------|-------|
| `src/clew/` | ✅ Yes | Core library — indexing, search, and ingestion engine |
| `packages/clew-api/` | ✅ Yes | FastAPI search and graph endpoints |
| `packages/clew-ingestion/` | ✅ Yes | Tree-sitter-based code parsing and indexing worker |
| `packages/clew-mcp/` | ✅ Yes | Universal MCP server (dual-mode: local/cloud) |
| `tests/` | ✅ Yes | Reproducibility and trust signal |
| `specs/` | ✅ Partial | Ship architecture and data model; exclude internal platform specs |
| `docs/` | ✅ Partial | API contracts, deployment guides; exclude roadmap/GTM |
| `pyproject.toml` | ✅ Yes | Install surface, dependencies, scripts |
| `docker-compose.yml` | ✅ Yes | Local development setup (Neo4j, Qdrant) |
| `LICENSE` | ✅ Yes | AGPL-3.0 full text |
| `CONTRIBUTING.md` | ✅ Yes | Contribution process and CLA requirement |
| `README.md` | ✅ Yes | Public-facing, stripped of internal references |
| `examples/` | ✅ Partial | Include language parsing examples; exclude proprietary integrations |
| `configs/` | ✅ Partial | Ship base configurations; exclude SaaS-specific overrides |
| `.claude`, `.agent` | ❌ No | Internal agent configs |
| `packages/clew-control-plane/` | ❌ No | Multi-tenant orchestration (proprietary platform layer) |
| `packages/clew-worker/` | ❌ No | Proprietary worker implementation |
| `packages/clew-webapp/` | ❌ No | Dashboard (proprietary SaaS UI) |
| `discussions/`, private ADRs | ❌ No | Contains internal strategic and architectural discussions |
| `.git` history | ❌ No | Fresh repo; history contains internal commit messages and platform code |

### Keeping the repos in sync

- Tag releases in the private repo (e.g., `v0.2.0`). A CI job copies the public subset,
  commits it to the public mirror, and creates a matching GitHub Release.
- Never commit to the public repo directly — treat it as a read-only release artifact.
  All development stays in the private repo.
- Pin the public `README.md` and `CONTRIBUTING.md` in the private repo under a
  `public/` directory so they version-control cleanly alongside the source.

> **⚠️ TODO (tracking):** The CI workflow that automates private→public sync on `release/*` tags
> does not exist yet. This is the primary operational risk once releases start. A GitHub Actions
> workflow (`.github/workflows/sync-public-mirror.yml`) should be created before the first
> public release. At minimum it should: (1) check out the private repo on `release/*` tag push,
> (2) rsync the public subset to a staging directory, (3) push a signed commit to the public
> org mirror, and (4) create a matching GitHub Release with release notes. Track this as a
> follow-up issue before cutting v0.1.0.

---

## 2. License Choice: AGPL-3.0

AGPL-3.0 ("Affero GPL") is GPL-3.0 with one additional clause: **if you run the software as a
network service, you must offer the source code of the running version to users of that service**.
This is intentional and strategically important:

- Anyone who wraps Clew Engine in a SaaS product **must** publish their modifications under AGPL
  or acquire a commercial license from you.
- This is the standard "open core with copyleft" moat: the indexing algorithm and search logic
  are visible (trust, adoption, reproducibility), but commercial SaaS use forces a conversation
  about a paid license.
- AGPL does **not** prevent users from self-hosting, using Clew in corporate AI agents,
  or running on-premises workers — those use cases are fully permitted.

### AGPL and the Clew Platform (proprietary SaaS)

If you offer **Clew Platform** as a hosted, multi-tenant service (job orchestration, cloud-hosted
workers, corpus management, audit logs), you are the copyright holder and are **not** bound by
AGPL on your own code. You can simultaneously:

1. Publish Clew Engine (core indexing and search) under AGPL (community adoption).
2. License the same code commercially to enterprise customers who do not want AGPL obligations.
3. Keep Clew Platform's proprietary extensions (multi-tenant control plane, worker orchestration,
   hosted infrastructure, corpus attestation, billing) closed-source — these are separate works
   that call the AGPL library.

This is **dual licensing** (also called a commercial exception). Prominent examples:
MySQL (GPL + commercial), Qt (LGPL + commercial), MongoDB (SSPL + commercial).

**Requirements for dual licensing to work:**
- You must hold copyright on all code in the repo. This means **you cannot merge external
  contributions without a Contributor License Agreement (CLA)** that grants you the right to
  relicense. Add a CLA requirement to `CONTRIBUTING.md` before accepting any PRs. See Section 4.
- Clearly document the commercial license offering in the `README.md` so enterprise users know
  where to go.

---

## 3. Public Repo Contents

### README.md (public)

The public README should:
- Open with a one-sentence positioning statement ("Hybrid graph + vector search engine for code").
- Include installation instructions (Docker Compose, pip).
- Show a quick-start example (ingest a repo, run a semantic search, traverse the graph).
- List supported languages (Python, JavaScript, initial support; extensible via Tree-sitter).
- Include architecture diagram (Neo4j + Qdrant, ingestion pipeline, MCP protocol).
- Link to the API contract and MCP tools documentation.
- Link to the commercial license offering ("Looking to deploy Clew as a proprietary SaaS
  or white-label service? Contact us for a commercial license.").
- **Not** reference internal platform code, strategy documents, or the private repository.

### CONTRIBUTING.md

See the `CONTRIBUTING.md` file at the root of this repo. Key elements:
- CLA requirement (see Section 4).
- Code style (ruff, pytest, poetry).
- Branch and PR conventions.
- Scope of accepted contributions (bug fixes, new language support, graph expansion features,
  test coverage, documentation).
- Scope of **declined** contributions (changes to the data model schema without discussion,
  removal of AGPL license, multi-tenant features) — direct these to issues for discussion first.

### docs/api-contract.md and docs/mcp-tools.md

Publish the full API specifications and MCP tool interface so users can:
- Deploy local instances without the cloud platform.
- Build agents using the MCP server.
- Integrate Clew into their own infrastructure.

### specs/architecture.md and specs/data-model.md

Publish the system design and entity schemas at a high level. This builds trust and invites
community feedback. Shield implementation details (cloud scaling strategies, multi-tenancy
tricks) behind a "Platform Architecture" boundary (see Section 5).

---

## 4. Contributor License Agreement (CLA)

A CLA is mandatory if you want dual licensing to remain viable. Without it, every external
contributor holds a co-copyright interest in their patch, and you cannot relicense their
contributions under commercial terms.

**Minimum viable CLA:**
- **Individual CLA:** Contributor assigns (or grants a broad license to) their contribution.
- **Corporate CLA:** Covers contributions made by employees within the scope of employment.

**Implementation options (lightest first):**
1. **CLA Assistant** (cla-assistant.io) — free, GitHub-integrated, stores signatures in a
   dedicated repo. Add the GitHub Action to enforce it on every PR.
2. **Developer Certificate of Origin (DCO)** — contributors sign-off commits with
   `git commit --signoff`. Less legally robust than a CLA but still establishes intent.
   Use DCO if you want the lowest friction for community contributions; note it does **not**
   grant you relicensing rights in the same way a full CLA does.
3. **Full CLA** via DocuSign / paper — appropriate for enterprise contributors but overkill
   for community PRs.

**Recommendation:** Use CLA Assistant for community PRs. Add a note that contributors from
organizations require a corporate CLA filed before their first PR is merged.

---

## 5. What Stays Closed (Open Core Boundary)

The indexing and search logic are the moat only if users trust the system. Publishing the
extraction, graph construction, and semantic search code is the trust signal. The following
should remain proprietary or access-controlled:

| Asset | Why |
|-------|-----|
| Curated code corpora (training data for embeddings) | Corpus curation is a defensible moat; third-party attribution is complex |
| Pre-trained embedding models (production `.onnx` / `.npz` files) | Models encode corpus fingerprints; releasing them devalues the hosted service |
| Multi-tenant control plane (platform orchestration) | Job scheduling, worker allocation, tenant isolation, billing |
| Corpus attestation service | Integrity verification and licensing authentication layer |
| Stored query cache and performance analytics | High-value operational data |
| Cloud infrastructure provisioning | Scaling, availability, disaster recovery strategies |
| Proprietary language parsers (beyond Tree-sitter) | Custom AST extensions for Tier 2 language support |
| Tier B inference steering adapters | Deep integration with proprietary inference APIs and model-specific optimizations |
| Billing, SSO, and audit logging | SaaS plumbing and compliance infrastructure |

**What to ship in the public repo:**
- A single small demo embedding model (`embeddings/demo-v0.onnx`, trained on a small,
  clearly attributed open-source corpus) so users can run the full pipeline without a hosted account.
- Tree-sitter parsing configs for supported languages (Python, JavaScript, Go, Rust).
- Document that production-grade embeddings and proprietary language support are available via
  Clew Platform or as a paid license.

---

## 6. Opportunities

| Opportunity | Notes |
|-------------|-------|
| **Developer trust and credibility** | Publishing core search logic invites scrutiny — and surviving scrutiny is the strongest signal for B2B infrastructure |
| **Community contribution** | Language support via Tree-sitter extensions, new entity types in the graph, edge-case test coverage; reduces your burden |
| **Ecosystem integrations** | Community will write VS Code plugins, GitHub Actions, Mendable/Perplexity adapters; widens adoption funnel |
| **Research validation** | The hybrid graph + vector paradigm is novel; OSS publication invites academic citations and external validation |
| **Recruiting signal** | A well-documented, trusted codebase attracts infra engineers who care about code quality |
| **AGPL as a sales tool** | The license itself generates enterprise sales conversations — companies needing AGPL exemptions become qualified leads |
| **Benchmark harness** | A public, reproducible search quality benchmark becomes a marketing asset with no ongoing cost |
| **Agent integration** | MCP server makes Clew a first-class integration target for AI agents (Claude, GitHub Copilot, custom agents) |

---

## 7. Risks

| Risk | Likelihood | Mitigation |
|------|-----------|------------|
| **Fork without CLA → relicensing blocked** | High if CLA is skipped | Enforce CLA from day one; do not accept PRs without it |
| **Competitor builds on OSS core + proprietary UI** | Medium | AGPL forces them to also open-source their search, or become a customer; real moat |
| **Corpus copyright exposure** | High if corpus is published carelessly | Never publish raw training data; publish only embedding weights with attribution metadata |
| **Theory docs / roadmap leaked via git history** | High if filter-branch used incorrectly | Use fresh-history mirror (Section 1); audit public repo git log before first push |
| **Platform-specific code in public commit diff** | Low but severe | CI automation of the sync step prevents accidental inclusion |
| **Community PR spree on data model** | Medium | Define decline scope in CONTRIBUTING.md (no breaking data model changes without RFC); use issue labels for triage |
| **AGPL scares away enterprise users** | Low for infrastructure | Enterprise developers understand AGPL; LGPL-3.0 is an alternative if adoption matters more than SaaS moat |
| **Embedding model leakage via reverse engineering** | Medium | Ship only a small demo model; document that production models are hosted-only or commercially licensed |
| **Enterprise confusion about what's open vs. closed** | Medium | Explicit open core boundary table in README (see Section 5); commercial license page with clear usage policies |

---

## 8. Clew Platform Path (Proprietary SaaS Layer)

Doing this in the right order matters:

1. **Before first public push:** Set up CLA, choose a commercial license template (MIT + Commons
   Clause or a bespoke commercial license), draft the "contact for commercial license" landing
   page or email alias.
2. **Alpha public release (now):** Publish Clew Engine under AGPL. Ship a demo embedding model.
   No commercial Clew Platform offering yet — focus on adoption signals and trust building.
3. **Clew Platform Beta (next phase):** Launch job orchestration, hosted workers, and multi-tenant
   corpus management. Offer it separately to enterprises needing hosted infrastructure.
4. **When first enterprise inquiry arrives:** Close the first commercial license deal manually
   (NDA + bespoke agreement). Use it to calibrate what enterprise customers actually need.
5. **Growth phase:** Automate commercial license sales (Stripe + license key). Launch metered
   pricing on Clew Platform (per-repo, per-query, per-worker). Introduce corpus attestation
   and audit logging as paid add-ons.
6. **Scale phase:** White-label Clew Platform, SLA-backed support, bespoke corpus curation
   contracts, vendor-specific language support, enterprise compliance modules.

**Protecting Clew Platform from AGPL bleed:**
- Keep platform extensions in a separate private repository. They import the AGPL library but
  are separate works. As the copyright holder of the AGPL library you are not restricted.
- Document the architecture boundary: "clew-engine (AGPL-3.0 open) ← clew-platform (proprietary)".
  This is standard open core practice.
- Clearly communicate that self-hosted deployments can use the AGPL-licensed engine freely,
  but the hosted, multi-tenant platform is a proprietary service.

---

## 9. Recommended Timeline

| Step | When | Owner |
|------|------|-------|
| Draft and review public README | Before first push | You |
| Set up CLA Assistant on public repo | Before first public commit | You |
| Add commercial license contact to README | Before first public commit | You |
| Audit docs/ for private references | Before first public commit | You |
| Initialize public shadow repo (fresh history) | Alpha release | You |
| Automate private→public sync in CI | First release tag | Engineering |
| Draft commercial license template | Before first enterprise inquiry | Legal / You |
| Publish search quality benchmarks (Stage 3) | Post-alpha | Engineering |
| Launch Clew Platform beta (hosted workers) | Growth phase | Engineering |

---

## 10. Key Success Metrics

Track these during and after public release:

- **GitHub Stars**: Indicates adoption and trust. Target: 500+ stars in first 6 months.
- **External Issues & PRs**: Community engagement. Target: 10+ quality external PRs in first year.
- **Language Support Expansion**: Community contributions. Target: JavaScript, Go, Rust by v0.3.
- **Downstream Integrations**: Ecosystem health. Target: 5+ agents or tools using Clew MCP by v0.5.
- **Enterprise Inquiries**: Commercial signal. Target: 3+ enterprise sales conversations in year one.
- **Research Citations**: Academic validation. Target: 2+ papers citing Clew architecture.

---

## 11. Handoff Checklist Before First Public Push

- [ ] AGPL-3.0 LICENSE file is in the public repo root
- [ ] CONTRIBUTING.md includes CLA requirement
- [ ] README.md references commercial license path
- [ ] All private docs removed from `docs/` (keep api-contract.md, mcp-tools.md, architecture.md)
- [ ] No `.claude`, `.agent`, or platform-specific files in the public tree
- [ ] Git history is fresh (no leaked commits from private repo)
- [ ] CLA Assistant GitHub Action is configured
- [ ] Release notes prepared for v0.1.0
- [ ] Internal announcement prepared (blog post / discussion thread)
- [ ] Commercial license template drafted (even if not finalized)
