# ADR-108: Neptune-First Production Graph Architecture

**Date:** 2026-03-16 | **Status:** ACCEPTED

## Context

Graqle Studio currently loads graphs from JSON files on S3, served through a Lambda Function URL. This works for the free tier demo but has critical limitations:

1. **JSON/S3 is not a graph database** — no traversal queries, no real-time updates, no multi-user writes
2. **Lambda cold start + 2.8MB JSON parse** = 17s through proxy, 2-3s direct — unacceptable for production
3. **Multiple projects** (Graqle, CrawlQ/TraceGov, others) need to coexist on one dashboard with per-project isolation
4. **Free and Pro users share the same infrastructure** — tier gating is at the feature level, not the infra level
5. **The graph is the product** — serving it from a flat file undermines the value proposition

### Graqle's Own Analysis (via `graq reason`)

The SDK's graph loading layer (`graqle/core/graph.py`) already supports:
- `from_neo4j()` — primary loader (72% confidence), bolt protocol
- `from_json()` — fallback for zero-dep mode
- `graqle/connectors/neo4j.py` — bolt-compatible connector (works with Neptune)

The Studio serving layer (`graqle/server/app.py`) loads graph into in-memory state via `create_app(graph_path)`. Switching to Neptune means the graph stays in Neptune and the serving layer queries it instead of loading into memory.

## Decision

**AWS Neptune Serverless** as the production graph database from day one. One infrastructure for all tiers — free users see gated features, Pro/Team users get full access.

### Architecture

```
graq compile (local)
    ↓
graq cloud push
    ↓
Lambda API ←→ Neptune Serverless (eu-central-1)
    ↓                    ↓
Function URL        Per-project namespaces
    ↓                    ↓
graqle.com/dashboard    Multi-project graph isolation
(all tiers)              via graph_name property on all nodes/edges
```

### Neptune vs Alternatives

| Factor | Neptune Serverless | Neo4j Aura | JSON/S3 |
|--------|-------------------|------------|---------|
| AWS-native | Yes (same VPC as Lambda) | No (external) | Yes |
| Cost at idle | ~$0.07/hr (min NCU) | $65/mo (free tier limited) | ~$0 |
| Bolt protocol | Yes | Yes | N/A |
| IAM auth | Yes | No (user/pass) | Yes |
| Cypher support | OpenCypher | Full Cypher | N/A |
| GDS algorithms | No | Yes | N/A |
| Multi-project isolation | Property-based | Database-per-project | File-per-project |
| Traversal queries | Native (~5ms) | Native (~5ms) | Python BFS (~200ms) |
| Managed backups | Yes | Yes | Manual |

**Verdict:** Neptune Serverless. AWS-native, IAM auth, scales to zero(ish), bolt-compatible with existing connector code.

### Per-Project Isolation

Every node and edge gets a `project_id` property:
```cypher
MATCH (n {project_id: 'graqle-sdk'}) RETURN n
```

This allows:
- One Neptune cluster serving ALL projects
- Lambda loads project-specific subgraphs
- Cross-project queries for Team tier (`graq link merge`)
- Free tier: read-only access to their own project graph

### Tier Gating (Feature-Level, Not Infra-Level)

| Feature | Free | Pro ($19/mo) | Team ($29/dev/mo) |
|---------|------|-------------|-------------------|
| Graph visualization | Full graph | Full graph | Full graph |
| Node count limit | 500 | 25,000 | Unlimited |
| Traversal queries | 2-hop BFS | Native Neptune | Native + GDS |
| Projects | 1 | 3 | Unlimited |
| Cross-project | No | No | Yes |
| DRACE governance | Current only | + 30 history | + team comparison |
| Real-time updates | No | Yes | Yes |
| API access | No | Rate-limited | Full |

### Lambda ↔ Neptune Connection

```python
# graqle/connectors/neptune.py (new)
import os
from graqle.connectors.neo4j import Neo4jConnector

class NeptuneConnector(Neo4jConnector):
    """Neptune-specific connector — extends Neo4j bolt connector with IAM auth."""

    def __init__(self):
        endpoint = os.environ['NEPTUNE_ENDPOINT']
        super().__init__(
            uri=f"bolt://{endpoint}:8182",
            user="",  # IAM auth
            password="",
        )
```

The existing `Neo4jConnector` works with Neptune (bolt protocol compatible). The `NeptuneConnector` subclass adds IAM-based auth.

### Migration Path

1. **Phase 1 (Now):** Lambda Function URL → Neptune for graph reads. JSON/S3 stays as import format.
2. **Phase 2:** `graq cloud push` writes directly to Neptune via bolt. S3 becomes backup-only.
3. **Phase 3:** Real-time graph updates from CLI → Neptune → Studio (WebSocket notifications).

## Consequences

### Positive
- Production-grade graph serving (~5ms traversal vs 200ms Python BFS)
- Multi-project dashboard with one cluster
- Same infrastructure for all tiers (simpler ops)
- Existing bolt connector code works unchanged
- IAM auth (no secrets management for Neptune)

### Negative
- Neptune Serverless minimum cost ~$0.07/hr (~$50/mo at idle)
- OpenCypher is subset of full Cypher — some Neo4j-specific queries may not work
- No GDS (Graph Data Science) library — PageRank/community detection must be done in Python
- VPC networking required (Lambda must be in same VPC as Neptune)

### Risks
- **VPC Lambda cold starts** are longer (~1-2s additional). Mitigate with provisioned concurrency.
- **Neptune Serverless scaling** may have spikes on cold clusters. Mitigate with min NCU setting.
- **Bolt protocol compatibility** — 99% compatible but edge cases exist. Test all traversal queries.

## Modules Affected (from Graqle Impact Analysis)

| Module | Change | Risk |
|--------|--------|------|
| `graqle/connectors/neo4j.py` | Add Neptune IAM auth | LOW |
| `graqle/server/app.py` | Load from Neptune instead of JSON | MEDIUM |
| `graqle/core/graph.py` | `from_neptune()` loader | LOW |
| `graqle/studio/app.py` | Pass Neptune state to Studio routes | LOW |
| `graqle/cli/commands/cloud.py` | `graq cloud push` to Neptune | NEW |
| Lambda deploy workflow | VPC config, Neptune endpoint env var | MEDIUM |
| Studio dashboard pages | Already use proxy routes — no change | NONE |
