Metadata-Version: 2.4
Name: claude-engram
Version: 0.0.1a1
Summary: Graph intelligence engine for Claude Code — semantic search, knowledge graphs, and decision lineage
Project-URL: Homepage, https://github.com/flavio-bongiovanni/claude-starchart
Project-URL: Repository, https://github.com/flavio-bongiovanni/claude-starchart
Author-email: Flavio Bongiovanni <flavio@earendil.com.br>
License-Expression: MIT
Keywords: ai,claude,context,decision-tracking,graph,knowledge-graph,llm,memory,temporal
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Requires-Dist: cachetools>=5.5.0
Requires-Dist: fastembed>=0.4.0
Requires-Dist: mcp<2.0.0,>=1.0.0
Requires-Dist: numpy>=1.26.0
Requires-Dist: scikit-learn>=1.3
Requires-Dist: sqlite-vec>=0.1.6
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.24.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.8.0; extra == 'dev'
Requires-Dist: types-cachetools>=5.5.0; extra == 'dev'
Description-Content-Type: text/markdown

# Claude Engram

Graph intelligence engine for Claude Code — semantic search, temporal knowledge graphs, and decision lineage tracking powered by SQLite-vec.

Part of the [Claude Starchart](https://github.com/flavio-bongiovanni/claude-starchart) monorepo.

## Capability Layers

| Layer | Tools | Description |
|-------|-------|-------------|
| **Memory** | store, search, recent, delete, expand, batch | Semantic store/recall with category TTL |
| **Graph** | add_relation, get_relations, traverse_graph | Knowledge graph with typed edges |
| **Temporal** | timeline, decision_chain, record_outcome | Chronological navigation and decision lineage |
| **Intelligence** | consolidate, cache_stats, cache_clear | HDBSCAN dedup, progressive disclosure, L2 cache |

Plus: hybrid search (FTS5 + vector + RRF fusion), project scoping, Unix socket IPC.

## Engram vs Claude Code Native Memory

Claude Code v2.1.32+ includes built-in memory. Here's when to use each:

| Capability | CC Native | Engram |
|------------|-----------|--------|
| Basic store/recall | Yes (automatic) | Yes |
| Semantic search | Basic | Advanced (hybrid FTS5+vector+RRF) |
| Progressive disclosure | No | Yes (index/summary/full — 10x token savings) |
| Knowledge graph | No | Yes (typed relations, traversal) |
| Decision chains | No | Yes (temporal lineage tracking) |
| Consolidation | No | Yes (HDBSCAN clustering dedup) |
| Temporal queries | No | Yes (timeline, point-in-time) |
| L2 semantic cache | No | Yes (query dedup, <50ms) |
| Cross-project | No | Yes (proj_tag scoping + global) |
| MCP accessible | No | Yes (any MCP client) |
| Category TTL | No | Yes (5min - permanent) |
| Setup | Zero (built-in) | Daemon + systemd |

**Recommendation**: Use CC native memory for simple facts. Use Engram when you need
search quality, decision tracking, graph relations, or cross-session intelligence.

## Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                        Claude Code                               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │  MCP Server  │  │     CLI      │  │  Star Bridge │          │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘          │
└─────────┼─────────────────┼─────────────────┼───────────────────┘
          │                 │                 │
          │    JSON over Unix Socket          │
          │   /run/user/$UID/claude-engram.sock
          ▼                 ▼                 ▼
┌─────────────────────────────────────────────────────────────────┐
│                      MemoryDaemon                                │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │ Request      │  │  TTLCache    │  │  Periodic    │          │
│  │ Dispatcher   │  │  (sessions)  │  │  Cleanup     │          │
│  └──────┬───────┘  └──────────────┘  └──────────────┘          │
│         │                                                        │
│  ┌──────▼───────────────────────────────────────────┐           │
│  │              MemoryStorage                        │           │
│  │  ┌────────────┐  ┌────────────┐  ┌────────────┐  │           │
│  │  │  SQLite    │  │ sqlite-vec │  │  fastembed │  │           │
│  │  │  (WAL)     │  │ (vectors)  │  │  (MiniLM)  │  │           │
│  │  └────────────┘  └────────────┘  └────────────┘  │           │
│  │  ┌────────────┐  ┌────────────┐  ┌────────────┐  │           │
│  │  │  Graph     │  │ Semantic   │  │  FTS5      │  │           │
│  │  │  Engine    │  │ Cache (L2) │  │  (hybrid)  │  │           │
│  │  └────────────┘  └────────────┘  └────────────┘  │           │
│  └──────────────────────────────────────────────────┘           │
└─────────────────────────────────────────────────────────────────┘
```

## Installation

```bash
# From monorepo root
pip install -e packages/engram

# Or standalone
pip install claude-engram
```

### Systemd Service (recommended)

```bash
# Copy service file
cp platforms/linux/claude-engram.service ~/.config/systemd/user/

# Enable and start
systemctl --user enable claude-engram
systemctl --user start claude-engram

# Check status
systemctl --user status claude-engram
```

## Usage

### CLI

```bash
# Daemon control
claude-engram start              # Start daemon (foreground)
claude-engram stop               # Stop daemon
claude-engram status             # Check if running
claude-engram health             # Detailed health check

# Memory operations
claude-engram store "Decision: use PostgreSQL for persistence" --category decision
claude-engram search "database choice"
claude-engram recent --limit 5
claude-engram delete 42
claude-engram stats
```

### MCP Tools (20)

#### Memory Layer
| Tool | Description |
|------|-------------|
| `search_memories` | Semantic/hybrid/keyword search with progressive disclosure |
| `store_memory` | Store with category TTL and supersession tracking |
| `recent_memories` | Chronological list (no ranking) |
| `delete_memory` | Delete by ID (project-scoped) |
| `expand_memory` | Drill into a specific memory (full detail) |
| `get_memories_batch` | Retrieve multiple memories by ID |

#### Graph Layer
| Tool | Description |
|------|-------------|
| `add_relation` | Create typed edge between memories |
| `get_relations` | Query relations for a memory |
| `traverse_graph` | Multi-hop graph traversal with cycle prevention |

#### Temporal Layer
| Tool | Description |
|------|-------------|
| `timeline` | Navigate chronologically around an anchor memory |
| `decision_chain` | Follow decision lineage (resulted_in edges) |
| `record_outcome` | Link a decision to its measured outcome |

#### Intelligence Layer
| Tool | Description |
|------|-------------|
| `consolidate_memories` | HDBSCAN-based duplicate detection and cleanup |
| `cache_stats` | L2 semantic cache statistics |
| `cache_clear` | Clear L2 cache |

#### Counters
| Tool | Description |
|------|-------------|
| `get_counters` | Read named counters |
| `set_counter` | Set counter value |
| `update_counter` | Increment/decrement counter |
| `delete_counter` | Remove counter |

#### System
| Tool | Description |
|------|-------------|
| `health_check` | Daemon and database health diagnostics |

### Example: Claude Code MCP Config

```json
{
  "mcpServers": {
    "engram": {
      "type": "stdio",
      "command": "python",
      "args": ["-m", "claude_engram.mcp_server"]
    }
  }
}
```

## Memory Categories

Categories determine automatic TTL (time-to-live) for memories:

| Category | TTL | Use Case |
|----------|-----|----------|
| `lesson` | **Permanent** | Insights, patterns, best practices |
| `decision` | 7 days | Architectural choices, trade-offs |
| `interface` | 7 days | API discoveries, function signatures |
| `file_ownership` | 7 days | Who owns what file |
| `session_summary` | 7 days | End-of-session context |
| `checkpoint` | 4 hours | Progress markers, state snapshots |
| `query_cache` | 5 minutes | Temporary search results |
| `general` | Permanent | Legacy category (v1 compatibility) |

### TTL Tiers

```
PERMANENT ─────────────────────────────────────────► never expires
SESSION   ─────────────────────────────► 7 days
VOLATILE  ───────────────► 4 hours
EPHEMERAL ──► 5 min
```

## Search Modes

Engram supports three search strategies via the `search_mode` parameter:

| Mode | Engine | Use Case |
|------|--------|----------|
| `vector` (default) | Cosine similarity on MiniLM-L6-v2 embeddings | Semantic meaning |
| `hybrid` | FTS5 BM25 + vector with RRF fusion | Best of both worlds |
| `keyword` | FTS5 BM25 text matching | Exact term matching |

Results are ranked by a composite score combining similarity (0.6), salience (0.2), recency decay (0.1), and access frequency (0.1).

## Progressive Disclosure

Token-efficient retrieval in three levels:

| Level | Content | Tokens/result |
|-------|---------|---------------|
| `index` | IDs, category, similarity, token_count only | ~50 |
| `summary` (default) | Index + first sentence/100 chars | ~100 |
| `full` | Complete memory text + metadata | ~500 |

Use `search_memories(detail_level="index")` first, then `expand_memory` or `get_memories_batch` for drill-down.

## Graph Relations

The knowledge graph supports six typed edge relations:

| Relation | Meaning |
|----------|---------|
| `updates` | Source updates information in target |
| `supersedes` | Source replaces target (target marked inactive) |
| `supports` | Source corroborates target |
| `contradicts` | Source conflicts with target |
| `resulted_in` | Decision led to this outcome |
| `related_to` | General association |

Graph traversal uses recursive CTEs with automatic cycle prevention (delimiter-wrapped path tracking).

## Output Format

All MCP tools return XML-structured responses:

```xml
<memories count="3">
  <memory id="42" category="decision" similarity="87%" scope="proj:org/repo" created="2026-02-05T10:30:00">
    Decision: Use PostgreSQL for WebSocket state persistence
  </memory>
  <memory id="38" category="lesson" similarity="72%" scope="global" created="2026-02-04T15:20:00">
    Lesson: SQLite WAL mode enables concurrent reads during writes
  </memory>
</memories>
```

```xml
<memory_stored id="43" category="lesson" scope="proj:org/repo"></memory_stored>
```

```xml
<memory_deleted id="42" status="success"></memory_deleted>
```

## Configuration

### Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `ENGRAM_EMBED_MODEL` | `all-MiniLM-L6-v2` | Fastembed model name |
| `ENGRAM_EMBED_DIM` | `384` | Embedding vector dimension |
| `ENGRAM_EMBED_THREADS` | `4` | ONNX threads for embedding model |
| `ENGRAM_CLEANUP_INTERVAL` | `3600` | Seconds between expired memory cleanup |
| `ENGRAM_SOCKET_CHECK_INTERVAL` | `30` | Seconds between socket health checks |
| `ENGRAM_CHECKPOINT_LIGHT` | `40` | Tool calls before checkpoint (light profile) |
| `ENGRAM_CHECKPOINT_BALANCED` | `25` | Tool calls before checkpoint (balanced) |
| `ENGRAM_CHECKPOINT_ULTRA` | `15` | Tool calls before checkpoint (ultra) |

### Socket Location

Socket path is resolved in order:

1. `CLAUDE_ENGRAM_SOCKET` env var
2. `$XDG_RUNTIME_DIR/claude-engram.sock`
3. `/run/user/$UID/claude-engram.sock`
4. `/tmp/claude-engram.sock` (fallback)

### Database Location

```
~/.local/share/claude-engram/
├── memories.db          # SQLite database
├── memories.db-wal      # WAL file (auto-checkpointed)
├── memories.db-shm      # Shared memory
└── daemon.pid           # PID file
```

## Database Schema

```sql
-- memories table (v10 schema)
CREATE TABLE memories (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    content TEXT NOT NULL,
    category TEXT DEFAULT 'general',
    metadata TEXT DEFAULT '{}',
    created_at TEXT NOT NULL,
    updated_at TEXT NOT NULL,
    proj_tag TEXT DEFAULT '',
    ttl_tier TEXT DEFAULT 'permanent',
    expires_at TEXT,
    content_hash TEXT UNIQUE,
    summary TEXT,
    token_count INTEGER,
    superseded_by INTEGER,
    is_active INTEGER DEFAULT 1,
    valid_from TEXT,
    valid_to TEXT,
    access_count INTEGER DEFAULT 0
);

-- Vector storage (sqlite-vec)
CREATE VIRTUAL TABLE memory_vectors USING vec0(
    id INTEGER PRIMARY KEY,
    embedding FLOAT[384]
);

-- Knowledge graph edges
CREATE TABLE memory_relations (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    source_id INTEGER NOT NULL,
    target_id INTEGER NOT NULL,
    relation_type TEXT NOT NULL,
    metadata TEXT DEFAULT '{}',
    created_at TEXT NOT NULL,
    valid_from TEXT,
    valid_to TEXT,
    invalidated_at TEXT,
    UNIQUE(source_id, target_id, relation_type)
);

-- FTS5 full-text search (external content)
CREATE VIRTUAL TABLE memory_fts USING fts5(
    content, category,
    tokenize='porter unicode61 remove_diacritics 2',
    content='memories', content_rowid='id'
);

-- Counters
CREATE TABLE counters (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    key TEXT NOT NULL,
    value INTEGER NOT NULL DEFAULT 0,
    proj_tag TEXT NOT NULL DEFAULT '',
    updated_at TEXT NOT NULL,
    UNIQUE(key, proj_tag)
);

-- Consolidation tracking
CREATE TABLE consolidation_meta (
    key TEXT PRIMARY KEY,
    value TEXT NOT NULL
);
```

## Performance Tuning

For long-running deployments (12-24h+), configure thread limits to prevent memory growth:

```ini
# In systemd service or environment
Environment=ENGRAM_EMBED_THREADS=4
Environment=OMP_NUM_THREADS=4
Environment=HF_HUB_DISABLE_TELEMETRY=1
```

### Why This Matters

ONNX runtime uses arena allocators that don't release memory back to the OS. Without thread limits:
- 51 threads -> 8 threads (-86%)
- 227 MB -> 151 MB (-33%)

### Monitoring

```bash
# Process metrics via daemon
echo '{"action":"stats"}' | nc -U /run/user/$UID/claude-engram.sock | jq .process

# Returns:
{
  "memory_mb": 151.5,
  "threads": 2,
  "cache_entries": 0
}

# Systemd status
systemctl --user status claude-engram
```

## Project Scoping

Memories are scoped by `proj_tag` for multi-project isolation:

```python
# Auto-detected from environment
STAR_PROJECT_TAG    -> "my-project"
CLAUDE_PROJECT_DIR  -> "proj:project-name"

# Or explicit
store_memory(content="...", proj_tag="proj:my-project")
```

Search includes both project-specific and global memories by default.

## Counters

Simple key-value counters for tracking state:

```bash
# CLI
claude-engram counter set session.tools 0
claude-engram counter update session.tools 1  # increment
claude-engram counter get session.tools

# MCP
set_counter(key="session.tools", value=0)
update_counter(key="session.tools", delta=1)
get_counters()
```

## Troubleshooting

### Daemon won't start

```bash
# Check if already running
claude-engram status

# Check socket
ls -la /run/user/$UID/claude-engram.sock

# Check logs
journalctl --user -u claude-engram -f
```

### Socket deleted (orphan state)

The daemon auto-recovers from orphan sockets (checked every 30s). If manual recovery needed:

```bash
systemctl --user restart claude-engram
```

### High memory usage

Check thread count and configure limits:

```bash
# Current threads
cat /proc/$(pgrep -f claude_engram.daemon)/status | grep Threads

# Apply limits
export ENGRAM_EMBED_THREADS=4
systemctl --user restart claude-engram
```

## License

MIT
