<div align="center">

# 🐑 Cortex by evaOS

**Drop-in persistent memory engine for AI agents.**

[![Version](https://img.shields.io/badge/version-v1.4.0-blueviolet.svg)](https://github.com/100yenadmin/electric-sheep/releases/tag/v1.4.0)
[![PyPI version](https://img.shields.io/pypi/v/evaos.svg)](https://pypi.org/project/evaos/)
[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://python.org)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Tests](https://img.shields.io/github/actions/workflow/status/100yenadmin/electric-sheep/ci.yml?label=tests)](https://github.com/100yenadmin/electric-sheep/actions)

*Your agent wakes up tomorrow and still knows who it is.*

</div>

---

## What is Cortex?

Cortex is a **drop-in persistent memory engine** for AI agents. It solves the three failure modes of naive AI memory: identity drift, junk accumulation, and no lifecycle.

**Key differentiators:**

- 🪨 **Cornerstones** — always-on identity memories that inject at the start of every session, bypassing semantic retrieval. Your agent's core identity is guaranteed to be present.
- 🌙 **Dream Cycles** — async memory consolidation via a persistent CircadianEngine. Memories are scored, promoted, decayed, and synthesized while your agent sleeps.
- 👥 **Multi-tenant** — full `owner_id` isolation. One Cortex instance serves multiple users with complete memory separation.
- 🔌 **Provider-agnostic** — pluggable LLM and embedding providers: Venice, Mistral, Google, Voyage. Switch without re-embedding.

**Production-ready out of the box:**

- 🚀 One-command Fly.io deploy via `cortex-fly.toml`
- 💾 Litestream continuous backup + GitHub Actions daily snapshot
- 📡 Sentry error tracking + performance spans

---

## What's New in v1.4.0

> v1.4.0 is the **memory scoping + entity intelligence** release.

- 🔭 **Entity-scoped retrieval** — filter memories by `entity_id` and `entity_type` on retrieve, search, and browse endpoints. Your agent can now recall memories about specific people, projects, or concepts (#923, #931)
- ⏳ **Temporal claims** — memories have `valid_from`/`valid_to` lifecycle. Expired claims are excluded by default; include with `include_expired=true`. New `GET /memories/timeline` endpoint for subject history (#925, #931)
- 👥 **Entity visibility lifecycle** — entities progress through `extracted → promoted → active` states based on `connection_count`. Dashboard-ready entity intelligence (#933, #938)
- 🎭 **Peer promotion** — peers promoted to canonical actor identities with `peer_type`, `status`, `permissions_json`, and `connection_count`. First-class support for multi-actor systems (#932, #941)
- 🧹 **Extraction quality** — predicate filter removes junk claims (code artifacts, bot names, ephemeral actions) + prompt tuning for better extraction (#929, #940)
- 🌐 **Cortex-proxy** — Supabase edge function for CORS-free dashboard access with path traversal protection (#928, #937)

**Previous highlights (v1.3.x):**

- 🦛 **Hippo LCM** — Lossless Context Management: compaction, summarization, retrieval (#869–#871)
- 🪨 **Cornerstone always-on injection** — identity cornerstones inject unconditionally every session (#733)
- 🔑 **Static API key auth** — `X-Cortex-API-Key` header replaces JWT complexity (#737)
- 🌙 **Dream engine persistent scheduler** — CircadianEngine starts at server boot (#776)

See the full [CHANGELOG](CHANGELOG.md) for all fixes and changes.

---

## Architecture

Cortex processes memory through a **five-layer pipeline**:

```
Capture → Extract → Reconcile → Retrieve → Dream
```

| Layer | What It Does |
|-------|-------------|
| **Capture** | Intercepts conversation turns and session events |
| **Extract** | LLM-powered claim extraction with noise filtering |
| **Reconcile** | Smart conflict resolution: ADD / UPDATE / SUPERSEDE / NOOP |
| **Retrieve** | Hybrid BM25 + vector search with Reciprocal Rank Fusion |
| **Dream** | Async consolidation, decay, synthesis, and contradiction resolution |

### Cornerstone Layer

Cornerstones are **identity memories that inject first, every session**. They bypass semantic retrieval entirely — they are always present, always loaded, and protected from accidental deletion. This is the foundation of identity preservation.

```
Session Start
     │
     ▼
┌─────────────────────┐
│  Cornerstone Load   │  ← unconditional, always first
│  (identity anchors) │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│  Semantic Retrieval │  ← query-dependent context
│  (BM25 + vector)    │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│  Context Injection  │
│  (system prompt)    │
└─────────────────────┘
```

### Provider System

Cortex uses a pluggable provider system for both LLM and embedding operations:

| Capability | Default Provider | Alternatives |
|---|---|---|
| LLM extraction | Venice (`mistral-small-3-2-24b-instruct`) | Google, OpenAI, Anthropic |
| Embeddings | Venice BGE-M3 (1024-dim) | Voyage, Google, OpenAI |
| Reranking | Voyage | — |

Switch providers at runtime via config or env vars — no re-embedding required.

### Full Pipeline

```
 ┌──────────────────── SESSION LIFECYCLE ────────────────────┐
 │                                                           │
 │  WAKE                    MID-TURN                  SLEEP  │
 │  ════                    ════════                  ═════  │
 │                                                           │
 │  ┌─────────┐  ┌──────────────┐  ┌────────────────────┐   │
 │  │  Wake   │  │   Retrieve   │  │   Sleep Pipeline   │   │
 │  │ Pipeline│  │   Pipeline   │  │  S1 Consolidation  │   │
 │  │ W1→W4   │  │  BM25+Vector │  │  S2 Promotion      │   │
 │  └────┬────┘  │  +RRF+Rerank │  │  S3 Handoff        │   │
 │       │       └───────┬──────┘  │  S4 Carry-Forward   │   │
 │       ▼               │         └────────────────────┘   │
 │  ┌─────────┐          │                ▲                  │
 │  │  Boot   │          ▼                │                  │
 │  │ Loader  │  ┌───────────────┐  ┌─────┴──────┐          │
 │  │(cornerstones│ │Task-Mode   │  │ Circadian  │          │
 │  │  identity│  │Classifier   │  │ Engine     │          │
 │  │  prefs) │  └───────┬───────┘  │ (dream,   │          │
 │  └────┬────┘          │          │  deep dream│          │
 │       │               ▼          │  decay)    │          │
 │       │       ┌───────────────┐  └────────────┘          │
 │       │       │Retrieval      │                           │
 │       │       │Policy Engine  │                           │
 │       │       └───────┬───────┘                           │
 │       │               │                                   │
 │       │               ▼                                   │
 │       │       ┌───────────────┐  ┌───────────────┐       │
 │       │       │   Context     │  │ Re-Retrieval  │       │
 │       │       │   Compiler    │◄─│ (topic shift,  │       │
 │       │       │ (bundle +     │  │  new entities) │       │
 │       │       │  compress)    │  └───────────────┘       │
 │       │       └───────┬───────┘                           │
 │       │               │                                   │
 │       │               ▼                                   │
 │       │       ┌───────────────┐                           │
 │       └──────►│  Injection    │                           │
 │               │  Slot Manager │                           │
 │               └───────────────┘                           │
 └───────────────────────────────────────────────────────────┘
```

---

## Install

```bash
pip install evaos
```

With vector search (recommended):
```bash
pip install "evaos[vec]"
```

All extras (LLM providers, HTTP API, MCP server):
```bash
pip install "evaos[all]"
```

---

## Quick Start

### Environment Variables

```bash
# Required
export CORTEX_API_KEY=your-secret-key

# Provider (Venice — default)
export VENICE_API_KEY=your-venice-key
export CORTEX_EMBEDDING_MODEL=text-embedding-bge-m3
export CORTEX_PROVIDER_LLM_MODEL=mistral-small-3-2-24b-instruct

# Optional: Voyage reranking
export VOYAGE_API_KEY=your-voyage-key
```

### CLI

```bash
# Initialize a new memory store
evaos init

# Teach it something
evaos remember "Andrew is the founder of 100Yen Org. He lives in Bangkok."

# Ask it later
evaos recall "Where does Andrew live?"
# → Andrew lives in Bangkok. He is the founder of 100Yen Org.

# Check config
evaos config show

# Validate provider config
evaos config validate
```

### Python SDK

```python
import asyncio
from evaos import Cortex

async def main():
    cortex = Cortex(db_path="my_memory.db")
    await cortex.initialize()

    # Start a session — cornerstones inject automatically
    await cortex.wake(session_id="session_001")

    # Store information — claim extraction happens automatically
    await cortex.remember("Andrew is the founder of 100Yen Org. He lives in Bangkok.")

    # Retrieve relevant memories for an LLM prompt
    context = await cortex.retrieve("Where does Andrew live?")
    print(context.context_block)
    # → "Andrew lives in Bangkok. He is the founder of 100Yen Org."

    # End the session — dream cycle consolidates
    await cortex.sleep(session_id="session_001")

asyncio.run(main())
```

### HTTP API

```bash
# Start the server
evaos serve --port 8420

# Store a memory
curl -X POST http://localhost:8420/api/v1/remember \
  -H "X-Cortex-API-Key: your-secret-key" \
  -H "Content-Type: application/json" \
  -d '{"content": "Andrew founded 100Yen Org in Bangkok."}'

# Retrieve memories
curl http://localhost:8420/api/v1/retrieve?query=Andrew \
  -H "X-Cortex-API-Key: your-secret-key"
```

### Deploy to Fly.io

```bash
# Clone and deploy
git clone https://github.com/100yenadmin/electric-sheep.git
cd electric-sheep
fly launch --config cortex-fly.toml
fly secrets set CORTEX_API_KEY=your-secret-key VENICE_API_KEY=your-venice-key
fly deploy
```

### Docker

```bash
git clone https://github.com/100yenadmin/electric-sheep.git
cd electric-sheep
export CORTEX_API_KEY=your-secret-key
export VENICE_API_KEY=your-venice-key
docker compose up -d
# HTTP API at http://localhost:8420
```

---

## Features

### 🧠 Memory Pipeline

| Layer | What It Does |
|-------|-------------|
| **Extraction** | LLM-powered claim extraction with noise filtering (skips "changed line 47 of auth.py") |
| **Entity Resolution** | Fuzzy deduplication — "Andrew", "andrew", "@andrew" → same person |
| **Reconciliation** | Smart conflict resolution: ADD / UPDATE / SUPERSEDE / NOOP |
| **Cornerstones** | Always-on identity anchors — inject first, every session, no keywords needed |
| **Token Budget** | Assembles memory context within configurable token ceilings |

### 🌙 Dream Cycles

A persistent `CircadianEngine` starts at server boot and runs async memory consolidation:
- **Standard Dream** — Ebbinghaus decay, contradiction resolution, cross-session synthesis
- **Deep Dream** — Generative memory synthesis; Gen1 Soul restoration (new in v1.3.0)
- **Extraction Failure Queue** — Failed extractions retried automatically in dream cycles

### 🔑 Auth & Multi-Tenancy

- **Static API key** — `X-Cortex-API-Key` header. Simple, secure, no JWT complexity.
- **Multi-tenant** — `owner_id` resolved per-request from API key. Complete memory isolation between users.
- **Soft-delete** — Deleted memories retained for 90 days, restorable via API. Dream cycle purges after expiry.

### 📋 Commitments & Open Loops

First-class tracking of agent commitments and unresolved threads. Both inject into session boot context for continuity across sessions.

### 💬 Dialectic Engine

Ask your memory natural-language questions. Fast path for simple lookups, agentic multi-step QueryPlanner for complex reasoning across your memory graph.

### 🔍 Hybrid Retrieval

BM25 full-text search + vector similarity with Reciprocal Rank Fusion. Optional Voyage reranking for high-stakes queries.

### 📡 Observability

- **Sentry** — error tracking + performance spans in wake pipeline and retrieval
- **Health endpoint** — version, dream cycle timestamp, system status
- **Metrics** — pipeline counters, histograms, gauges, quality gates

### 🔌 Five Interfaces

| Interface | Use Case |
|-----------|----------|
| **CLI** | `evaos remember`, `evaos recall`, `evaos ask`, `evaos dream`, `evaos config` |
| **HTTP API** | FastAPI REST server — `evaos serve` |
| **MCP Server** | Model Context Protocol for agent frameworks — `evaos mcp` |
| **TypeScript SDK** | `@evaos/client` — typed client for Node.js applications |
| **OpenClaw Plugin** | `cortex` — drop-in memory plugin for OpenClaw agents |

---

## OpenClaw Plugin

Cortex ships an [OpenClaw](https://openclaw.com) plugin (`cortex`) that bridges your agent's memory to a running Cortex instance.

```jsonc
// openclaw.plugin.json (included in openclaw-plugin/)
{
  "id": "cortex",
  "kind": "memory"
  // configSchema: cortexUrl, apiKey, ownerId, autoRecall, autoCapture, ...
}
```

**What it does:**
- **before_agent_start** — loads cornerstones unconditionally, then retrieves semantic context
- **agent_end** — captures conversation as new memories (fire-and-forget)
- **session_start / session_end** — calls wake/sleep for session lifecycle

**Tools exposed:** `cortex_search`, `cortex_remember`, `cortex_forget`, `cortex_update_commitment`, `cortex_resolve_open_loop`

See [`openclaw-plugin/`](openclaw-plugin/) for full source and configuration schema.

---

## Configuration

```toml
# cortex.toml

[cortex]
storage_backend = "sqlite"

[cortex.storage]
db_path = "cortex.db"

[cortex.extraction]
model = "mistral-small-3-2-24b-instruct"
coding_noise_filter = true

[cortex.embedding]
model = "text-embedding-bge-m3"
provider = "venice"

[cortex.cornerstones]
max_count = 7
drift_threshold = 0.05

[cortex.circadian]
dream_cycle_enabled = true
dream_cycle_hour_utc = 8
deep_dream_enabled = true

[cortex.deprecation]
forgetting_curve = "ebbinghaus"
soft_delete_retention_days = 90
```

See [`docs/product/guides/configuration.md`](docs/product/guides/configuration.md) for the full reference.

---

## Documentation

| Document | Path |
|----------|------|
| Getting Started | [`docs/product/guides/getting-started.md`](docs/product/guides/getting-started.md) |
| HTTP API Reference | [`docs/product/api/http-reference.md`](docs/product/api/http-reference.md) |
| Configuration Guide | [`docs/product/guides/configuration.md`](docs/product/guides/configuration.md) |
| Fly.io Deployment | [`docs/product/ops/fly-deployment-guide.md`](docs/product/ops/fly-deployment-guide.md) |
| Architecture Overview | [`docs/product/architecture/overview.md`](docs/product/architecture/overview.md) |
| Changelog | [`CHANGELOG.md`](CHANGELOG.md) |

---

## Python SDK Reference

```python
from evaos import Cortex

# Default — Venice provider, stores in cortex.db
cortex = Cortex()

# Customised
cortex = Cortex(
    db_path="~/my_app/memory.db",
    profile="companion",
    llm_provider="venice",
    extract_model="mistral-small-3-2-24b-instruct",
    embed_model="text-embedding-bge-m3",
)
```

### Core Methods

| Method | Description |
|--------|-------------|
| `await cortex.initialize()` | Initialize storage + apply migrations |
| `await cortex.wake(session_id)` | Start session, load cornerstones unconditionally |
| `await cortex.remember(content)` | Store text; extraction is automatic |
| `await cortex.retrieve(query)` | Hybrid BM25 + vector search, returns context block |
| `await cortex.ask(query)` | Dialectic engine — ask your memory questions |
| `await cortex.sleep(session_id)` | End session, trigger consolidation |
| `await cortex.dream()` | Run dream cycle manually |
| `await cortex.seal_cornerstone(label, content)` | Pin an identity anchor |
| `await cortex.check_drift()` | Drift scores for all cornerstones |
| `await cortex.health()` | System health dict |
| `await cortex.stats()` | Memory counts (claims, entities, cornerstones, sessions) |
| `await cortex.export(format="json")` | Export all active memories |

---

## TypeScript SDK

```bash
npm install @evaos/client
```

```typescript
import { CortexClient } from '@evaos/client';

const client = new CortexClient({
  baseUrl: 'http://localhost:8420',
  apiKey: 'your-api-key',
});

await client.remember('Andrew is the founder of 100Yen Org.');
const results = await client.recall('Who is Andrew?');
```

See [`sdks/typescript/`](sdks/typescript/) for full documentation.

---

## CLI Reference

```bash
evaos init [--profile companion|coding|enterprise]
evaos remember "text"
evaos recall "query"
evaos ask "question"
evaos ask "question" --agentic
evaos cornerstones list
evaos cornerstones seal --label "name" --content "text"
evaos dream
evaos config show
evaos config set-provider
evaos config validate
evaos config export
evaos config apply-fly
evaos stats
evaos health
evaos export [--format json|sql]
evaos serve [--port 8000]
evaos mcp
evaos backup [--output path]
```

---

## Project Structure

```
cortex/                 # Python modules (~52K lines)
├── core/               # Extraction, entity resolution, reconciliation, retrieval,
│                       #   working memory, token budget, task-mode, boot loader
├── storage/            # SQLite adapter with FTS5 + vector support, v3-v16 migrations
├── brain_graph/        # Document memory graph with file watching
├── circadian/          # Dream engine: CircadianEngine, Deep Dream, CuriosityEngine
├── sleep/              # Staged sleep pipeline: S1→S2→S3→S4
├── capture/            # Wake pipeline: W1-W4 activation routing
├── commitments/        # Commitment/open loop tracking
├── deprecation/        # Ebbinghaus decay + soft-delete pipeline
├── identity/           # Cornerstone guardian + drift calculator
├── config/             # TOML config + `evaos config` CLI
├── api/                # FastAPI HTTP server (85+ endpoints)
├── integrations/       # MCP server + plugin interface
├── dialectic/          # Dialectic engine (ask your memory)
├── peers/              # Peer modeling (relationship tracking)
├── events/             # Event emitter + webhook dispatcher
├── llm/                # Multi-provider LLM abstraction (Venice, Google, OpenAI)
├── cli.py              # Click-based CLI
├── sdk.py              # High-level Cortex class
└── types.py            # Shared dataclasses and types

dashboard/              # 8-page web UI (SPA)
sdks/typescript/        # @evaos/client TypeScript SDK
openclaw-plugin/        # cortex OpenClaw plugin (TypeScript)
deploy/                 # cortex-fly.toml, Litestream config, startup scripts
```

---

## Benchmarks

```bash
# Full suite
python -m benchmarks.run_benchmarks

# Skip slow 10K retrieval
python -m benchmarks.run_benchmarks --skip-retrieval
```

Results saved to `benchmarks/results/latest.json`. Nightly CI runs LoCoMo eval at 02:00 UTC.

---

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for dev setup, testing, and PR guidelines.

---

## License

[MIT](LICENSE) — 100Yen Org
