# TrustMem — Zero Trust Memory for AI Agents

**sator Inc.** | Python SDK | BSL-1.1

---

TrustMem is a memory engine for AI agents built on **Zero Trust Memory (ZTM) v2** — a confidence-based trust model where LLM evaluates reliability at write time.

## Key Features

| Feature | Description |
|---|---|
| **ZTM v2 Confidence Model** | LLM-judged confidence (high/medium/low) assigned at write time. No dynamic recalculation |
| **Contradiction Detection** | Automatic CONTRADICT detection with winner/loser logic — winner gets `high`, loser gets demoted |
| **Fact Verification** | LLM verifies factual claims against its knowledge (e.g., political leaders, capitals) |
| **Coexistence Rules** | Multiple preferences/attributes coexist without false contradictions |
| **User/Assistant Separation** | Only user-stated facts are extracted; assistant responses are filtered |
| **Content-Addressable Store** | SHA-256 content hashing for integrity verification |
| **Knowledge Graph** | Entity-relation graph with confidence-based entity tracking |
| **User Scoping** | user_id scoping with shared knowledge via `visibility="organization"` |
| **Conversation Extraction** | Automatic memory extraction from conversation messages via LLM |
| **Hybrid Search** | Vector + keyword (inverted index with trigram-like tokenization) + RRF fusion |
| **Memory Types** | `semantic` (default) for facts, `document` for RAG/document chunks |

---

## Install

```bash
# Embedded SDK (OpenAI included as required dependency)
pip install trustmem

# Embedded SDK + FastAPI server
pip install "trustmem[server]"

# HTTP client only (separate package, no trustmem dependency)
pip install trustmem-client
```

## Quick Start

### Configuration

Create a `.env` file:

```
OPENAI_API_KEY=sk-...
```

### Embedded SDK

```python
from trustmem import TrustMem

m = TrustMem()

# Add memories from conversation messages (LLM auto-extraction)
messages = [
    {"role": "user", "content": "Alex likes sushi"},
    {"role": "assistant", "content": "Noted!"},
]
m.add(messages, user_id="user-123")

# Search (hybrid: vector + keyword + RRF fusion)
result = m.search("food preferences", user_id="user-123")
print(result)  # {"results": [...], "relations": [...]}

# Shared knowledge (user_id=NULL)
m.add(messages, visibility="organization")
result = m.search("food preferences", visibility="organization")

# List all memories (scope-aware)
all_memories = m.get_all(user_id="user-123")

# Admin: list all memories (ignores scope)
all_memories = m.list_all()

# Delete
m.delete(memory_id)
```

### HTTP Server + Client

```bash
# Start server
python -m trustmem
# or: trustmem-server
```

```python
from trustmem_client import TrustMemClient

client = TrustMemClient(base_url="http://localhost:8080")
client.add([{"role": "user", "content": "Hello world"}], user_id="user-123")
result = client.search("Hello", user_id="user-123")
```

---

## Architecture

```
Data Model:
  user_id (data owner)
    ├── Memory (semantic / document)
    ├── Entity
    └── Edge
  user_id=NULL
    └── shared knowledge (via visibility="organization")

Embedded (local):
  TrustMem (29 methods)  ->  PersistentHandler  ->  SQLite/Qdrant/Neo4j
    ├── Core CRUD: add, search, search_raw, get, get_all, list_all, delete
    └── Management: verify, stats, reset, ...

Remote (via server):
  TrustMemClient (5 methods)  --HTTP-->  FastAPI Server  ->  TrustMem
    └── Core CRUD only: add, search, get_all, delete, close
```

### Access Layers

| Package | Methods | Use Case |
|---|---|---|
| `pip install trustmem` | 29 methods | Embedded engine, all operations |
| `pip install "trustmem[server]"` | + FastAPI server | Remote access via HTTP |
| `pip install trustmem-client` | 5 methods | Lightweight HTTP client |

### Storage Backends

| Component | Default | Alternative | Env Var |
|---|---|---|---|
| Database | SQLite | — | `DATABASE_URL` |
| Vector Search | SQLite | Qdrant | `VECTOR_STORE` |
| Graph Store | SQLite | Neo4j | `GRAPH_STORE` |
| AI (required) | OpenAI | Azure / Compatible | `EMBED_PROVIDER` |

---

## SDK Methods (29 total)

Scoped methods require either `user_id` or `visibility="organization"` (mutually exclusive). `visibility="organization"` is syntactic sugar for `user_id=NULL` (shared knowledge).

| Category | Method | Scoped | Description |
|---|---|---|---|
| **Core** | `add(data)` | Yes | Add memories from conversation messages (LLM auto-extraction) |
| **Core** | `search(query)` | Yes | Hybrid search (vector + keyword + RRF). `search_mode`: `"hybrid"` / `"vector"` / `"keyword"`. Returns `{"results": [...], "relations": [...]}` |
| **Core** | `search_raw(query)` | Yes | Pure cosine similarity search (no ZTM ranking) |
| **Core** | `get(memory_id)` | — | Get a single memory by ID |
| **Core** | `get_all()` | Yes | Get all memories + relations (scope-aware) |
| **Core** | `list_all()` | — | Admin: get all memories (ignores scope) |
| **Core** | `delete(memory_id)` | — | Delete a memory |
| **Core** | `close()` | — | Release resources (supports `with` statement) |
| **Raw** | `add_raw(content)` | Yes | Add memory directly, bypassing LLM pipeline |
| **Raw** | `update_raw(memory_id, content)` | — | Version management (supersede + derived_from) |
| **ZTM** | `verify(memory_id)` | — | Integrity verification (content_hash check) |
| **ZTM** | `verify_all()` | — | Batch verify all memories |
| **Investigation** | `audit_trail(memory_id)` | — | Get audit log for a memory |
| **Investigation** | `contradictions(memory_id)` | — | Get contradiction relationships |
| **Investigation** | `provenance(memory_id)` | — | Get provenance links (derived_from, etc.) |
| **Audit Chain** | `verify_audit_chain()` | — | Verify audit trail hash chain integrity |
| **Audit Chain** | `verify_access_chain()` | — | Verify access log hash chain integrity |
| **Access Log** | `access_log()` | — | Query read access log |
| **Access Log** | `purge_access_log(before)` | — | Purge old access log records |
| **Graph** | `add_relation(subj, pred, obj)` | Yes | Add entity relation |
| **Graph** | `search_relations(entity)` | Yes | Search relations |
| **Graph** | `get_graph(entity)` | Yes | Get subgraph |
| **Stats** | `stats()` | — | Memory statistics (includes confidence_distribution) |
| **Stats** | `entity_stats()` | — | Entity statistics |
| **Maintenance** | `backfill_hashes()` | — | Backfill content hashes |
| **Maintenance** | `cleanup_audit_trail(before)` | — | Count old audit trail records |
| **Maintenance** | `cleanup_expired()` | — | Delete expired memories (TTL) |
| **GDPR** | `forget(entity)` | — | Delete all data related to an entity |
| **Development** | `reset()` | — | Clear all data (development only) |

---

## ZTM v2 — Confidence-Based Trust Model

Confidence (high/medium/low) is assigned at write time by LLM screening and updated only at CONTRADICT time.

### Confidence Levels

| Confidence | Criteria |
|---|---|
| `high` | Specific, plausible, clearly stated, AND factually correct |
| `medium` | Vague, uncertain, hearsay, or cannot verify (default) |
| `low` | Factually wrong, outdated, or self-contradictory |

### CONTRADICT + Winner/Loser

When LLM detects a contradiction between new and existing facts:
- Both memories are kept; a `contradicts` provenance edge is recorded
- LLM determines `winner` (`new` / `existing` / `unknown`)
- Winner's confidence → `high`; Loser's confidence → `loser_confidence` (LLM-judged)

```
"Abe is PM" (existing) + "Kishida is PM" (new)
  → CONTRADICT, winner: "new"
  → Kishida: confidence="high", Abe: confidence="low"
```

### Extraction Pipeline

| Stage | Description |
|---|---|
| Stage 1 | Extract atomic facts from conversation + evaluate confidence |
| Stage 2 | MinHash pre-filter (Jaccard >= 0.98 → auto-NOOP) |
| Stage 3 | Batch conflict resolution (ADD / CONTRADICT / NOOP) |

### Freshness (informational only)

Search results include `freshness` (days since `last_evaluated_at`) and `last_evaluated_at`. No penalty applied — provided as context for the application.

---

## HTTP Server

FastAPI server exposing core CRUD + admin endpoints. All endpoints use `user_id` query parameter.

| HTTP | Path | Auth | Description |
|---|---|---|---|
| POST | `/v1/memories` | Yes (if configured) | Add memory |
| POST | `/v1/memories/search` | Yes (if configured) | Search memories |
| GET | `/v1/memories` | Yes (if configured) | List all memories |
| DELETE | `/v1/memories/{memory_id}` | Yes (if configured) | Delete a memory |
| GET | `/v1/memories/{memory_id}` | Yes (if configured) | Get a single memory |
| GET | `/health` | No | Health check |
| GET | `/v1/version` | Yes (if configured) | Server version |
| GET | `/playground` | No | Interactive playground UI |

### Authentication

Set `TRUSTMEM_API_KEY` to enable Bearer token auth on all `/v1/*` endpoints. Unset = no auth (development).

---

## Environment Variables

### Core

| Variable | Default | Description |
|---|---|---|
| `OPENAI_API_KEY` | — | OpenAI API key (required for `openai` provider) |
| `DATABASE_URL` | `~/.trustmem/trustmem.db` | SQLite database path |
| `EMBED_PROVIDER` | `openai` | `openai` \| `azure` \| `openai_compat` |
| `LLM_PROVIDER` | `openai` | `openai` \| `azure` \| `openai_compat` |
| `EMBED_MODEL` | `text-embedding-3-small` | Embedding model name |
| `LLM_MODEL` | `gpt-4o-mini` | LLM model name |
| `EMBED_DIM` | `1536` | Embedding vector dimension |

### Server

| Variable | Default | Description |
|---|---|---|
| `HOST` | `0.0.0.0` | Server bind address |
| `PORT` | `8080` | Server port |
| `TRUSTMEM_API_KEY` | — | API key for Bearer auth (unset = no auth) |
| `CORS_ORIGINS` | `localhost` | Comma-separated allowed CORS origins |

### Client (trustmem-client)

| Variable | Default | Description |
|---|---|---|
| `TRUSTMEM_BASE_URL` | `http://localhost:8080` | Server URL for TrustMemClient |
| `TRUSTMEM_API_KEY` | — | API key for Bearer auth |

### Storage Backends

| Variable | Default | Description |
|---|---|---|
| `VECTOR_STORE` | `sqlite` | `sqlite` \| `qdrant` |
| `GRAPH_STORE` | `sqlite` | `sqlite` \| `neo4j` |
| `QDRANT_URL` | `http://localhost:6333` | Qdrant server URL |
| `QDRANT_API_KEY` | — | Qdrant API key |
| `QDRANT_COLLECTION` | `trustmem` | Qdrant collection name |
| `NEO4J_URI` | `bolt://localhost:7687` | Neo4j connection URI |
| `NEO4J_USER` | `neo4j` | Neo4j username |
| `NEO4J_PASSWORD` | — | Neo4j password |
| `NEO4J_DATABASE` | `neo4j` | Neo4j database name |

### Azure OpenAI

| Variable | Default | Description |
|---|---|---|
| `AZURE_OPENAI_API_KEY` | — | Azure OpenAI API key |
| `AZURE_OPENAI_ENDPOINT` | — | Azure OpenAI endpoint URL |
| `AZURE_API_VERSION` | `2024-02-01` | Azure API version |

### OpenAI Compatible

| Variable | Default | Description |
|---|---|---|
| `OPENAI_BASE_URL` | — | Base URL for OpenAI-compatible API |

All variables can be set in a `.env` file (auto-loaded via python-dotenv).

---

## File Structure

```
trustmem/
├── README.md
├── LICENSE
├── pyproject.toml
├── .env.example
├── src/
│   └── trustmem/                        <- pip install trustmem
│       ├── __init__.py
│       ├── sdk.py                       <- TrustMem class (29 methods)
│       ├── server.py                    <- FastAPI server (core CRUD + admin endpoints)
│       ├── __main__.py                  <- python -m trustmem entrypoint
│       ├── config.py                    <- Environment config
│       ├── constants.py                 <- Shared validation constants
│       ├── object_store.py              <- Content-addressable store (SHA-256)
│       ├── ai/
│       │   └── client.py                <- AIClient (OpenAI/Azure/compatible)
│       ├── handler/
│       │   ├── protocol.py              <- HandlerProtocol
│       │   ├── persistent.py            <- PersistentHandler
│       │   └── factory.py               <- Handler factory
│       ├── extraction/
│       │   ├── pipeline.py              <- Memory extraction pipeline
│       │   ├── entities.py              <- Entity extraction helpers
│       │   └── prompts.py               <- LLM prompts (Stage 1/3)
│       ├── vectorstore/
│       │   ├── base.py                  <- VectorStore ABC
│       │   ├── sqlite.py               <- SQLite vector store
│       │   └── qdrant.py               <- Qdrant vector store
│       ├── graphstore/
│       │   ├── base.py                  <- GraphStore ABC
│       │   ├── sqlite.py               <- SQLite graph store
│       │   └── neo4j.py                <- Neo4j graph store
│       ├── ztm/
│       │   ├── pipeline.py              <- ZTM v2 (confidence model definition)
│       │   └── constants.py             <- ZTM constants (CONFIDENCE_LEVELS, DEFAULT_CONFIDENCE)
│       └── storage/
│           └── sqlite_store.py          <- SQLite store
├── tests/                               <- 623 tests (<2s)
├── trustmem-client/                     <- pip install trustmem-client
│   ├── pyproject.toml
│   ├── src/
│   │   └── trustmem_client/
│   │       ├── __init__.py
│   │       └── client.py               <- TrustMemClient (5 methods)
│   └── tests/
│       └── test_client.py               <- 21 tests
└── docs/
    ├── design/
    │   └── TrustMem_Design.md           <- Full design specification
    └── guides/
        └── Quickstart.md                <- Quickstart guide
```

---

## Testing

```bash
# SDK + server tests (623 tests)
python -m pytest tests/ -x -q

# Client tests
cd trustmem-client
pip install -e ".[dev]"
python -m pytest tests/ -x -q
```

---

## Tech Stack

| Item | Detail |
|---|---|
| Language | Python 3.12 |
| Embedding Model | `text-embedding-3-small` (OpenAI / Azure / compatible) |
| LLM | `gpt-4o-mini` (OpenAI / Azure / compatible) |
| Graph Engine | NetworkX 3.3 |
| Tests | 623 tests (<2s) |

---

## Documentation

| Document | Description |
|---|---|
| [`docs/design/TrustMem_Design.md`](docs/design/TrustMem_Design.md) | Full design specification |
| [`docs/guides/Quickstart.md`](docs/guides/Quickstart.md) | Quickstart guide |

---

## License

Business Source License 1.1 (BSL-1.1)

- **Free to use**: embed in your own apps, use internally, develop and test
- **One restriction**: you may not offer TrustMem as a hosted memory service to third parties
- **Converts to Apache-2.0** on 2029-04-01

See [LICENSE](LICENSE) for full terms.

---

*TrustMem — Because AI agents deserve trustworthy memory.*
