# Observational Memory

Middleware that automatically compresses conversation history into structured observations for agents. Runs as invisible infrastructure — the agent just talks normally while observational memory maintains compressed context behind the scenes.

## Architecture

Two background agents process conversation history:

- **Observer**: Extracts structured observations (tasks, decisions, facts, preferences, context, progress) from messages when token thresholds are exceeded.
- **Reflector**: Compresses accumulated observations into dense reflections when observation tokens grow too large.

The conversation flow:

```
Messages (100k+ tokens) → Observer → Observations → Reflector → Reflections (~3k tokens)
```

## Usage

```python
from emdash_observational_memory import ObservationalMemory

memory = ObservationalMemory()

# Post-turn: feed new messages (auto-observes and reflects when thresholds hit)
await memory.process_turn("thread-1", [
    {"role": "user", "content": "..."},
    {"role": "assistant", "content": "..."},
])

# Pre-turn: get compressed context for system prompt injection
context = memory.get_prompt_context("thread-1")
```

### Agent Integration

```python
from emdash_observational_memory import (
    ObservationalMemory,
    build_enhanced_prompt,
    process_iteration_messages,
)

class MyAgent(BaseAgent):
    def __init__(self, ...):
        super().__init__(...)
        self._obs_memory = ObservationalMemory()

    def _build_system_prompt(self) -> str:
        base = super()._build_system_prompt()
        return build_enhanced_prompt(base, self._obs_memory, self.thread_id)

    def _on_iteration_complete(self, messages: list[dict]) -> None:
        super()._on_iteration_complete(messages)
        import asyncio
        asyncio.create_task(
            process_iteration_messages(self._obs_memory, self.thread_id, messages)
        )
```

## Configuration

Environment variables:

- `OPENAI_API_KEY` — API key for Observer/Reflector LLM calls
- `OBSERVATIONAL_MEMORY_STORAGE_DIR` — Storage directory (default: `.emdash/observational_memory`)
- `OBSERVATIONAL_MEMORY_OBSERVER_MODEL` — Observer model (default: `gpt-4o-mini`)
- `OBSERVATIONAL_MEMORY_REFLECTOR_MODEL` — Reflector model (default: `gpt-4o-mini`)
- `OBSERVATIONAL_MEMORY_OBSERVER_THRESHOLD` — Token threshold for observation (default: `30000`)
- `OBSERVATIONAL_MEMORY_REFLECTOR_THRESHOLD` — Token threshold for reflection (default: `40000`)
