# CLAUDE.md — Synix

## What Is Synix

Synix is **a build system for agent memory**. Declarative pipelines define how raw conversations become searchable, hierarchical memory with full provenance tracking. Change a config, only affected layers rebuild. Think `make` or `dbt`, but for AI agent memory.

The fundamental output: **system prompt + RAG**, built from raw conversations with full lineage tracking.

## Core Concepts

- **Artifact** — immutable, versioned build output (transcript, episode, rollup, core memory). Content-addressed via SHA256.
- **Layer** — named level in the memory hierarchy. Layers form a DAG (transcripts → episodes → rollups → core).
- **Pipeline** — declared in Python. Defines layers, transforms, grouping strategies, and projections.
- **Projection** — materializes artifacts into usable outputs (search index via SQLite FTS5, context doc as markdown).
- **Provenance** — every artifact traces back to its inputs. Always included in search results.
- **Cache/Rebuild** — hash comparison: if inputs or prompt changed, rebuild. Otherwise skip.

Full entity model, storage format, and dataclass definitions: [docs/entity-model.md](docs/entity-model.md)
Pipeline Python API and examples: [docs/pipeline-api.md](docs/pipeline-api.md)

## Module Structure

```
src/synix/
├── __init__.py
├── cli.py              # Click CLI — synix run, synix search, synix lineage
├── pipeline/
│   ├── config.py       # Parse pipeline Python module into Pipeline/Layer objects
│   ├── dag.py          # DAG resolution — build order, rebuild detection
│   └── runner.py       # Execute pipeline — walk DAG, run transforms, cache artifacts
├── artifacts/
│   ├── store.py        # Artifact storage — save/load/query (filesystem-backed)
│   └── provenance.py   # Provenance tracking — record and query lineage chains
├── transforms/
│   ├── base.py         # Base transform interface
│   ├── parse.py        # Source parsers — ChatGPT/Claude JSON → transcript artifacts
│   ├── summarize.py    # LLM transforms — episode, rollup, core synthesis
│   └── prompts/        # Prompt templates as text files
├── projections/
│   ├── base.py         # Base projection interface
│   ├── search_index.py # SQLite FTS5 — materialize and query
│   └── flat_file.py    # Render core memory as context document
└── sources/
    ├── chatgpt.py      # ChatGPT export parser
    └── claude.py       # Claude export parser
```

## Key Module Interfaces

**pipeline/runner.py** calls:
- `artifacts.store.{save,load}_artifact()`, `get_artifact_id()` — cache checking
- `transforms.*.execute(inputs, config) -> Artifact`
- `projections.*.materialize(artifacts, config)` — after build
- `artifacts.provenance.record(label, parent_ids, prompt_id, model_config)`

**cli.py** calls:
- `pipeline.config.load(path) -> Pipeline`
- `pipeline.runner.run(pipeline, source_dir) -> RunResult`
- `projections.search_index.query(query, layers) -> list[SearchResult]`
- `artifacts.provenance.get_chain(label) -> list[ProvenanceRecord]`

## CLI Commands

```bash
synix run pipeline.py [--source-dir ./exports]   # Build pipeline + materialize projections
synix search "query" [--layers episodes,core]     # Search with provenance chains
synix lineage <artifact-id>                       # Provenance tree view
synix status                                      # Build summary table
```

CLI UX requirements (Rich formatting, colors, progress): [docs/cli-ux.md](docs/cli-ux.md)

## Environment & Build

- Python 3.11+, SQLite (stdlib), `ANTHROPIC_API_KEY` in env
- No external databases, no Docker, no web server
- UV-native: `uv sync`, `uv run synix`, `uv run pytest`

## Contributing

**Before pushing any changes**, run the full release check:

```bash
uv run release
```

This runs: sync templates → ruff fix → ruff check → pytest → verify all demos. All must pass before pushing. CI runs the same checks — if release passes locally, CI will pass.

To run just the demo verifications standalone (faster feedback during UX work):

```bash
uv run verify-demos
```

**Workflow changes** (`.github/workflows/`): test locally with [`act`](https://github.com/nektos/act) before pushing:

```bash
act push -W .github/workflows/<workflow>.yml --secret-file .secrets
```

Store local secrets in `.secrets` (gitignored):
```
ANTHROPIC_API_KEY=sk-ant-...
```

## PR & Issue Workflow

Every PR must link to the GitHub issues it addresses:

1. **Before creating a PR**, check open issues: `gh issue list --state open --json number,title`
2. **Review each issue** against the PR's changes — map features to issues
3. **Include `Closes #N`** in the PR body for each issue fully resolved by the PR
4. **Reference without closing** (`#N`) for issues only partially addressed
5. **PR body format**: `Closes #N` directives at the top, then `## Summary` with bullet points linking each feature back to its issue number

## Critical Rules

- **Customer-facing docs** (READMEs, templates, `synix init` output) must use `uvx synix` for all CLI commands. Internal dev docs (CLAUDE.md, test files) use `uv run synix`.
- **DO NOT** refactor core engine or abstract prematurely
- **DO NOT** implement StatefulArtifact, branching, eval harness, or any v0.2 feature
- **DO NOT** add Postgres, Neo4j, or any external database — SQLite + filesystem only
- **DO NOT** build a web UI
- **Every module must have at least basic tests**
- Write tests BEFORE or ALONGSIDE the module, never after
- Mock the LLM for unit and integration tests — only E2E hits real API
- Use `tmp_path` for all filesystem tests — no shared state
- **Every functional behavior change must have e2e test coverage** — unit tests alone are insufficient. If a change affects CLI output, plan display, cache detection, or artifact metadata, write an e2e test that exercises the full path.
- **Always consider template/demo impact** — changes to transforms, plan output, artifact metadata, or CLI formatting will affect golden files in `templates/*/golden/`. Regenerate goldens (`uv run synix demo run <template> --update-goldens`) and verify normalization rules in `demo_commands.py._normalize_output()` still cover new output patterns.

## Reference Docs

| Doc | Contents |
|-----|----------|
| [docs/entity-model.md](docs/entity-model.md) | Dataclass definitions, storage format, FTS5 schema, cache logic, architecture north star |
| [docs/pipeline-api.md](docs/pipeline-api.md) | Python pipeline definition, examples, config change demo |
| [docs/cli-ux.md](docs/cli-ux.md) | CLI UX requirements, color scheme, per-command formatting |
| [docs/prompt-templates.md](docs/prompt-templates.md) | All prompt template contents (episode, rollup, core) |
| [docs/test-plan.md](docs/test-plan.md) | Test structure, fixtures, all unit/integration/E2E test specs |
| [docs/build-phases.md](docs/build-phases.md) | Phase 1-5 implementation breakdown |
| [docs/DESIGN.md](docs/DESIGN.md) | Vision, origin story, full design narrative |
| [docs/BACKLOG.md](docs/BACKLOG.md) | Deferred items from v0.9 development |
