# IDEA-017: Provider-Agnostic Architecture — Universal Agent Orchestration

**Date:** 2026-03-04
**Source:** user
**Status:** researched

## Core Vision

Build iEvo as a universal agent orchestration layer that works with Claude Code, OpenAI Codex, Ollama, and llama.cpp natively — without replacing or limiting the user's access to any native CLI. The experience is "Apple-style": pick a provider, it just works. Pre-built pipelines and agent teams out of the box.

---

## Discussion Log

### 2026-03-04 — Initial Requirements

**User:** Want to use Claude Code and Codex (each is better at different tasks). Don't want to lose native CLI features. Want to pipe through natively, transparent passthrough. Pre-built pipelines + agents out of the box. Also Ollama for cheap tasks. Also llama.cpp.

**Key constraint articulated:** "preserve full native CLI access — don't limit the user."

**Also raised:** pluggable I/O (TUI, Tauri/Electron, Telegram, Discord, web), input/output provider concept. Plugin system to avoid Android vs iPhone problem.

---

### 2026-03-04 — SDK Selection Discussion

**Question:** Which agent SDK to use for multi-provider support?

**Research done:** Compared 11 SDKs by GitHub stars + activity:

| SDK | Stars | Decision |
|-----|-------|----------|
| LangChain | 128k | Too heavy, abstraction hell |
| AutoGen | 55k | Merging into MS Agent Framework — transition risk |
| LlamaIndex | 47k | Best for RAG, overkill for iEvo |
| CrewAI | 45k | Closest to iEvo model, uses uv, but LangChain under hood |
| Agno | 38k | Multi-provider, batteries included |
| OpenAI Agents SDK | 19k | Claude not natively supported, needs proxy |
| Pydantic AI | 15k | Type-safe, native Claude+OpenAI+Ollama, minimal |

**Decision: Pydantic AI**

**Rationale:**
- Native Claude (AnthropicModel) + OpenAI (OpenAIModel) + Ollama (OllamaModel) — no proxy needed
- llama.cpp via `llama-server` OpenAI-compatible endpoint → same OpenAIModel class
- Pydantic already in iEvo's dependency stack
- Type-safe structured outputs — critical for judge agent in benchmark pipeline (REQ-001 in Eva)
- Minimal abstractions — easy to debug, no magic
- From Pydantic creators — trusted, well-maintained

**Why not Agno:**
- Batteries included (memory, RAG) duplicate iEvo's own memory system
- More magic under the hood = harder to debug
- iEvo's memory system is the right place for memory, not the SDK layer

**Provider matrix for Pydantic AI:**

| Provider | Class | Notes |
|----------|-------|-------|
| Claude | `AnthropicModel("claude-sonnet-4-6")` | Native |
| OpenAI / Codex | `OpenAIModel("gpt-4o")` | Native |
| Ollama | `OllamaModel("llama3.2")` | Native |
| llama.cpp | `OpenAIModel("llama-3", base_url="http://localhost:8080/v1")` | Via llama-server |
| Groq, Mistral, any OpenAI-compat | `OpenAIModel(..., base_url=...)` | Via endpoint swap |

---

### 2026-03-04 — Architecture Research Findings

**Research done:** Deep research on native CLI piping, universal agent format, provider routing, pluggable I/O, plugin systems, Tauri desktop. 7 PROP files created.

**Key findings:**

#### PTY Passthrough
- `CLIProxyAPI` already wraps Claude Code + Codex + Gemini as OpenAI-compatible API
- `pexpect` for Python PTY management
- **To build:** thin PTY bridge (~200 LOC) with event interception. Transparent — user sees native CLI.

#### Universal Agent Format (UAF) — this is iEvo's core IP
- Claude Code format and Codex format are **semantically incompatible** — not just different serializations
  - Claude Code: `hooks`, `skills`, `memory`, `mcpServers` — rich orchestration model
  - Codex: `model_reasoning_effort`, `sandbox_mode`, `developer_instructions` — execution model
- **No existing transpiler exists** — greenfield
- **Decision:** UAF extending current `.md` frontmatter with abstract model tiers (`fast|standard|powerful|reasoning`) + provider-specific override blocks + transpiler → Claude `.md` / Codex `.toml` / Ollama config
- User writes agent once, iEvo deploys it to the right provider

#### Provider Routing
- LiteLLM (20k ⭐, MIT) handles API-level routing to 100+ providers — use it, don't build
- RouteLLM (Stanford, ICLR 2025) — ML-based routing by query complexity, 85% token cost reduction
- **Gap:** CLI-level routing (which CLI tool to spawn) doesn't exist yet — must build
- **Decision:** two-layer routing: (1) static per-agent from UAF config, (2) LiteLLM for API providers

#### Pluggable I/O
- **[OpenClaw](https://github.com/openclaw/openclaw)** (163k ⭐ as of Mar 2026) — self-hosted AI agent gateway. Exploded from a weekend hack to 100k stars in 2 months (Jan 2026). Formerly known as Clawdbot/Moltbot.
  - **Scale:** 163k stars, 5,700+ community skills, 50+ channels
  - **Channels:** WhatsApp, Telegram, Discord, Slack, Signal, iMessage, Teams, Matrix, IRC, LINE, Twitch, Google Chat, and more
  - **5-component architecture:**
    - **Gateway** — routes messages from channels
    - **Brain** — orchestrates LLM calls via ReAct loop
    - **Memory** — persistent context in Markdown files
    - **Skills** — plug-in capabilities (5,700+ community skills)
    - **Heartbeat** — scheduler and inbox monitor
  - **Plugin system:** extensions live in `extensions/*` (workspace packages), skills = rule definitions + scripts running alongside agents, 5,700+ community-built
  - **Agent routing:** `openclaw agents bind/unbind` for account-scoped route management, ACP thread-bound agents as first-class runtimes
  - **Key pattern:** `Platform Connector → Channel Adapter → Agent Core` — agent core never sees platform-specific code
  - **Very relevant:** community article ["How I Built a Deterministic Multi-Agent Dev Pipeline Inside OpenClaw"](https://dev.to/ggondim/how-i-built-a-deterministic-multi-agent-dev-pipeline-inside-openclaw-and-contributed-a-missing-4ool) — directly applicable to iEvo's SDD pipeline
  - Docs: https://docs.openclaw.ai

- **[IronClaw](https://github.com/nearai/ironclaw)** — Rust fork of OpenClaw by NEAR AI, focused on privacy and security
  - **Key differentiators vs OpenClaw:**
    - Per-tool **WASM sandboxes** (not shared Docker) — capability-based permissions, explicit HTTP allowlist
    - **Credentials injected at host boundary** — LLM never sees secrets (OpenClaw exposes them to LLM context)
    - **PostgreSQL** instead of SQLite
    - Compile-time memory safety (Rust)
    - Leak detection on credential injection
  - Works with any OpenAI-compatible backend — point at Ollama for fully local/offline setup
  - NEAR AI cloud is optional, not required
  - **Relevance for iEvo:** WASM sandbox model is a more principled approach to tool isolation than Docker for agent security — worth evaluating as an alternative to Docker sandbox

- **Langclaw** — Python, `app.add_channel()` pattern, built on LangChain/LangGraph. Simpler, more inspectable

- **Decision:** Python `IOAdapter` Protocol. TUI, CLI, Telegram, Discord = adapters. Each channel is a plugin implementing the protocol. OpenClaw is a reference implementation worth studying closely — same problem, solved at massive scale. IronClaw's WASM sandbox model is worth evaluating for iEvo's tool isolation layer.

#### Plugin System
- 5 contribution points: providers / adapters / agents / skills / hooks
- `plugin.yaml` manifest for discovery
- Current marketplace = official bundled plugin

#### Desktop (Phase 3)
- **Tauri v2** over Electron — 2-5 MB vs 150 MB
- `tauri-plugin-pty` + xterm.js for terminal embedding
- Python backend as `ievo --serve` (WebSocket) → Tauri shell on top
- Separate repo `ievo-ai/ievo-desktop`

---

### 2026-03-04 — Native SDK Features Question

**Question raised:** How to preserve native features — Claude extended thinking, Codex reasoning mode, memory compaction?

**Answer:** UAF transpiler must preserve provider-specific fields in override blocks. Example:

```yaml
# In UAF agent frontmatter
model: powerful  # abstract tier → maps to claude-opus-4-6 or gpt-4o depending on provider

providers:
  claude:
    extended_thinking: true
    memory_compaction: true
  codex:
    model_reasoning_effort: high
    sandbox_mode: read-only
```

When transpiling to Claude `.md` → extended_thinking fields pass through.
When transpiling to Codex `.toml` → reasoning_effort fields pass through.
Native features are **never emulated** — they're passed directly to the native CLI.

---

## What to Take from OpenClaw and IronClaw

### From OpenClaw

| What | How to apply in iEvo |
|------|---------------------|
| **5-component architecture** (Gateway, Brain, Memory, Skills, Heartbeat) | Direct mapping to iEvo: Gateway=PTY bridge, Brain=agent runner, Memory=.ievo/memory, Skills=marketplace skills, Heartbeat=startup sync. Study how they separated concerns. |
| **Skills as first-class plugins** | 5,700 community skills = proof that a simple plugin format (Markdown + script) scales. iEvo's skill format is already similar — validate against OpenClaw's spec. |
| **Channel adapter pattern** | Copy directly: `Platform Connector → Channel Adapter → Agent Core`. iEvo's IOAdapter Protocol should mirror this. Don't invent — adopt. |
| **ACP thread-bound agents** | Agent routing via `bind/unbind` per account/session — useful model for iEvo's multi-agent pipeline where different agents handle different REQ stages. |
| **Community skill marketplace** | 5,700 skills in months — the flywheel is real. iEvo marketplace should study how OpenClaw structured skill discovery, versioning, and trust. |
| **Deterministic multi-agent pipeline article** | Read carefully — community already solved "how to build a reliable SDD-style pipeline inside OpenClaw". May save months of trial-and-error. [Link](https://dev.to/ggondim/how-i-built-a-deterministic-multi-agent-dev-pipeline-inside-openclaw-and-contributed-a-missing-4ool) — full analysis below. |

#### Deep dive: "How I Built a Deterministic Multi-Agent Dev Pipeline Inside OpenClaw"

**Problem:** Build programmer → reviewer (up to 3 retries) → tester pipeline that is deterministic — a state machine, not LLMs making routing decisions.

**Why deterministic matters (key insight):**
> LLMs shouldn't orchestrate flows — they are unreliable routers. They misinterpret iteration counts, forget to signal transitions, and make non-deterministic routing decisions. YAML beats prompt engineering for coordination.

This is exactly the problem iEvo faces with its SDD pipeline (spec-writer → architect → coder → reviewer → QA → acceptance).

**Solution architecture:**
- 3 peer agents with isolated workspaces, distinct tool access, separate models
  - Claude Opus for programming (expensive = quality)
  - Claude Sonnet for reviewing (cheaper = optimize cost)
- **Lobster** — OpenClaw's YAML-based workflow engine for deterministic sequencing
- `agentToAgent` tool for peer-to-peer messaging between agents
- Session key addressing: `pipeline:<project>:<role>` — no database needed, string composition as data model
- Webhook triggers for external initiation

**How the retry loop works:**
```yaml
# Lobster sub-workflow (code-review.lobster)
programmer writes → reviewer evaluates → LLM-task parses approval as JSON
loop condition: if not approved AND iterations < 3 → restart cycle
on approval: return to parent workflow → tester → notification
```

**The missing piece contributed to Lobster (PR #20):**
Sub-workflow steps with loop support — 315 lines (129 impl + 186 tests):
- `lobster` field: embed `.lobster` files as workflow steps
- `loop.maxIterations`: ceiling
- `loop.condition`: shell command evaluated after each iteration, env vars: `LOBSTER_LOOP_JSON`, `LOBSTER_LOOP_STDOUT`, `LOBSTER_LOOP_ITERATION`

**What was rejected before finding the solution:**
- Ralph Orchestrator — elegant but no inter-agent coordination primitives
- OpenClaw sub-agents — non-deterministic (LLM decides routing) + concurrency limits
- Custom event bus — would have needed Redis Streams/NATS + external state machine

**Direct lessons for iEvo SDD pipeline:**

| Lesson | iEvo implication |
|--------|-----------------|
| State machine, not LLM routing | iEvo pipeline stages (spec→plan→code→review) must be driven by a deterministic engine, not by Claude deciding "what's next" |
| Session keys as data model | `pipeline:<req>:<stage>` addressing could replace complex state tracking in iEvo |
| YAML workflow > prompt engineering | iEvo's pipeline rules in IEVO.md are prompts — consider a Lobster-style YAML engine for enforcement |
| Contribute missing pieces early | If iEvo's pipeline engine needs a feature, contribute upstream rather than wrapping around |
| Cost routing per stage | Opus for complex tasks, Sonnet for review — iEvo should do the same per UAF tier mapping |
| 12 concurrent sessions (4 projects × 3 roles) | True parallelism via async messaging — iEvo's pipeline should be async-capable |

### From IronClaw

| What | How to apply in iEvo |
|------|---------------------|
| **Credentials injected at host boundary** | Critical security principle: LLM should never see secrets. iEvo's Docker sandbox currently passes `CLAUDE_CODE_OAUTH_TOKEN` as env var — visible to the model. IronClaw's approach: inject at host boundary *after* LLM decides what to do. Adopt this. |
| **WASM sandbox per tool** | More granular than Docker — each tool runs in isolated WASM container with explicit capability grants (HTTP allowlist, secrets access). Better for iEvo's tool-level isolation principle. Evaluate as Docker alternative for Phase 2. |
| **Capability-based permissions** | Explicit opt-in per tool: "this tool can make HTTP calls to these domains only". Stronger than `--allowedTools` flag. Design iEvo's tool permission model this way. |
| **Offline-first with OpenAI-compatible backend** | IronClaw works fully offline via Ollama + local PostgreSQL. iEvo should have the same guarantee — no cloud dependency required. |

### What NOT to take

| What | Why skip |
|------|---------|
| OpenClaw's memory (Markdown files) | iEvo already has a more structured memory system — don't regress |
| OpenClaw's Brain (ReAct loop) | iEvo uses Claude Code's native agent loop — more powerful, no need to reimplement |
| IronClaw's PostgreSQL | SQLite is fine for iEvo's scale, overkill to add PostgreSQL dependency |
| OpenClaw's full plugin format | iEvo's Claude Code plugin format is already defined — only take the channel adapter pattern |

---

## Key Architectural Decision: Deterministic Pipeline Engine is Core iEvo

**Date:** 2026-03-04
**Decision:** iEvo's SDD pipeline must be driven by a deterministic workflow engine — not by LLMs making routing decisions.

**Rationale:**
- LLMs as routers are unreliable: they misinterpret iteration counts, forget to signal transitions, make non-deterministic decisions
- iEvo's pipeline (spec → plan → code → review → QA → acceptance) has strict ordering, retry logic, and gate conditions — exactly what a state machine solves
- IDEA-016 (smart spec pipeline) describes a judge/gate agent that blocks progress — this only works reliably with a deterministic engine underneath
- OpenClaw community validated this pattern at scale — it works

**What this means for iEvo:**
- Pipelines become first-class citizens in iEvo, not conventions written in IEVO.md prompts
- Each pipeline stage has explicit: entry condition, agent to invoke, exit artifact, retry policy, failure handling
- Users can define custom pipelines in YAML — iEvo ships pre-built pipelines (SDD, review-only, spec-only, etc.)
- The pipeline engine is provider-agnostic — same YAML runs whether agents are on Claude, Codex, or Ollama
- Pre-built pipelines = the "Apple experience" — pick a provider, pipelines just work out of the box

**Reference implementation to study:** Lobster (OpenClaw's workflow engine) — evaluate for adoption or as inspiration for iEvo's own engine.

**Status:** CONFIRMED DIRECTION — not a raw idea, this is the architectural foundation.

---

## 2026-03-04 — Credential Security Layer

### LLM API Keys: LiteLLM Proxy

**Decision: use LiteLLM proxy on host for all provider API keys.**

```
Host process:
  LiteLLM proxy :8001  ← реальные ключи здесь (из credentials.enc)
    ├── Anthropic → api.anthropic.com  + auth header
    ├── OpenAI   → api.openai.com     + auth header
    └── Ollama   → localhost:11434

Docker container (agent):
  ANTHROPIC_BASE_URL=http://host.docker.internal:8001
  ANTHROPIC_API_KEY=sk-fake  ← бесполезен без proxy
```

LLM в контейнере никогда не видит реальный API ключ. Даже `echo $ANTHROPIC_API_KEY` вернёт фейк. LiteLLM добавляет auth на хосте.

Бонусы LiteLLM proxy: rate limiting, audit logging, spend tracking — из коробки.

---

### Tool Credentials: PreToolUse Hook + Vault

**Problem:** когда LLM вызывает `Bash("curl -H 'Authorization: token $GITHUB_PAT_TOKEN'")` — нужно подменить `$GITHUB_PAT_TOKEN` на реальное значение из vault, не раскрывая его LLM заранее.

**Solution:**

```
LLM видит только имена:
  available_credentials: ["GITHUB_PAT_TOKEN", "AWS_ACCESS_KEY"]

LLM вызывает:
  Bash("curl -H 'Authorization: token $GITHUB_PAT_TOKEN' https://api.github.com")

PreToolUse hook:
  1. Парсит аргументы tool call
  2. Находит паттерны: $VAR, ${VAR}, env var refs
  3. Резолвит из vault → реальное значение
  4. Подменяет in-place
  5. Передаёт модифицированный вызов дальше

PostToolUse hook (leak detector):
  1. Сканирует output regex-паттернами (ghp_*, sk-*, AKIA*, etc.)
  2. Маскирует или блокирует если нашёл утечку
```

**Vault options (pluggable):**

| Уровень | Инструмент | Когда |
|---------|-----------|-------|
| Personal | `~/.ievo/credentials.enc` (RSA+AES) — уже в архитектуре | по умолчанию |
| Team | HashiCorp Vault plugin | enterprise |
| Git-friendly | SOPS (Mozilla) | monorepo secrets |
| Simple | age encryption | dev/staging |

---

### Critical Gap: Hooks по провайдерам

**Исследование показало:**

| Provider | Hook система | Статус |
|----------|-------------|--------|
| Claude Code | PreToolUse / PostToolUse / Stop | ✅ Полная |
| Codex CLI | Hooks | ❌ Запрошено в [Discussion #2150](https://github.com/openai/codex/discussions/2150), не реализовано |
| Grok CLI | Hook система | ❌ Нет. Только tool calling через API |

**Решение — PTY bridge как универсальный hook point:**

```
Claude Code → нативные PreToolUse/PostToolUse hooks (чище, надёжнее)
Codex CLI   → PTY bridge перехватывает stdout/stdin → парсит tool calls → inject
Grok CLI    → PTY bridge то же самое

iEvo PTY bridge = единственное место где credential injection
                  работает для ВСЕХ провайдеров одинаково
```

Credential injection логика одна, адаптеры разные:
- `ClaudeCredentialAdapter` — через нативные hooks
- `PTYCredentialAdapter` — через PTY stream parsing (Codex, Grok, будущие провайдеры)

**Связь с существующим:**
- PreToolUse/PostToolUse hooks → уже в REQ-012 инфраструктура (debug capture)
- `credentials.enc` → уже описан в CLAUDE.md
- LiteLLM → уже в стеке обсуждения (IDEA-014, provider routing)

---

## Open Questions (to resolve in next discussion)

- [ ] How does extended thinking interact with PTY passthrough? Does thinking output stream through transparently or need special handling?
- [ ] UAF model tiers: who decides the mapping `powerful → claude-opus-4-6`? User config? iEvo defaults? Per-project override?
- [ ] When an agent is deployed to both Claude and Codex, which runs when? Static config or dynamic routing?
- [ ] Plugin isolation: do plugins run in Docker or in the same process?
- [ ] Telegram/Discord adapter: structured interview mode — how does multi-turn conversation state persist across messages?
- [ ] PTY credential adapter: how to reliably parse tool call JSON from raw PTY stream? What if stream is chunked mid-JSON?
- [ ] Vault plugin API: what's the minimal interface a vault plugin must implement?

---

## Related

- PROP-2026-03-04-universal-orchestration-architecture.md — master architecture proposal
- PROP-2026-03-04-universal-agent-format.md — UAF transpiler proposal
- PROP-2026-03-04-pty-passthrough-layer.md — PTY bridge proposal
- PROP-2026-03-04-provider-routing.md — routing proposal
- PROP-2026-03-04-pluggable-io-adapters.md — I/O adapters proposal
- PROP-2026-03-04-plugin-system.md — plugin system proposal
- PROP-2026-03-04-tauri-desktop-wrapper.md — Tauri desktop proposal
- IDEA-012 — OpenAI Codex multi-agent config format
- IDEA-013 — iEvo MCP server
- IDEA-014 — Docs knowledge base tooling
- IDEA-015 — Package screening agent
