# Deliberate - Project Documentation

## Current Work: Hybrid CLI + MCP Architecture

### Status: ✅ MCP Adapter Implemented

We now have a **hybrid architecture** supporting both CLI and MCP adapters:
- **CLI adapters** for fast, simple execution (planning, review)
- **MCP adapters** for rich tool access (execution, refinement)

**Key Files:**
- `src/deliberate/adapters/mcp_adapter.py` - MCP JSON-RPC adapter
- `src/deliberate/adapters/cli_adapter.py` - CLI subprocess adapter
- `tests/unit/test_mcp_adapter.py` - MCP adapter tests (6/6 passing)
- `.deliberate-mcp-hybrid.yaml` - Example hybrid configuration

---

## Previous Work: CLI Native Integration & Structured Output

### Problem Statement
The initial CLI adapter implementation had several issues:
1. Hardcoded cost estimates ($0.20/1K tokens) - highly inaccurate
2. No structured output support - agents returned free-form text
3. Manual CLI flag configuration in YAML - fragile and error-prone
4. Agent names in worktree paths biased reviews

### Solution Approach: Native CLI Integration

Instead of manually configuring CLI flags in YAML, we're integrating deeply with each CLI's native capabilities:

#### Native CLI Capabilities Discovered

**Claude CLI:**
- `--json-schema <schema>` - Enforces structured output via JSON Schema
- `--output-format json` - Returns structured JSON response
- `--dangerously-skip-permissions` - Headless operation
- Provides `total_cost_usd` in response - actual API cost!
- Provides detailed token usage including cache tokens

**Gemini CLI:**
- `--output-format json` - JSON output
- `-y` - YOLO mode (auto-approve tools)
- No JSON schema validation support

**Codex CLI:**
- `--json` - JSONL output
- `--output-schema <FILE>` - JSON Schema file support
- `-s danger-full-access -a never` - Headless operation

### Implementation: EnhancedCLIAdapter

Created `src/deliberate/adapters/cli_adapter_v2.py` with:

1. **Auto-detection**: Identifies CLI tool from command[0]
2. **Structured Output**: Automatically adds appropriate flags
3. **JSON Parsing**: Extracts structured responses
4. **Accurate Costs**: Uses CLI-provided costs when available
5. **Token Tracking**: Extracts real token counts from responses

### Experimental Results

#### Test 1: Claude with Execution Schema ✅

```bash
uv run python test_cli_structured_output.py
```

**Results:**
- Duration: 31.85s
- Tokens: 166,903 (includes 57K cache creation + 108K cache read!)
- Actual Cost: $0.2747 (from Claude API)
- Structured Output: Perfect JSON matching our schema

**Sample Output:**
```json
{
  "summary": "The CLI help text is functional but could be enhanced...",
  "changes": [
    {
      "file": "src/deliberate/cli.py",
      "description": "Enhance the task argument help text..."
    }
  ],
  "reasoning": "..."
}
```

**Key Insights:**
- Claude's `--json-schema` works perfectly
- Prompt caching significantly increases "token" count
- Actual cost from API is more accurate than estimates
- Structured output ensures consistent parsing

#### Test 2: Gemini with JSON Output ❌

**Results:**
- Duration: 31.43s
- Tokens: 51
- Content: Empty string
- Issue: Response not being parsed correctly

**Status:** Needs investigation - likely incorrect response field extraction

#### Test 3: Codex with JSON Output ❌

**Results:**
- Duration: 0.04s (suspiciously fast)
- Tokens: 15
- Content: Empty string
- Issue: Command likely failing silently or not configured correctly

**Status:** Needs investigation - may need different flags or correct model selection

### Fixes Applied

1. **Anonymous Worktree Paths** (src/deliberate/phases/execution.py:43)
   - Changed: `exec-{agent.name}-{uuid}` → `exec-{uuid}`
   - Prevents review bias from seeing agent names

2. **Structured Output Extraction** (cli_adapter_v2.py)
   - Extracts `structured_output` field from Claude responses
   - Proper JSON parsing for each CLI format
   - Token usage extraction including cache tokens

3. **Actual Cost Tracking**
   - Extracts `total_cost_usd` from CLI responses
   - Falls back to estimates when not available
   - Stores in `_actual_cost_usd` for budget tracker

### Integration with genai-prices

**API Discovered:**
```python
from genai_prices import calc_price, Usage

# Need to create Usage object with input/output tokens
# Then call calc_price(usage, model_ref, provider_id=...)
```

**Status:** Need to update adapter to use correct API

### Gemini CLI: Stdin Piping Required! ✅

**Discovery from Production CI Script:**
- Gemini CLI expects task via **stdin**, not as argument
- Command: `echo "task" | gemini --yolo` (NOT `gemini --yolo -p "task"`)
- Using `-p` flag or argument causes issues

**Implementation:**
- Added `_should_pipe_stdin()` method to detect Gemini
- Modified `run_agentic()` to pipe task via `stdin` for Gemini
- Command: `["gemini", "--yolo"]` with task sent via `proc.communicate(input=task.encode())`

**Results:**
- Duration: 20-23s
- Tokens: 51-79
- Content: Plain text output (works!)
- `--output-format json` causes timeouts (issue to investigate)

**Auth Notes:**
- Requires `GEMINI_API_KEY` environment variable
- May show "Token file corrupted" warnings but still works
- Uses cached credentials after first auth

### Next Steps

1. ✅ Fix genai-prices integration
2. ✅ Fix Gemini stdin piping
3. ⏳ Debug Codex empty response issue
4. ⏳ Create review phase schema & test
5. ⏳ Integrate EnhancedCLIAdapter into orchestrator
6. ⏳ Run full deliberate workflow with structured output

### Lessons Learned

1. **Use Native CLI Features**: Each CLI has built-in support for what we need
2. **Structured Output is Essential**: JSON schemas eliminate parsing ambiguity
3. **Cache Tokens Matter**: Claude's caching dramatically affects token counts
4. **Trust API Costs**: CLI-provided costs are more accurate than estimates
5. **Test Each CLI Independently**: Different CLIs have different behaviors
6. **Anonymous IDs Prevent Bias**: Reviewers shouldn't see agent names

### Configuration Updates

**test_cli_review_config.yaml:**
- Added Gemini env var support (removed hardcoded API key)
- Added Codex agent configuration
- Changed agents to `[gemini, codex]` for testing
- Tracing endpoint: `https://alloy.partly.pro` (still UNIMPLEMENTED error)

### Files Created/Modified

**New Files:**
- `src/deliberate/adapters/cli_adapter_v2.py` - Enhanced adapter
- `src/deliberate/schemas/execution.json` - Execution response schema
- `src/deliberate/schemas/review.json` - Review response schema
- `test_cli_structured_output.py` - CLI testing script
- `CLI_NATIVE_SUPPORT.md` - Documentation of CLI capabilities
- `PROJECT.md` - This file

**Modified Files:**
- `pyproject.toml` - Added genai-prices dependency
- `src/deliberate/phases/execution.py` - Anonymous worktree paths
- `test_cli_review_config.yaml` - Updated agent configs

### Tracing Status

**Endpoint:** `https://alloy.partly.pro`

**Error:** `StatusCode.UNIMPLEMENTED`

**Possible Issues:**
- Missing `/v1/traces` or `/otlp/v1/traces` path
- Requires authentication
- Endpoint not configured for OpenTelemetry

**Workaround:** Tracing is optional, workflow works without it

## Summary of CLI Integration Status

### ✅ Claude CLI - Fully Working
- **Structured Output**: Perfect with `--json-schema`
- **Cost Tracking**: Provides actual `total_cost_usd` in response!
- **Token Usage**: Detailed breakdown including cache tokens
- **Command**: `claude --dangerously-skip-permissions -p --output-format json --json-schema '...' "task"`
- **Performance**: 30s, 166K tokens, $0.27 actual cost
- **Status**: Production ready

### ✅ Gemini CLI - Working (Text Output)
- **Structured Output**: Plain text only (JSON format causes timeouts)
- **Command Pattern**: `echo "task" | gemini --yolo`
- **Key Discovery**: Requires stdin piping, not argument passing
- **Performance**: 20-23s, 51-79 tokens
- **Auth**: Uses `GEMINI_API_KEY` env var
- **Status**: Working for text tasks, JSON output needs investigation

### ⏳ Codex CLI - Needs Investigation
- **Issue**: Returns empty content
- **Command Tried**: `codex exec -s danger-full-access -a never -p "task"`
- **Duration**: 0.03s (suspiciously fast)
- **Next Steps**: Test with different flags, check stderr output
- **Status**: Not yet functional

## Architecture Decisions

### Why Enhanced Adapter?

The original `CLIAdapter` had hardcoded assumptions that didn't work:
```python
# OLD - Broken
def _build_agentic_command(self, task):
    if "claude" in self.command[0]:
        return ["claude", "--print", "-p", task]  # Ignores config!
```

The new `EnhancedCLIAdapter` respects each CLI's native capabilities:
```python
# NEW - Flexible
def run_agentic(self, task, ...):
    cmd = list(self.command)  # Start with config
    if schema_name:
        cmd = self._add_structured_output_flags(cmd, schema_name)  # Add native flags
    if self._should_pipe_stdin():
        proc.communicate(input=task.encode())  # Pipe for Gemini
    else:
        cmd.append(task)  # Append for Claude/Codex
```

### Cost Tracking Strategy

1. **Use actual costs when available** (Claude provides this!)
2. **Extract real token counts** from CLI responses
3. **Fall back to estimates** only when necessary
4. **Future**: Integrate genai-prices for accurate estimation

### Structured Output Approach

- **Claude**: Use `--json-schema` for enforced schemas
- **Gemini**: Use plain text, parse naturally (or investigate JSON mode)
- **Codex**: TBD - supports `--output-schema` but needs testing
