# Implementation Plan: JSON EMU Support + Batch Propose Endpoint

**Date:** 2025-12-19
**Status:** Planning
**Author:** Claude Code Assistant

---

## Overview

Add support for JSON-based EMU definition files and implement batch registration via the `POST /v1/emus/propose` endpoint.

### Goals

1. Support JSON EMU files alongside existing Python files
2. Add `propose_emus()` client method for batch registration
3. Update CLI to use batch propose instead of individual registration calls

---

## Design Decisions

### JSON File Format

**Decision:** Strict `{"emus": [...]}` wrapper format (no flexible single-object support)

**Rationale:**
- **Consistency** — One format, no guessing, predictable parsing
- **Extensibility** — Can add file-level metadata later (version, project, etc.)
- **Explicit over implicit** — Follows Python/industry philosophy
- **Tooling friendly** — JSON schema validation is cleaner with one shape
- **Industry standard** — How Terraform, K8s, and other config systems work

**Example:**
```json
{
  "emus": [
    {
      "emu_key": "support.vip_escalation",
      "trigger": "state.customer.tier == \"vip\" AND state.ticket.priority >= 3",
      "action": {
        "type": "route",
        "route": "vip-support-queue"
      },
      "policy": {
        "mode": "auto",
        "priority": 9,
        "cooldown": {"seconds": 3600, "gate": "activation"}
      },
      "expected_utility": 0.95,
      "confidence": 0.88,
      "intent": "Route VIP customer tickets to specialized support queue",
      "state": "draft"
    }
  ]
}
```

### API Endpoint

**Endpoint:** `POST /v1/emus/propose`
**Payload:** Flat array of EMU objects `[{emu1}, {emu2}, ...]`

---

## Files to Modify

### 1. `cli/discovery.py`

**Purpose:** Add JSON file discovery alongside Python

**Changes:**

| Change | Description |
|--------|-------------|
| Add `import json` | Required for JSON parsing |
| Add `load_emus_from_json_file()` | New function to parse JSON EMU files |
| Modify `discover_project_emus()` | Glob `*.json` files in addition to `*.py` |

**New Function:**
```python
def load_emus_from_json_file(file_path: Path, project_name: str) -> tuple[list[DiscoveredEMU], list[str]]:
    """
    Load EMU definitions from a JSON file.

    JSON files must have the structure: {"emus": [{...}, {...}]}

    Args:
        file_path: Path to the JSON file
        project_name: Name of the project

    Returns:
        Tuple of (list of discovered EMUs, list of errors)
    """
    emus: list[DiscoveredEMU] = []
    errors: list[str] = []

    try:
        with open(file_path, "r", encoding="utf-8") as f:
            data = json.load(f)

        # Validate structure
        if not isinstance(data, dict):
            errors.append(f"JSON file must be an object with 'emus' array: {file_path}")
            return emus, errors

        if "emus" not in data:
            errors.append(f"JSON file must have 'emus' key: {file_path}")
            return emus, errors

        if not isinstance(data["emus"], list):
            errors.append(f"'emus' must be an array: {file_path}")
            return emus, errors

        # Process each EMU
        for i, emu_def in enumerate(data["emus"]):
            if not isinstance(emu_def, dict):
                errors.append(f"EMU at index {i} must be an object: {file_path}")
                continue

            if not is_emu_definition(emu_def):
                errors.append(f"EMU at index {i} missing required fields (emu_key, trigger): {file_path}")
                continue

            name = emu_def.get("emu_key", f"emu_{i}")
            emus.append(DiscoveredEMU(
                name=name,
                definition=emu_def,
                source_file=file_path,
                project=project_name,
            ))

    except json.JSONDecodeError as e:
        errors.append(f"Invalid JSON in {file_path}: {e}")
    except Exception as e:
        errors.append(f"Error reading {file_path}: {e}")

    return emus, errors
```

**Modified `discover_project_emus()`:**
```python
def discover_project_emus(project_path: Path) -> DiscoveryResult:
    result = DiscoveryResult()
    project_name = project_path.name

    emus_dir = project_path / "emus"
    if not emus_dir.is_dir():
        result.warnings.append(f"No 'emus' directory found in {project_path}")
        return result

    # Find all Python files (existing)
    py_files = [
        f for f in emus_dir.glob("*.py")
        if f.name != "__init__.py" and not f.name.startswith("_")
    ]

    # Find all JSON files (new)
    json_files = [
        f for f in emus_dir.glob("*.json")
        if not f.name.startswith("_")
    ]

    if not py_files and not json_files:
        result.warnings.append(f"No EMU files found in {emus_dir}")
        return result

    # Load Python files
    for py_file in py_files:
        file_emus, file_errors = load_emus_from_file(py_file, project_name)
        result.emus.extend(file_emus)
        result.errors.extend(file_errors)

    # Load JSON files
    for json_file in json_files:
        file_emus, file_errors = load_emus_from_json_file(json_file, project_name)
        result.emus.extend(file_emus)
        result.errors.extend(file_errors)

    return result
```

---

### 2. `ami/client.py`

**Purpose:** Add batch propose method

**Changes:**

| Change | Description |
|--------|-------------|
| Add `propose_emus()` method | New method for batch EMU registration |

**New Method:**
```python
def propose_emus(
    self,
    emus: list[dict],
    *,
    workspace: Optional[str] = None,
    project: Optional[str] = None,
) -> dict:
    """
    Propose multiple EMUs for registration in a single batch.

    Args:
        emus: List of EMU definitions (dicts with emu_key, trigger, action, etc.)
        workspace: Workspace slug/ID (overrides config)
        project: Project slug/ID (overrides config)

    Returns:
        Dict with batch registration results

    Raises:
        AMIBadRequest: If emus list is empty or contains invalid EMUs
        AMINotFound: If workspace or project doesn't exist

    Example:
        ```python
        emus = [
            {"emu_key": "welcome", "trigger": "...", "action": {...}},
            {"emu_key": "alert", "trigger": "...", "action": {...}},
        ]
        response = client.propose_emus(emus, workspace="prod", project="api")
        ```
    """
    # Validate input
    if not emus:
        raise AMIBadRequest("emus list cannot be empty")

    # Resolve workspace and project
    workspace_resolved = workspace or self.config.workspace or self.config.workspace_id
    project_resolved = project or self.config.project or self.config.project_id

    if not workspace_resolved:
        raise AMIBadRequest("workspace slug or ID is required (config or method parameter)")
    if not project_resolved:
        raise AMIBadRequest("project slug or ID is required (config or method parameter)")

    # Build request path
    path = "/v1/emus/propose"

    # Payload is flat array of EMUs
    payload = emus

    # Make request
    response_data = self.transport.post(
        path,
        json=payload,
        retry_policy=RetryPolicy.IDEMPOTENT,
    )

    return response_data
```

---

### 3. `cli/main.py`

**Purpose:** Update register command to use batch propose

**Changes:**

| Change | Description |
|--------|-------------|
| Update `register()` command | Collect all EMUs and call `propose_emus()` once |
| Update imports | Add `load_emus_from_json_file` if needed for `--file` option |

**Modified Registration Logic:**
```python
@app.command()
def register(
    # ... existing parameters ...
) -> None:
    # ... existing discovery logic ...

    # Validate locally first (existing)
    validation_errors = []
    for emu in discovered:
        result = validate_emu(emu.definition)
        if not result.valid:
            validation_errors.append((emu, result.errors))

    if validation_errors:
        # ... existing error handling ...
        raise typer.Exit(1)

    console.print("[green]Local validation passed[/green]\n")

    if dry_run:
        console.print("[yellow]Dry run mode - skipping registration[/yellow]")
        _display_emus_table(discovered)
        return

    # Register with API using batch propose
    try:
        client = get_ami_client()
    except typer.Exit:
        raise

    # Prepare all EMUs for registration
    prepared_emus = []
    for emu in discovered:
        prepared = prepare_emu_for_registration(
            emu.definition,
            workspace=workspace,
            project=emu.project,
        )
        prepared_emus.append(prepared)

    try:
        response = client.propose_emus(
            prepared_emus,
            workspace=workspace or os.environ.get("AMI_WORKSPACE"),
            project=os.environ.get("AMI_PROJECT"),  # or use first EMU's project
        )

        console.print(f"[green]Successfully proposed {len(prepared_emus)} EMU(s)[/green]")
        # Handle response format (may include per-EMU status)
        if isinstance(response, dict):
            if "results" in response:
                for result in response["results"]:
                    status = result.get("status", "unknown")
                    emu_key = result.get("emu_key", "?")
                    if status == "success":
                        console.print(f"  [green]OK[/green] {emu_key}")
                    else:
                        console.print(f"  [red]FAIL[/red] {emu_key}: {result.get('error', 'unknown')}")

    except Exception as e:
        console.print(f"[red]Failed to propose EMUs:[/red] {e}")
        raise typer.Exit(1)
```

---

### 4. Example JSON File (New)

**Path:** `projects/example_project/emus/alerts.json`

**Purpose:** Demonstrate JSON EMU format

```json
{
  "emus": [
    {
      "emu_key": "alerts.critical_system_down",
      "trigger": "state.system.status == \"down\" AND state.system.severity == \"critical\"",
      "action": {
        "type": "tool_call",
        "intent": "Page on-call engineer for critical system outage",
        "tool": {
          "tool_id": "pagerduty",
          "version": "2.0.0",
          "args": {
            "urgency": "high",
            "escalation_policy": "critical-systems"
          }
        }
      },
      "policy": {
        "mode": "auto",
        "priority": 10,
        "cooldown": {
          "seconds": 300,
          "gate": "activation"
        },
        "exclusion_groups": ["system_alerts"]
      },
      "expected_utility": 0.99,
      "confidence": 0.95,
      "intent": "Immediately alert on-call for critical system failures",
      "state": "draft"
    },
    {
      "emu_key": "alerts.degraded_performance",
      "trigger": "state.system.latency_p99 > 500 AND state.system.error_rate > 0.01",
      "action": {
        "type": "route",
        "route": "performance-triage-queue"
      },
      "policy": {
        "mode": "auto",
        "priority": 7,
        "cooldown": {
          "seconds": 900,
          "gate": "ack"
        }
      },
      "expected_utility": 0.85,
      "confidence": 0.80,
      "intent": "Route degraded performance incidents for triage",
      "state": "draft"
    }
  ]
}
```

---

## Implementation Order

| Step | File | Task |
|------|------|------|
| 1 | `cli/discovery.py` | Add `load_emus_from_json_file()` function |
| 2 | `cli/discovery.py` | Update `discover_project_emus()` to glob `*.json` |
| 3 | `ami/client.py` | Add `propose_emus()` method |
| 4 | `cli/main.py` | Update `register` command to use batch propose |
| 5 | `projects/example_project/emus/alerts.json` | Create example JSON file |
| 6 | Tests | Add unit tests for new functionality |

---

## Testing Plan

### Unit Tests

1. **Discovery Tests** (`tests/cli/test_discovery.py`)
   - `test_load_emus_from_json_file_valid` — Valid JSON with multiple EMUs
   - `test_load_emus_from_json_file_missing_emus_key` — Error on missing `emus` key
   - `test_load_emus_from_json_file_invalid_json` — Error on malformed JSON
   - `test_load_emus_from_json_file_missing_required_fields` — Error on EMU without emu_key/trigger
   - `test_discover_project_emus_json_and_py` — Mixed discovery

2. **Client Tests** (`tests/ami/test_client.py`)
   - `test_propose_emus_success` — Successful batch propose
   - `test_propose_emus_empty_list` — Error on empty list
   - `test_propose_emus_missing_workspace` — Error without workspace

3. **Integration Tests**
   - End-to-end: JSON file discovery → validation → batch propose

---

## Rollback Plan

If issues arise:
1. JSON discovery is additive — Python files still work
2. `propose_emus()` is a new method — `register_emu()` still available
3. CLI changes can be reverted to use individual registration

---

## Future Considerations

1. **JSON Schema Validation** — Add `jsonschema` for strict validation
2. **File-level Metadata** — Support `version`, `project`, `description` at file level
3. **YAML Support** — Add `.yaml` file support using same pattern
4. **Deprecate Python Files** — If JSON adoption is high, consider deprecating `.py` EMU files

---

## Appendix: EMU Schema Reference

```typescript
interface EMU {
  emu_key: string;        // Required: 3-128 chars, alphanumeric + :_.-
  trigger: string;        // Required: DSL expression
  action: Action;         // Required: tool_call | decision_prompt | route | context_directive
  policy?: Policy;        // Optional: mode, priority, cooldown, exclusion_groups
  expected_utility?: number; // Optional: 0.0-1.0 (default 0.8)
  confidence?: number;    // Optional: 0.0-1.0 (default 0.9)
  intent?: string;        // Optional: human-readable description
  state?: string;         // Optional: draft | active | canary | shadow | archived
  metadata?: object;      // Optional: custom metadata
}

interface Action {
  type: "tool_call" | "decision_prompt" | "route" | "context_directive";
  // ... type-specific fields
}

interface Policy {
  mode?: "auto" | "require_human" | "advisory";
  priority?: number;      // 0-9
  cooldown?: {
    seconds: number;
    gate: "activation" | "ack";
  };
  exclusion_groups?: string[];
}
```
