# astra-insight-mcp

A standalone MCP server that provides a **self-contained agent optimization loop** — read data, update instructions, run tests, evaluate, and iterate — all through the Astra Gateway public API.

## CLI (`astra`)

Install the unified command-line tool (interactive REPL, MCP server, OAuth):

```bash
go install github.com/ClareAI/astra-insight-mcp/cmd/astra@latest
```

### pip (Python wrapper)

The PyPI distribution name is **`wati-astra-cli`**. Installing runs **`go build`** on the machine and ships the `astra` binary inside the package; the console script is still **`astra`** (same as `go install`).

```bash
# From repo root (development)
pip install .

# After install: set up LLM key once, then use the REPL
astra configure
astra
```

#### Using `pip install wati-astra-cli`

That only works after the project is **published to PyPI** (or to a private index that your `pip` is configured to use).

**Option A — Publish to [PyPI](https://pypi.org)** (public `pip install`):

1. Create a [pypi.org](https://pypi.org) account and an **API token** under account settings.
2. From the repository root:

   ```bash
   pip install build twine
   python -m build    # e.g. dist/wati_astra_cli-0.1.0.tar.gz and .whl
   twine upload dist/*
   ```

   First upload: use username `__token__` and the token as the password.

3. On any machine (with **Go** installed for sdist builds; see below):

   ```bash
   pip install wati-astra-cli
   astra version
   ```

   Dry-run on [TestPyPI](https://test.pypi.org): `twine upload --repository testpypi dist/*`, then  
   `pip install -i https://test.pypi.org/simple/ wati-astra-cli`.

**Option B — Install from Git** (no PyPI):

```bash
pip install "git+https://github.com/ClareAI/astra-insight-mcp.git@main"
```

Adjust URL/branch for your fork; **Go** is still required because the package builds from source.

**Option C — Private index (DevPI, Artifactory, etc.)**: Set `extra-index-url` or `index-url` in `pip.conf` and point `twine` at your registry; flow is otherwise like Option A.

Optional local builds:

```bash
pip install build
python -m build
# dist/*.tar.gz (sdist) and *.whl; the wheel embeds a binary built for the current platform
```

**Note:** Installing from **sdist** runs `go build` on the user machine, so **Go** must be installed. The default wheel tag is `py3-none-any` but the embedded binary is **platform-specific**. To offer “no Go required” installs on PyPI, build and upload **platform-specific wheels** from CI for linux / macOS / Windows (or publish sdist only).

Then use `astra` (similar layout to other coding CLIs: default action is the REPL; server is a subcommand).

| Command | Description |
|--------|-------------|
| `astra` | Interactive agent optimizer (REPL) |
| `astra chat` | Same as `astra` |
| `astra server` | Streamable HTTP MCP server (`/mcp`, `/healthz`) |
| `astra mcp` | Alias for `astra server` |
| `astra auth …` | Browser OAuth or `--token` (see below) |
| `astra configure` | Save LLM API key to `~/.astra/llm.json` (like Claude-style setup) |
| `astra version` | Print version |
| `astra help` | Usage |

Global flag before subcommands: `-debug` / `--debug` (gateway curl traces). Legacy entrypoints still work: `go run ./cmd/main.go` (server only) and `go run ./cmd/cli` (REPL / `auth`).

## Architecture

```
┌──────────────────────────────────────────────────────────────────────┐
│                    astra-insight-mcp (this repo)                     │
│                                                                      │
│  Streamable HTTP MCP Server (spec 2025-03-26)                        │
│  POST /mcp  — JSON-RPC 2.0 (JSON or SSE response)                   │
│  GET  /mcp  — SSE notification stream                                │
│  DELETE /mcp — Terminate session                                     │
│  GET /healthz — Health check                                         │
│                                                                      │
│  Session management (Mcp-Session-Id header)                          │
│  Per-IP rate limiting (configurable req/min)                         │
│                                                                      │
│  64+ tools across skills:                                            │
│  Read / Write / Test / Integrations / Webhooks / Analytics           │
└───────────────────────────┬──────────────────────────────────────────┘
                            │  HTTP (Bearer token)
                            ▼
                 Astra Gateway API
              astra-gateway.wati.io/v1
```

**Zero changes** to any existing service. All data flows through the public Gateway API.

## Optimization Loop

```
 Understand
 ─────────
  1. list_agents                         → discover available agents
  2. get_agent                           → current instruction & rules
  3. get_agent_config                    → goals, style, lead fields
  4. get_runtime_config                  → live vs draft config comparison
  5. get_brandkit                        → brand colors, logos, company info
  6. list_actions / get_action           → what tools the agent can use
  7. list_connections                    → platform integrations
  8. list_knowledge_documents            → what knowledge the agent has
  9. list_annotations / get_annotation   → existing FAQ overrides
 10. retrieve_knowledge                  → test knowledge base coverage
 11. list_templates / get_template       → reference templates
 12. get_voice_config                    → voice agent settings

 Analyze
 ───────
 13. get_conversation_analytics          → conversation volume & trends
 14. get_lead_analytics                  → lead funnel & daily breakdown
 15. get_performance_analytics           → visitor → lead → qualified funnel
 16. get_lead_distribution               → hot/warm/cold/unqualified breakdown
 17. list_conversations / get_conversation → browse real conversations
 18. get_conversation_messages           → deep-dive into specific chats
 19. get_conversation_lead_qualification → lead qualification per conversation
 20. get_contacts_and_leads / get_contact → customer profiles & lead scores
 21. get_contact_conversations           → all conversations for a contact
 22. list_conversation_feedback          → user feedback per conversation
 23. get_lead_qualifications             → lead quality patterns
 24. get_voice_conversation / messages   → voice call transcripts

 Improve
 ───────
 25. update_agent_instruction            → improve instruction
 26. update_agent_config                 → update style, goals, lead fields
 27. create_annotation / update / delete → manage FAQ overrides
 28. create_knowledge_document           → add text to knowledge base
 29. delete_knowledge_document           → remove outdated knowledge
 30. import_knowledge_from_url           → add a web page
 31. import_knowledge_from_website       → crawl & import entire website
 32. clone_agent                         → create safe copy for experiments
 33. publish_agent                       → activate changes

 Verify — Manual
 ───────────────
 34. send_test_message                   → test with sample queries

 Verify — Systematic Testing Pipeline
 ─────────────────────────────────────
 35. start_test_pipeline                 → one-click: generate → execute → evaluate → analyze
 36. create_test_suite                   → auto-generate test cases (6 categories)
 37. add_test_case                       → manually add test cases
 38. create_test_run                     → execute test suite against agent
 39. get_test_run                        → inspect individual results
 40. create_run_analysis                 → AI analyzes failures
 41. analyze_failures                    → Phase 1: diagnose issues
 42. generate_instruction                → Phase 2: generate improved instruction
     → apply instruction and re-run tests
```

## Tools

### Read — Agent Config & Setup

| Tool | Description |
|------|-------------|
| `list_agents` | List all agents — names, IDs, types, statuses, channels |
| `get_agent` | Agent name, instructions, system rules, status |
| `get_agent_config` | Communication style, goals, lead fields, qualification criteria |
| `get_runtime_config` | Live/draft runtime config — deployed instructions, rules, actions |
| `get_brandkit` | Brand kit — colors, logos, company info, appearance settings |
| `list_actions` | Configured tools/actions (HubSpot, Calendly, web search, etc.) |
| `get_action` | Single action details — config, platform, connection status |
| `list_connections` | Platform connections and their status |

### Read — Knowledge & Annotations

| Tool | Description |
|------|-------------|
| `list_knowledge_documents` | Documents in the knowledge base (names, word counts, hit counts) |
| `list_annotations` | FAQ annotations (Q&A pairs, hit counts) |
| `get_annotation` | Single annotation details |
| `retrieve_knowledge` | Search knowledge base with a query |

### Read — Templates

| Tool | Description |
|------|-------------|
| `list_templates` | Pre-built agent templates, filter by category/type |
| `list_template_categories` | Available template categories |
| `get_template` | Template details — instructions, config, goals |

### Read — Performance Data

| Tool | Description |
|------|-------------|
| `get_conversation_analytics` | Conversation count analytics, daily breakdown |
| `get_lead_analytics` | Lead funnel — visitors, leads, hot/warm/cold, daily trends |
| `get_performance_analytics` | High-level funnel — visitor → lead → qualified count |
| `get_lead_distribution` | Lead score distribution (hot/warm/cold/unqualified) |
| `list_conversations` | Browse real user conversations with filters |
| `get_conversation` | Single conversation — status, channel, contact, metadata |
| `get_conversation_messages` | Full message history for a conversation |
| `get_conversation_lead_qualification` | Lead qualification for a specific conversation |
| `get_contacts_and_leads` | Contact profiles, lead scores, AI insights |
| `get_contact` | Single contact — email, phone, company, lead score, AI summary |
| `get_contact_conversations` | All conversations for a specific contact |
| `list_conversation_feedback` | User feedback (thumbs up/down, tags, comments) |
| `get_lead_qualifications` | Lead level, score, AI summary per lead |

### Read — Voice

| Tool | Description |
|------|-------------|
| `get_voice_config` | Voice agent configuration — model, language, style |
| `get_voice_conversation` | Voice conversation details — duration, status, phone |
| `get_voice_messages` | Voice conversation transcript |

### Write — Modify Agent

| Tool | Description |
|------|-------------|
| `update_agent_instruction` | Update draft instructions |
| `update_agent_config` | Update config — communication style, goals, lead fields |
| `publish_agent` | Publish draft to make it live |
| `clone_agent` | Clone an agent for safe experimentation |

### Write — Knowledge & Annotations

| Tool | Description |
|------|-------------|
| `create_annotation` | Add FAQ annotation (guaranteed-correct Q&A) |
| `update_annotation` | Update an annotation's title, questions, or answer |
| `delete_annotation` | Delete an annotation |
| `create_knowledge_document` | Add a text document to the knowledge base |
| `delete_knowledge_document` | Delete a document from the knowledge base |
| `import_knowledge_from_url` | Import a web page into the knowledge base |
| `import_knowledge_from_website` | Crawl and import an entire website |

### Webhooks

| Tool | Description |
|------|-------------|
| `list_webhooks` | List registered webhooks and subscribed events |
| `create_webhook` | Register a webhook for real-time event notifications |
| `update_webhook` | Update webhook active status or events |
| `delete_webhook` | Delete a webhook |

### Utility

| Tool | Description |
|------|-------------|
| `get_import_status` | Check knowledge base import job progress |

### Test — Manual Verification

| Tool | Description |
|------|-------------|
| `send_test_message` | Send a message and get agent response (supports multi-turn) |

### Test — Systematic Pipeline

The testing pipeline provides automated test generation, execution, evaluation, and analysis.

**Quick start:** Use `start_test_pipeline` for one-click end-to-end testing.
**Advanced:** Use individual tools for fine-grained control.

| Tool | Description |
|------|-------------|
| `start_test_pipeline` | Full pipeline: generate → execute → evaluate → analyze |
| `list_pipeline_jobs` | List past/running pipeline jobs |
| `get_pipeline_job` | Get pipeline job status and results |
| `create_test_suite` | Auto-generate test cases (6 categories) |
| `list_test_suites` | List test suites, filter by status |
| `get_test_suite` | Get test suite with all test cases |
| `get_test_suite_progress` | Lightweight progress polling during generation |
| `add_test_case` | Manually add a test case to a suite |
| `create_test_run` | Execute and evaluate a test suite |
| `list_test_runs` | List test run history |
| `get_test_run` | Get full run results (pass/fail, scores, responses) |
| `create_run_analysis` | AI-powered analysis of test results |
| `get_run_analysis` | Get analysis — suggestions and recommended instruction |
| `analyze_failures` | Phase 1: diagnose failures |
| `generate_instruction` | Phase 2: generate improved instruction from analysis |
| `get_agent_test_overview` | Agent testing summary — suites, runs, pass rate |

## Running Modes

### 1. HTTP MCP Server (Streamable HTTP)

Serves `/mcp` (Streamable HTTP, spec 2025-03-26) and `/healthz`. Compatible with
Cursor Remote MCP, Claude Desktop, and any MCP-compliant client.

```bash
astra server
# or: go run ./cmd/main.go
```

### 2. Interactive CLI Agent

Chat-style interface with a built-in LLM that orchestrates all 64 tools autonomously.

**Recommended first-time setup (pip / `go install`):** save your LLM key once, then start the REPL:

```bash
astra configure    # choose OpenRouter / OpenAI / Gemini, then API key (hidden), base URL, model → ~/.astra/llm.json
astra
```

Non-interactive examples:

```bash
astra configure --provider openai --api-key sk-...
astra configure --provider gemini --api-key AIza...    # Google AI Studio key
astra configure --provider openrouter --api-key sk-or-...
```

**Or** use environment variables (they **override** the file when set):

```bash
export LLM_API_KEY=sk-...          # or OPENAI_API_KEY
export LLM_MODEL=gpt-4o            # optional
export LLM_BASE_URL=https://...    # optional

astra
# or: go run ./cmd/cli/main.go
```

`astra configure` also accepts flags for scripts: `astra configure --api-key sk-... --base-url https://openrouter.ai/api/v1 --model anthropic/claude-sonnet-4`. Use `astra configure --show-path` to print the config file path.

If no key is in the file **and** `LLM_API_KEY` / `OPENAI_API_KEY` is unset, a **TTY** session prompts for **provider (1–3)** then **API key** (hidden), and applies the right default base URL and model. Non-interactive use requires `astra configure --api-key ...` (optional `--provider`) or env vars.

Default API bases: **OpenRouter** `https://openrouter.ai/api/v1`, **OpenAI** `https://api.openai.com/v1`, **Gemini** `https://generativelanguage.googleapis.com/v1beta/openai` (AI Studio key, OpenAI-compatible).

Commands: `exit`/`quit`/`q` to quit, `clear`/`reset` to start a new conversation.

## OAuth Login (Browser)

The CLI supports browser-based OAuth login using PKCE (no client secret needed). After authorization, the access token is saved locally and used automatically for all subsequent tool calls.

### Quick Login

```bash
# Login via browser (default: dev environment)
astra auth --oauth --env dev

# Login to production
astra auth --oauth --env prod

# Print authorize URL without opening browser
astra auth --oauth --env dev --no-open
```

### How It Works

1. The CLI generates a PKCE code challenge (S256) and starts a local HTTP server on `127.0.0.1:8787/callback`
2. Your browser opens the Astra authorization page
3. After you log in and approve, the browser redirects to the local server with an authorization code
4. The CLI exchanges the code for an access token (using the PKCE code verifier)
5. The token is saved to `~/.astra/oauth_token.json`

### Auth Flags

| Flag | Default | Description |
|------|---------|-------------|
| `--oauth` | — | Run browser-based OAuth PKCE flow |
| `--token <tok>` | — | Manually save an access token |
| `--env` | `dev` | Environment: `prod`, `stage`, `dev`, `qa` |
| `--scope` | `chat,agents:read,agents:write` | OAuth scopes |
| `--listen-host` | `127.0.0.1` | Callback server bind address |
| `--listen-port` | `8787` | Callback server port |
| `--timeout` | `120` | Seconds to wait for callback |
| `--no-open` | `false` | Don't open browser; print URL only |

### Auth Token Resolution

When making API calls, tokens are resolved in this order:

1. `api_key` argument passed explicitly in tool call
2. `ASTRA_API_KEY` environment variable
3. `ASTRA_ACCESS_TOKEN` environment variable
4. `ASTRA_OAUTH_ACCESS_TOKEN` environment variable
5. `~/.astra/oauth_token.json` (saved by `auth --oauth`)

Existing API key workflows are fully compatible -- OAuth login is additive and does not break any existing authentication method.

## Quick Start

```bash
go run ./cmd/main.go

# List all tools
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

# List agents
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":2,"method":"tools/call",
    "params":{"name":"list_agents","arguments":{"api_key":"sk_live_xxx"}}
  }'

# Start full test pipeline
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":3,"method":"tools/call",
    "params":{"name":"start_test_pipeline","arguments":{
      "api_key":"sk_live_xxx","agent_id":"uuid","user_id":"tester"
    }}
  }'

# Generate improved instruction from test failures
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":4,"method":"tools/call",
    "params":{"name":"generate_instruction","arguments":{
      "api_key":"sk_live_xxx",
      "instruction":"You are a helpful sales agent...",
      "improvement_suggestions":["Be more specific about pricing","Handle objections better"]
    }}
  }'
```

## Docker Deployment

```bash
# Build
docker build -t astra-insight-mcp .

# Run (image entrypoint: astra server)
docker run -p 8080:8080 \
  -e ASTRA_GATEWAY_BASE_URL=https://astra-gateway.wati.io \
  -e ASTRA_API_KEY=sk_live_... \
  -e RATE_LIMIT_PER_MIN=60 \
  astra-insight-mcp

# Verify
curl http://localhost:8080/healthz
# → ok

# Initialize MCP session
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'
# → returns Mcp-Session-Id header
```

### Kubernetes

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: astra-insight-mcp
spec:
  replicas: 2
  selector:
    matchLabels:
      app: astra-insight-mcp
  template:
    metadata:
      labels:
        app: astra-insight-mcp
    spec:
      containers:
        - name: mcp
          image: astra-insight-mcp:latest
          ports:
            - containerPort: 8080
          env:
            - name: ASTRA_GATEWAY_BASE_URL
              value: "https://astra-gateway.wati.io"
            - name: ASTRA_API_KEY
              valueFrom:
                secretKeyRef:
                  name: astra-secrets
                  key: api-key
            - name: RATE_LIMIT_PER_MIN
              value: "120"
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 30
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 10
          resources:
            requests:
              memory: "64Mi"
              cpu: "100m"
            limits:
              memory: "256Mi"
              cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
  name: astra-insight-mcp
spec:
  selector:
    app: astra-insight-mcp
  ports:
    - port: 80
      targetPort: 8080
  type: ClusterIP
```

## Authentication

Every tool call requires an auth token. The token is resolved automatically from multiple sources (highest priority first):

| Priority | Source | Example |
|----------|--------|---------|
| 1 | `api_key` argument in tool call | `"api_key": "sk_live_xxx"` |
| 2 | `ASTRA_API_KEY` env var | `export ASTRA_API_KEY=sk_live_xxx` |
| 3 | `ASTRA_ACCESS_TOKEN` env var | `export ASTRA_ACCESS_TOKEN=astra_at_xxx` |
| 4 | `ASTRA_OAUTH_ACCESS_TOKEN` env var | `export ASTRA_OAUTH_ACCESS_TOKEN=astra_at_xxx` |
| 5 | `~/.astra/oauth_token.json` file | `{"access_token": "astra_at_xxx"}` |

Both API keys (`sk_live_*`) and OAuth access tokens (`astra_at_*`) are accepted -- the gateway treats them identically as Bearer tokens.

### Using OAuth tokens

If you authenticate via the Astra OAuth flow (e.g. `astra auth --oauth`), the CLI writes a token file to `~/.astra/oauth_token.json`. astra-insight-mcp will pick it up automatically -- no additional configuration needed.

```bash
# Option A: set an env var
export ASTRA_ACCESS_TOKEN=astra_at_xxx
astra

# Option B: use the token file (written by astra auth)
# ~/.astra/oauth_token.json -> {"access_token": "astra_at_xxx", ...}
astra
# -> token is auto-resolved, no env var needed
```

## Environment Variables

### MCP Server

| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `8080` | Server listen port |
| `ASTRA_GATEWAY_BASE_URL` | `https://astra-gateway.wati.io` | Gateway API base URL |
| `ASTRA_API_KEY` | -- | Astra API key (`sk_live_*`), auto-injected into all tool calls |
| `ASTRA_ACCESS_TOKEN` | -- | OAuth access token (`astra_at_*`), used if `ASTRA_API_KEY` is not set |
| `ASTRA_OAUTH_ACCESS_TOKEN` | -- | Alternative OAuth env var name |
| `RATE_LIMIT_PER_MIN` | `60` | Max requests per IP per minute |
| `ASTRA_LOG_LEVEL` | `info` | Service JSON log level: `debug`, `info`, `warn`, `error`. Agent/LLM iteration traces only at `debug` |
| `ASTRA_LOG_DEBUG` | — | Set to `1` to force `debug` (same as `ASTRA_LOG_LEVEL=debug`) |

### CLI Agent (additional)

| Variable | Default | Description |
|----------|---------|-------------|
| `~/.astra/llm.json` | — | Created by `astra configure` (`api_key`, `base_url`, `model`, optional `fast_model`; file mode `0600`) |
| `LLM_API_KEY` / `OPENAI_API_KEY` | -- | If set, overrides `api_key` from the file |
| `LLM_BASE_URL` | — | If set, overrides `base_url` from the file; if still empty, the LLM client defaults to OpenAI’s API URL |
| `LLM_MODEL` | `anthropic/claude-sonnet-4` | If `LLM_MODEL` is unset, the value from the file is used; otherwise this default |
| `LLM_FAST_MODEL` | (derived) | Optional; overrides derived fast model from the file or heuristics |
