# MCP Observatory

MCP Observatory now includes a **two-phase execution pattern** for high-risk MCP tool calls, with a generic proposer/verifier wrapper that can be reused by any tool:

1. **PROPOSE**: plan/simulate, evaluate uncertainty/integrity, no side effects.
2. **COMMIT**: execute side effects only when a signed commit token is valid.

## Two-Phase Sequence (Text Diagram)

```text
Client
  -> transfer_funds_propose(amount,to)
      -> scoring(output_instability, numeric_variance, prompt_drift)
      -> decision:
          - blocked: deterministic fallback (create_draft), no side effects
          - allowed: issue signed commit_token bound to tool args hash
  <- {proposal_id, commit_token?}

Client
  -> transfer_funds_commit(proposal_id, commit_token, amount, to)
      -> verify signature + expiry + proposal existence + args_hash binding + nonce replay
      -> if valid: perform side effect (funds transfer)
      -> else: block with explicit reason
  <- commit outcome
```

## New Modules

- `mcp_observatory/proposal_commit/hashing.py`
  - canonical JSON hashing for stable `tool_args_hash`
  - normalized `prompt_hash`
- `mcp_observatory/proposal_commit/scoring.py`
  - `output_instability = 1 - jaccard_similarity`
  - `numeric_variance` from extracted numbers
  - `prompt_drift` from prompt hash vs baseline
  - weighted renormalized `composite_score`
  - demo `model_generate(prompt, temperature)` stub
- `mcp_observatory/proposal_commit/token.py`
  - HMAC-SHA256 token issue/verify
  - payload fields: `token_id, proposal_id, tool_name, tool_args_hash, issued_at, expires_at, nonce, composite_score`
- `mcp_observatory/proposal_commit/proposer.py`
  - generic `ToolProposer.propose(...)` for any tool name/args
  - deterministic blocked fallback
- `mcp_observatory/proposal_commit/verifier.py`
  - commit verification and nonce replay protection
- `mcp_observatory/proposal_commit/storage.py`
  - in-memory storage fallback
  - optional Postgres storage via `asyncpg`
- `mcp_observatory/core/wrapper_api.py`
  - generic `InvocationWrapperAPI` for wrapping either `agent` or `model` invocations
  - captures input/output hashes, token/cost metrics, and emits `allow/review/block` decisions
- `mcp_observatory/instrument.py`
  - adds `instrument_wrapper_api(...)` helper for fast wrapper setup
- `mcp_observatory/demo/server.py`
  - MCP-like tools:
    - `transfer_funds_propose`
    - `transfer_funds_commit`
- `mcp_observatory/demo/run_demo.py`
  - propose -> commit -> replay-attempt demo
- `sql/schema.sql`
  - Postgres tables: `proposals`, `commits`, `nonces`, `tool_prompt_baselines`

## Security / Verification Rules

Commit verifies all of the following:

- token signature is valid (`bad_signature` on failure)
- token not expired (`expired`)
- proposal exists and was allowed (`unknown_proposal`)
- commit args hash equals token payload args hash (`args_hash_mismatch`)
- nonce has not already been used (`nonce_replay`)

## Deterministic Fallback on Proposal Block

Blocked proposal response is deterministic and side-effect free:

```json
{
  "status": "blocked",
  "action": "create_draft",
  "reason": "low_integrity",
  "draft": {"tool": "transfer_funds", "amount": 100, "to": "acct_123"}
}
```

## Running the Demo

### Without Postgres (default)

No env vars needed; in-memory store is used.

```bash
python -m mcp_observatory.demo.run_demo
```

### With Postgres

1. Set DSN:

```bash
export MCP_OBSERVATORY_PG_DSN='postgresql://user:pass@localhost:5432/postgres'
```

2. Apply schema:

```bash
psql "$MCP_OBSERVATORY_PG_DSN" -f sql/schema.sql
```

3. Run demo:

```bash
python -m mcp_observatory.demo.run_demo
```

## Testing

```bash
PYTHONPATH=. pytest -q
```

The suite includes tests for token verification, hash stability, replay protection, and expired-token rejection.

## Real-World MCP Scenario Demo (10 End-to-End Flows)

A realistic MCP server example is available at:

- `mcp_observatory/demo/real_world_server.py`
- executable shim: `examples/real_world_mcp_server.py`
- executable client: `examples/real_world_mcp_client.py`
- prompt-to-invocation MVP: `examples/prompt_to_mcp_invocation_mvp.py`
- OpenAI GPT utility: `examples/openai_prompt_to_mcp_invocation.py`

It includes:

- 10 distinct prompts mapped to 10 different MCP tool handlers
- per-invocation annotations (e.g. `destructiveHint`, `idempotentHint`, `openWorldHint`)
- **proposal/commit** execution for HIGH-risk tools (no secondary-response gating)
- irreversible actions never pass a secondary LLM response
- simulated LLM responses and grounding summaries for standard-risk tools
- deterministic fallback routing for blocked/review-required scenarios
- prompt-to-invocation MVP now extracts required tool parameters from user prompts before server invocation
- openai-gpt utility for service selection + parameter extraction + MCP invocation

Run server demo:

```bash
python examples/real_world_mcp_server.py
```

Run client demo (client interacting with server):

```bash
python examples/real_world_mcp_client.py
```

Run prompt -> LLM planner -> server invocation MVP:

```bash
python examples/prompt_to_mcp_invocation_mvp.py
```

Run OpenAI GPT utility (manual, requires `OPENAI_API_KEY`):

```bash
python examples/openai_prompt_to_mcp_invocation.py
```

Optional (recommended): install OpenAI SDK for first-class client support (client auto-detects and uses SDK when available; otherwise it uses HTTPS fallback):

```bash
pip install -e .[openai]
```


## Wrapper API (Agent or Model Invocation)

Use the wrapper to route either agent-side orchestration calls or direct model calls through a single observability envelope.

```python
from mcp_observatory.instrument import instrument_wrapper_api

wrapper = instrument_wrapper_api("my-service")

result = await wrapper.invoke(
    source="agent",
    model="gpt-4o-mini",
    prompt="Generate deployment plan",
    input_payload={"request_id": "abc123", "task": "deployment_plan"},
    call=lambda: {"plan": "blue-green rollout"},
)

print(result.decision.action)  # allow/review/block
print(result.span.cost_usd)
```

Dual-run measurement is supported with `dual_invoke=True` and shadow parameters (`shadow_source`, `shadow_model`, `shadow_agent_params`, `shadow_model_params`, and `shadow_call`) to compare alternate execution paths and capture disagreement metrics.

The wrapper output (`WrapperResult`) includes:

- `output`: raw callable output
- `span`: captured telemetry metrics (tokens, cost, hashes, timing)
- `decision`: policy decision suitable for downstream execution routing
- `shadow_output` and `shadow_span` (when `dual_invoke=True`) with comparison metrics on primary span (`shadow_disagreement_score`, `shadow_numeric_variance`)
