# 🔨 mcp-forge

**pytest for MCP servers — test any server, no LLM required.**

[![PyPI](https://img.shields.io/pypi/v/forge-mcp)](https://pypi.org/project/forge-mcp/)
[![Python](https://img.shields.io/pypi/pyversions/forge-mcp)](https://pypi.org/project/forge-mcp/)
[![CI](https://github.com/ioteverythin/forgemcp/actions/workflows/ci.yml/badge.svg)](https://github.com/ioteverythin/forgemcp/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

mcp-forge is a testing framework for [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) servers. It works with **any** MCP server — Python, TypeScript, Go, Rust — and requires **zero LLM calls**. Tests are deterministic, fast, and free to run in CI/CD.

## Why?

Most MCP server testing today looks like this: spin up Claude Desktop, type a prompt, squint at the output, and hope it called the right tool. That's vibe-testing.

mcp-forge gives you real tests:

```python
import pytest
from mcp_forge import MCPTestClient

@pytest.fixture
async def client():
    async with MCPTestClient.from_command("python my_server.py") as c:
        yield c

async def test_tool_exists(client):
    tools = await client.list_tools()
    assert "search_files" in [t.name for t in tools]

async def test_search_works(client):
    result = await client.call_tool("search_files", {"query": "README"})
    assert result.is_success
    assert "README.md" in result.text

async def test_search_snapshot(client, forge_snapshot):
    result = await client.call_tool("search_files", {"query": "README"})
    forge_snapshot.assert_match(result)
```

## Installation

```bash
pip install forge-mcp
```

With FastMCP in-memory testing support:
```bash
pip install "forge-mcp[fastmcp]"
```

## Quick Start

### 1. Write tests directly

```python
from mcp_forge import MCPTestClient

async def test_my_server():
    async with MCPTestClient.from_command("python my_server.py") as client:
        # Check tools are exposed
        tools = await client.list_tools()
        assert len(tools) > 0

        # Call a tool
        result = await client.call_tool("greet", {"name": "World"})
        assert result.is_success
        assert "Hello" in result.text
```

### 2. Auto-generate tests from a running server

```bash
forge init "python my_server.py" -o test_my_server.py
```

This connects to your server, discovers all tools, and generates a complete test file with example calls for each tool.

### 3. Record fixtures for snapshot testing

```bash
forge record "python my_server.py" -o fixtures/
```

### 4. Validate schemas

```bash
forge schema "python my_server.py"
```

## Connecting to Servers

### STDIO (local servers — most common)

```python
# Python server
async with MCPTestClient.from_command("python my_server.py") as client:
    ...

# TypeScript server
async with MCPTestClient.from_command("npx my-mcp-server") as client:
    ...

# uvx-installed server
async with MCPTestClient.from_command("uvx my-mcp-package") as client:
    ...

# With environment variables
async with MCPTestClient.from_command(
    "python my_server.py",
    env={"API_KEY": "test-key", "DATABASE_URL": "sqlite:///test.db"},
) as client:
    ...
```

### FastMCP in-memory (zero overhead)

```python
from my_server import mcp  # your FastMCP instance

async with MCPTestClient.from_fastmcp(mcp) as client:
    ...
```

## Features

### Tool Testing

```python
async def test_tool_call(client):
    result = await client.call_tool("calculate", {"expression": "2 + 2"})
    assert result.is_success
    assert result.text == "4"

async def test_tool_error_handling(client):
    result = await client.call_tool("calculate", {"expression": ""})
    assert result.is_error

async def test_tool_timeout(client):
    # Tool calls have configurable timeouts
    result = await client.call_tool("slow_operation", timeout=5.0)
```

### Snapshot Testing

```python
async def test_snapshot(client, forge_snapshot):
    result = await client.call_tool("get_user", {"id": "123"})

    # First run: saves snapshot to __snapshots__/
    # Next runs: compares against saved snapshot
    forge_snapshot.assert_match(result)

    # Ignore volatile fields
    forge_snapshot.assert_match(result, ignore_keys=["timestamp", "request_id"])

async def test_schema_snapshot(client, forge_snapshot):
    tools = await client.list_tools()
    # Detect when tool schemas change unexpectedly
    forge_snapshot.assert_schema_unchanged(tools)
```

Update snapshots:
```bash
pytest --update-snapshots
```

### Schema Validation

```python
from mcp_forge import assert_tool_schema, assert_result_matches_schema

async def test_tool_has_correct_schema(client):
    tools = await client.list_tools()
    search_tool = next(t for t in tools if t.name == "search")

    assert_tool_schema(search_tool, {
        "type": "object",
        "required": ["query"],
        "properties": {
            "query": {"type": "string"},
            "limit": {"type": "integer"},
        },
    })
```

### Resource Testing

```python
async def test_resources(client):
    resources = await client.list_resources()
    assert len(resources) > 0

    content = await client.read_resource("file:///README.md")
    assert content.is_success
```

### Prompt Testing

```python
async def test_prompts(client):
    prompts = await client.list_prompts()
    assert "summarize" in [p.name for p in prompts]

    prompt = await client.get_prompt("summarize", {"text": "Hello world"})
    assert prompt.is_success
```

## CLI Reference

```
forge init <command>       Generate test file from a running server
forge record <command>     Record tool responses as test fixtures
forge schema <command>     Dump tool schemas (JSON/YAML/table)
forge check <command>      Run built-in health checks
```

## Pytest Fixtures

mcp-forge auto-registers these fixtures via the pytest plugin:

| Fixture | Description |
|---------|-------------|
| `forge_snapshot` | Snapshot manager for the current test |
| `forge_server` | Factory for creating MCPTestClient instances |

## Comparison with Existing Tools

| Feature | mcp-forge | FastMCP testing | pytest-mcp | mcp-validator |
|---------|-----------|-----------------|------------|---------------|
| Any MCP server | ✅ | ❌ FastMCP only | ✅ | ✅ |
| No LLM required | ✅ | ✅ | ❌ | Partial |
| pytest-native | ✅ | Manual | ✅ | ❌ CLI only |
| Snapshot testing | ✅ | ❌ | ❌ | ❌ |
| Auto-gen fixtures | ✅ | ❌ | ❌ | ❌ |
| Schema validation | ✅ | ❌ | ❌ | ✅ |

## Roadmap

- **v0.1** (current): pytest plugin, STDIO transport, snapshots, schema validation
- **v0.2**: Fuzz testing, contract testing, HTTP/SSE transport, GitHub Action
- **v0.3**: VSCode extension, performance testing, coverage reports

## Contributing

Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

## License

MIT
