# qmcp Server

A Model Context Protocol (MCP) server that connects AI coding assistants to q/kdb+ databases.

## Why qmcp? Performance Results

The best LLMs have 57% failure rates on q/kdb+ programming tasks (26% with Pass@10). **qmcp gives LLMs interactive access to q servers, reducing failures to 6.71%** on the full HumanEval corpus. With optional Qython translation (Python-like syntax → q code), **failure rates drop to 0.61%**.

📊 [Full benchmark results](https://medium.com/@gabiteodoru/0-6-failure-rate-solving-llm-code-generation-for-q-kdb-4b3ed29f64bd)

🎥 [Installation tutorial for Claude Code CLI](https://www.youtube.com/watch?v=htrPzm6yY2Y) (4m30s)

## What is Qython?

**Qython** is a Python-like language that compiles to q/kdb+ code. Instead of learning q's unfamiliar syntax, AI assistants can write familiar Python-like code that gets translated to efficient q.

**qmcp** provides the infrastructure for AI assistants to work with q/kdb+ databases, with optional cloud-based Qython translation.

## Deployment Options

### Local Mode (Default)
**Privacy-First • Zero Cloud Dependencies**

- Direct q connection with raw q console output
- Your code **never leaves your machine**
- Open source and fully auditable
- Perfect for: Sensitive/proprietary code, production systems
- Configuration: `cloud_enabled = false` (default)

### Cloud-Enhanced Mode
**With Qython Translation & Enhanced Formatting**

- **Qython code translator**: AI writes Python-like code → cloud translates to q
- **Enhanced output formatting**: Raw q results → cloud formats for readability
- Your code AND query results are sent to our cloud service
- Free for evaluation, learning, and non-commercial use (API key via email)
- **Production/commercial use requires enterprise license**
- Perfect for: Learning q, evaluating Qython, non-sensitive development
- Configuration: `cloud_enabled = true` + API key

### Enterprise On-Premise

For organizations requiring Qython translation with complete data sovereignty, we offer on-premise deployment where all translation happens in your infrastructure. Contact for licensing information.

## Core Features

- Connect to q/kdb+ servers
- Execute q queries and commands
- Persistent connection management
- Intelligent async query handling with configurable timeouts
- Programmatic query cancellation (Ctrl+C equivalent)
- Graceful handling of long-running queries
- Table introspection tools (list tables, describe schema)

## Qython Features (Cloud or Enterprise)

- **Qython language translator**: Write Python-like syntax that compiles to q
- **Enhanced output formatting**: Beautiful display of q data structures
- **Rich type handling**: Proper representation of q nulls, infinities, and special values
- **QythonDB**: SQL-like table operations with Python syntax
- **Comprehensive documentation**: Built-in help resources for Qython syntax

## Windows Users: WSL Recommendation

**⚠️ Important for Windows users**: For optimal functionality, it is highly recommended to run both the MCP server and your q session inside WSL (Windows Subsystem for Linux). This ensures the server can interrupt infinite loops and runaway queries that LLMs might accidentally generate.

Running the MCP server on Windows (outside WSL) disables SIGINT-based query interruption functionality, which is critical for escaping problematic queries during AI-assisted development sessions.

## Architecture & Design Philosophy

### Intended Goals

**qmcp** is designed to provide AI coding assistants with **controlled access** to q/kdb+ databases for development and debugging workflows:

1. **Development-Focused**: Optimized for coding tools working with debug/dev q servers
2. **Query Control**: AI can interrupt long-running queries (equivalent to developer Ctrl+C)
3. **Predictable Behavior**: Sequential execution prevents resource conflicts during development
4. **Configurable Timeouts**: Customizable timing for different development scenarios

### Design Logic

The server architecture makes deliberate choices for AI-assisted development workflows:

#### **Single Connection Model**
- **Why**: Simplifies development debugging - one connection, clear state
- **Benefit**: Matches typical developer workflow with single q session
- **Implementation**: One persistent connection per MCP session

#### **Sequential Query Execution**
- **Why**: Development environments don't need concurrent query support
- **Benefit**: Predictable resource usage, easier debugging, prevents query interference
- **Implementation**: New queries rejected while another is running

#### **Smart Async Switching with Configurable Timeouts**
```
Fast Query (< async switch timeout)  →  Return result immediately
Slow Query (> async switch timeout)  →  Switch to async mode
                                     →  Auto-interrupt after interrupt timeout (if configured)
```
- **Why**: Keeps AI coding sessions responsive while allowing complex development queries
- **Benefit**: Immediate feedback for quick queries, progress tracking for analysis
- **Customization**: All timeouts configurable via MCP tools

#### **AI-Controlled Query Interruption**
- **Why**: AI coding tools need ability to cancel runaway queries (like developer Ctrl+C)
- **How**: MCP server locates q process by port and sends SIGINT after configurable timeout
- **Benefit**: Prevents development sessions from hanging on problematic queries
- **Limitations**: SIGINT functionality disabled when:
  - MCP server runs on Windows (outside WSL)
  - MCP server and q session run on opposite sides of WSL/Windows divide

#### **Development-Oriented Process Management**
- **Why**: Coding tools work with user-managed development q servers
- **Benefit**: Developer controls q server lifecycle, AI controls query execution
- **Design**: MCP server provides query interruption capability without server lifecycle management

### Why This Design Makes Sense for Coding Tools

1. **Development Workflow**: Matches how developers interact with q - single session, iterative queries
2. **AI Safety**: Prevents AI from overwhelming development environments with concurrent requests
3. **Debugging-Friendly**: Sequential execution makes it easier to trace issues
4. **Responsive**: Async handling prevents AI coding sessions from blocking
5. **Configurable**: Timeouts can be tuned for different development scenarios

This architecture provides AI coding assistants with effective q/kdb+ access while maintaining the predictable, controlled environment that development workflows require.

## Requirements

- Python 3.8+
- Access to a q/kdb+ server
- `uv` (for lightweight installation) or `pip` (for full installation)

## Quick Start

For first-time users, the fastest way to get started:

1. Start a q server:
   ```bash
   q -p 5001
   ```
2. Add qmcp to Claude CLI:
   ```bash
   claude mcp add qmcp uvx qmcp
   ```
3. Start using Claude CLI:
   ```bash
   claude
   ```
   Then interact with qmcp:
   ```
   > connect to port 5001 and compute 2+2

   ● qmcp:connect_to_q (MCP)(host: "5001")
     ⎿  true

   ● qmcp:query_q (MCP)(command: "2+2")
     ⎿  4
   ```

## Installation

### Prerequisites: Installing uv (Recommended)

If you don't have `uv` installed, install it first:

**Linux/macOS/WSL:**
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

**Windows (PowerShell):**
```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```

This adds `uv` (and `uvx`) to your PATH automatically.

### MCP Server Installation (Primary Use)

**Recommended: uvx (simplest)**
```bash
claude mcp add qmcp uvx qmcp
```

Benefits:
- One command setup after uv is installed
- Auto-downloads qmcp from PyPI when needed
- No virtual environment management
- Uses efficient global cache

**For Claude Desktop/VSCode/Cursor:**

```json
{
  "mcpServers": {
    "qmcp": {
      "type": "stdio",
      "command": "uvx",
      "args": ["qmcp"]
    }
  }
}
```

**Alternative: pip (if you already have Python in PATH)**

On modern Linux systems (Ubuntu 22.04+, Debian 12+, WSL), use `--user` to avoid system Python restrictions:
```bash
pip install --user qmcp
claude mcp add qmcp qmcp
```

Or use a virtual environment:
```bash
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install qmcp
claude mcp add qmcp venv/bin/qmcp
```

On other systems:
```bash
pip install qmcp
claude mcp add qmcp qmcp
```

**For Claude Desktop/VSCode/Cursor (pip-installed):**

```json
{
  "mcpServers": {
    "qmcp": {
      "type": "stdio",
      "command": "qmcp"
    }
  }
}
```

### Advanced: Python Library Usage

If you need to import qmcp functions in your Python code:

```bash
# Using pip
pip install qmcp

# Using uv (recommended for projects)
uv add qmcp
```

Then in your code:
```python
from qmcp import <functionality>  # Library API documentation coming soon
```

### Advanced: Running from Source

For development or contributing:

```bash
# Clone the repository
git clone https://github.com/gabiteodoru/qmcp.git
cd qmcp

# Claude CLI
claude mcp add qmcp uvx --from . qmcp

# Claude Desktop
# Add this to your config file:
```

```json
{
  "mcpServers": {
    "qmcp": {
      "command": "uv",
      "args": [
        "--directory",
        "/absolute/path/to/qmcp",
        "run",
        "qmcp"
      ]
    }
  }
}
```

## Configuration

### Default Configuration

By default, qmcp operates in **local mode** (privacy-first, no cloud dependencies):

```toml
[default]
cloud_enabled = false  # No cloud services, your code stays local
output_format = "q"    # Raw q console output
```

To find your configuration file location, ask your AI assistant: "Where is my qmcp config file?"

### Enabling Cloud Features

Cloud services provide two types of enhancement:

1. **Code Translation**: When you use `translate_qython_to_q` and related tools
   - Python-like code → q code translation
   - Requires `cloud_enabled = true`

2. **Output Formatting**: When you run ANY q code with `output_format = "qython"`
   - Raw q output → beautifully formatted display
   - Handles q nulls, infinities, tables, lists, etc. with proper formatting
   - Applies to all q expressions: calculations, function results, data queries
   - Requires `cloud_enabled = true` AND `output_format = "qython"`

Both features send data to the cloud. To enable them:

#### Step 1: Request an API Key

Ask your AI assistant:
```
Can you get me a qmcp API key? My email is user@example.com
```

The AI will call the `request_api_key` tool and send you an email with your API key.

#### Step 2: Configure API Key

Your AI assistant can automatically add the API key to your config, or you can add it manually:

```toml
[cloud]
api_key = "sk_xxxxxxxxxxxxx"  # Paste your API key from email
```

#### Step 3: Enable Cloud Services

```toml
[default]
cloud_enabled = true       # Enable cloud features
output_format = "qython"   # Use enhanced formatting
```

That's it! Qython translation is now available.

#### Privacy Note

**What gets sent to the cloud:**

When `cloud_enabled = true`:
- **Code translation tools**: Your Qython code is sent for translation to q
- **Output formatting** (if `output_format = "qython"`): All q output (function results, calculations, data, etc.) is sent for formatting
- **API keys**: Stored locally in plain text in your config file (never sent over network except as auth header)
- All requests may be logged for service improvement

**For production use or proprietary code**: Keep `cloud_enabled = false` (raw q output only) or contact us about on-premise deployment.

**Terms of Service**: By using the cloud API, you agree to our terms:
- ✅ Free for: Personal projects, evaluation, learning, testing
- ⚠️ Production/commercial use requires enterprise license
- See full terms in your API key email or contact for details

### Configuration Reference

```toml
[default]
# Enable cloud services for Qython code translation and enhanced output formatting
cloud_enabled = false  # Set to true to enable cloud features

# Output format for query results
# "q" = raw q console output (local, no cloud)
# "qython" = enhanced formatting (requires cloud_enabled = true)
output_format = "q"

# LLM mode (reserved for future use)
LLM = "claude"  # claude, copilot

[cloud]
# API key for cloud services (obtain via request_api_key tool)
api_key = "your-key-here"
```

## Usage

### Starting the MCP Server

**After full installation:**
```bash
qmcp
```

**With lightweight installation:**
The server starts automatically when Claude CLI uses it (no manual start needed).

### Configuration

qmcp creates a configuration file in your default user directory for server settings, credentials, and behavior. For advanced use cases requiring multiple configurations, you can override this by placing `qmcp.toml` or `.qmcp.toml` in the directory where you start qmcp.

**Configuration structure:**
```toml
[default]
# Operational defaults (can be overridden per-server)
connection_timeout = 2      # Seconds to wait for connection
async_timeout = 1           # Seconds before switching to async mode
interrupt_timeout = 10      # Seconds before auto-interrupt (0 to disable)
print_to_async = true       # true = async IPC (LLM), false = console
# console_size = [25, 80]   # [rows, cols] - omit or [] to not change on connect

[servers.default]
host = "localhost"
port = 5001
user = ""  # blank means no authentication
password = ""
# Inherits all [default] operational settings

[servers.prod]
host = "prod-server.com"
port = 5001
user = "produser"
password = "prod-secret"
print_to_async = false      # Override: production uses console
interrupt_timeout = 30      # Override: longer timeout for prod
# Other settings inherit from [default]
```

**Get config path:** Use `get_config_file_path()` tool to find your config file location.

### Connection Logic

The `connect_to_q(host)` tool uses flexible connection modes:

1. **No parameters**: Load `servers.default` from config
   - `connect_to_q()` → Uses all default settings
2. **Port number**: Load `servers.default`, override port
   - `connect_to_q("5002")` → localhost:5002 with default auth
3. **Server name**: Load `servers.{name}` from config
   - `connect_to_q("prod")` → Full prod server config
4. **Connection string** (has `:`): Use as-is, bypass config
   - `connect_to_q("myhost:5001:user:pass")` → Direct connection

**Auto-configuration:** After connecting, qmcp automatically:
- Sets print mode from server's `print_to_async` setting
- Configures console dimensions if `console_size = [rows, cols]` is set (omitted by default)
- Applies timeout settings for async operations and interrupts

**Configuration inheritance:** Servers inherit operational settings from `[default]` but can override any setting. Server identity fields (host, port, user, password) never inherit between servers.

### Available Tools

#### Core Tools

**Configuration:**
- `get_config_file_path` - Get the path to the qmcp configuration file
- `reload_config` - Reload configuration from file after manual edits

**Connection Management:**
- `connect_to_q` - Connect to q/kdb+ server with flexible fallback logic
- `set_timeout_connection` - Configure connection timeout

**Query Execution:**
- `query_q` - Execute q queries with intelligent async timeout control
- `get_current_task_result` - Retrieve result of completed async query
- `interrupt_current_query` - Send SIGINT to interrupt running queries (Ctrl+C equivalent)
- `get_last_query_result_q_view` - Get the q console view of the last query result
- `set_maximum_console_view` - Set maximum console view dimensions (rows, cols)

**Timeout Configuration:**
- `set_timeout_switch_to_async` - Configure when queries switch to async mode
- `set_timeout_interrupt_q` - Configure when to auto-interrupt long queries
- `get_timeout_settings` - View current timeout configuration

**Table Introspection:**
- `list_tables` - List all tables with metadata (type, row count, columns, partitioning)
- `describe_table` - Get column names and types for a table

#### Cloud Authentication

- `request_api_key` - Request API key via email for cloud services

#### Qython Tools

**Translation:**
- `translate_qython_to_q` - Translate Python-like syntax to q code
  - ⚠️ **EXPERIMENTAL**: Limited vocabulary, may produce incorrect code
  - Requires cloud services (`cloud_enabled = true`) or enterprise installation
  - **Please verify all output before use**

- `translate_q_to_qython` - Translate q code to Python-like syntax with AI disambiguation
  - ⚠️ **EXPERIMENTAL**: Uses ParseQ + AI to convert q to readable Qython
  - **Requires q connection first** - run `connect_to_q` before using
  - **Namespace Impact**: Creates variables/functions in `.parseq` namespace
  - May produce incorrect translations for complex expressions
  - **Please verify all output before use**

- `translate_and_run_qython` - Translate Qython code and execute it via IPC
  - Requires active q connection

**File Operations:**
- `translate_qython_file_to_q` - Translate Qython file to q code (returns string)
- `translate_qython_to_q_file` - Translate Qython code string, write to q file
- `translate_qython_file_to_q_file` - Translate Qython file, write to specified q file path
- `run_qython_file_via_IPC` - Translate and execute Qython file via IPC
- `run_q_file_via_IPC` - Execute q file via IPC
- `export_qython_namespace` - Export Qython runtime dependencies to file

**Setup & Documentation:**
- `setup_qython_namespace` - Load Qython runtime utilities into q session
- `qython_help` - Get Qython documentation and help

Report bugs at [GitHub Issues](https://github.com/gabiteodoru/qmcp/issues)

## Known Limitations

When using the MCP server, be aware of these limitations:

### Query Interruption (SIGINT) Limitations
- **Windows Platform**: Query interruption disabled when MCP server runs on Windows (outside WSL)
- **Cross-Platform Setup**: Query interruption disabled when MCP server and q session run on opposite sides of WSL/Windows divide
- **Impact**: LLM cannot automatically escape infinite loops or cancel runaway queries in these configurations

### Data Conversion Limitations
- **Keyed tables**: Operations like `1!table` may fail during pandas conversion
- **String vs Symbol distinction**: q strings and symbols may appear identical in output
- **Type ambiguity**: Use q's `meta` and `type` commands to determine actual data types when precision matters
- **Pandas conversion**: Some q-specific data structures may not convert properly to pandas DataFrames

For type checking, use:
```q
meta table           / Check table column types and structure
type variable        / Check variable type
```

## WSL2 Port Communication (Windows Users)

*Skip this section if you're not on Windows.*

Since Claude CLI is WSL-only on Windows, but you might want to use Windows IDEs or tools to connect to your q server, you need proper port communication between WSL2 and Windows.

### WSL2 Configuration for Port Communication

#### .wslconfig File Setup
Location: `C:\Users\{YourUsername}\.wslconfig`

Add mirrored networking configuration:
```ini
# Mirrored networking mode for seamless port communication
networkingMode=mirrored
dnsTunneling=true
firewall=true
autoProxy=true
```

#### Restart WSL2
Run from Windows PowerShell/CMD (NOT from within WSL):
```powershell
wsl --shutdown
# Wait a few seconds, then start WSL again
```

#### Verify Configuration
Check if mirrored networking is active:
```bash
ip addr show
cat /etc/resolv.conf
```

#### Test Port Communication

Test WSL2 → Windows (localhost):
```bash
# In WSL2, start a server
python3 -m http.server 8000

# In Windows browser or PowerShell
curl http://localhost:8000
```

Test Windows → WSL2 (localhost):
```powershell
# In Windows PowerShell
python -m http.server 8001

# In WSL2
curl http://localhost:8001
```

#### What Mirrored Networking Provides

- ✅ Direct localhost communication both ways
- ✅ No manual port forwarding needed
- ✅ Better VPN compatibility
- ✅ Simplified networking (Windows and WSL2 share network interfaces)
- ✅ Firewall rules automatically handled

### ⚠️ Port 5000 Special Case

**Issue**: Port 5000 has limited mirrored networking support due to Windows service binding.

**Root Cause**:
- Windows `svchost` service binds to `127.0.0.1:5000` (localhost only)
- Localhost-only bindings are not fully mirrored between Windows and WSL2
- This creates an exception to the general mirrored networking functionality

**Port 5000 Communication Matrix**:
- ✅ Windows ↔ Windows: Works (same localhost)
- ❌ WSL2 ↔ Windows: Fails (different localhost interpretation)
- ✅ WSL2 ↔ WSL2: Works (same environment)

**Solutions for Port 5000**:
1. **Use different ports**: 5001, 5002, etc. (recommended)
2. **Stop Windows service**: If not needed
3. **Traditional port forwarding**: For specific use cases

#### Common Services That May Have Localhost-Only Binding
- **Flask development servers** (default `127.0.0.1:5000`)
- **UPnP Device Host service**
- **Windows Media Player Network Sharing**
- **Various development tools**

#### Known Limitations of Mirrored Networking
1. **Localhost-only services**: Not fully mirrored (as confirmed with port 5000)
2. **mDNS doesn't work** in mirrored mode
3. **Some Docker configurations** may have issues
4. **Requires Windows 11 22H2+** (build 22621+)

## License

This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

**Note**: The cloud translation service has separate Terms of Service provided when you request an API key.