# Product Requirements Document: Crustdata MCP Server

## Executive Summary

This document outlines the requirements for building a Model Context Protocol (MCP) server that exposes Crustdata's B2B data platform APIs to AI assistants and LLM-powered applications. Using FastMCP, we will create a production-ready server that enables AI agents to search, enrich, and monitor company and people data through natural language interactions.

---

## Table of Contents

1. [Problem Statement](#problem-statement)
2. [Goals and Objectives](#goals-and-objectives)
3. [Target Users](#target-users)
4. [Scope](#scope)
5. [Functional Requirements](#functional-requirements)
6. [Technical Architecture](#technical-architecture)
7. [API Tool Mappings](#api-tool-mappings)
8. [Security Requirements](#security-requirements)
9. [Testing Strategy](#testing-strategy)
10. [Success Metrics](#success-metrics)
11. [Implementation Phases](#implementation-phases)

---

## 1. Problem Statement

### Current State
- Crustdata provides a comprehensive B2B data platform with 20+ API endpoints
- Users must manually construct API calls, understand complex query parameters, and process JSON responses
- Integration into AI workflows requires custom code for each use case
- No standardized way for AI assistants to access Crustdata's rich company/people data

### Desired State
- AI assistants can directly query Crustdata APIs using natural language
- Tools are discoverable and self-documenting with proper schemas
- Complex multi-step workflows (search → enrich → monitor) are seamless
- Credit usage is transparent and tracked
- Authentication is handled securely

---

## 2. Goals and Objectives

### Primary Goals

1. **Expose all Crustdata APIs as MCP Tools**: Map all 20 API endpoints to well-defined MCP tools with proper input validation and documentation
2. **Enable AI-Powered Data Discovery**: Allow LLMs to search, filter, and analyze company/people data
3. **Support Complex Workflows**: Enable multi-step operations like "find companies → get details → set up monitoring"
4. **Maintain Security**: Implement proper authentication, rate limiting, and credit tracking

### Success Criteria

- All 20 API endpoints exposed as MCP tools
- 95%+ test coverage on tool implementations
- Response time <2s for standard queries (excluding long-running exports)
- Zero security vulnerabilities in auth flow
- Clear error messages for all failure scenarios

---

## 3. Target Users

### Primary Users

1. **AI Assistants (Claude, GPT, etc.)**: Primary consumers of MCP tools
2. **Developer Tools**: IDE integrations, coding assistants needing company data
3. **Automation Platforms**: Workflow automation tools connecting to MCP servers

### User Personas

| Persona | Use Case | Key Tools Needed |
|---------|----------|------------------|
| Sales Rep AI | Find leads matching criteria | Company Search, People Search, Enrichment |
| Research Analyst AI | Deep-dive company analysis | Enrichment, LinkedIn Posts, Web Search |
| Market Intelligence AI | Track competitors | Watcher, LinkedIn Posts, Company Search |
| HR/Recruiting AI | Find decision makers | People Search, People Enrichment |

---

## 4. Scope

### In Scope

- **Company APIs**: Identification, Enrichment, Search (realtime & in-db), LinkedIn posts
- **People APIs**: Enrichment, Search (realtime & in-db), LinkedIn posts
- **Watcher APIs**: Create, Update, Simulate watches
- **Dataset APIs**: Job listings
- **Utility APIs**: Credits, Autocomplete (Company/Person/LinkedIn filters)
- **Web APIs**: Web search, Web fetch
- **Authentication**: Bearer token authentication
- **Resources**: Expose API documentation and schemas as MCP resources

### Out of Scope (Phase 1)

- Admin panel functionality
- Bulk export endpoints (separate async handling needed)
- OAuth flow management (users provide tokens)
- Custom data pipeline integrations
- Real-time streaming/WebSocket connections

---

## 5. Functional Requirements

### FR-1: Tool Definitions

Each API endpoint shall be exposed as an MCP tool with:
- Clear, descriptive name following `crustdata_{category}_{action}` pattern
- Comprehensive docstring explaining purpose, parameters, and example use cases
- Input validation using Pydantic models
- Proper error handling with retriable vs non-retriable errors
- Response formatting optimized for LLM consumption

### FR-2: Resource Definitions

Expose supporting data as MCP resources:
- `crustdata://docs/company-fields` - Company field dictionary
- `crustdata://docs/people-fields` - People field dictionary
- `crustdata://docs/filters` - Available filter options
- `crustdata://credits` - Current credit balance

### FR-3: Authentication

- Accept Crustdata API token via MCP authentication flow
- Validate token on each request
- Pass token to all backend API calls
- Handle token expiration gracefully

### FR-4: Error Handling

| Error Type | HTTP Code | MCP Response | Retriable |
|------------|-----------|--------------|-----------|
| Invalid input | 400 | ToolError with validation details | No |
| Auth failure | 401 | Auth error, prompt for new token | No |
| Not found | 404 | Empty result with message | No |
| Rate limited | 429 | ToolError with retry-after | Yes |
| Server error | 500 | ToolError with generic message | Yes |

### FR-5: Response Formatting

All tool responses shall:
- Return structured JSON that LLMs can parse
- Include relevant metadata (count, pagination info)
- Truncate large arrays with indication of total count
- Format monetary values consistently (USD)
- Convert dates to ISO 8601 format

---

## 6. Technical Architecture

### 6.1 Project Structure

```
mcp-crustdata/
├── mcp/                              # MCP server package
│   ├── src/
│   │   └── crustdata_mcp/
│   │       ├── __init__.py
│   │       ├── server.py              # Main FastMCP server
│   │       ├── config.py              # Configuration management
│   │       ├── auth.py                # Authentication handlers
│   │       ├── client.py              # Crustdata API client
│   │       ├── client_factory.py      # Per-request client management
│   │       ├── tools/
│   │       │   ├── __init__.py
│   │       │   ├── company.py         # Company-related tools
│   │       │   ├── people.py          # People-related tools
│   │       │   ├── watcher.py         # Watcher tools
│   │       │   ├── dataset.py         # Dataset tools
│   │       │   ├── web.py             # Web search/fetch tools
│   │       │   └── utility.py         # Utility tools (credits, autocomplete)
│   │       ├── resources/
│   │       │   ├── __init__.py
│   │       │   └── documentation.py   # Documentation resources
│   │       ├── models/
│   │       │   ├── __init__.py
│   │       │   ├── company.py         # Company input/output models
│   │       │   ├── people.py          # People input/output models
│   │       │   └── common.py          # Shared models
│   │       └── utils/
│   │           ├── __init__.py
│   │           ├── formatting.py      # Response formatters
│   │           └── validation.py      # Input validators
│   ├── tests/
│   │   ├── __init__.py
│   │   ├── conftest.py               # Pytest fixtures
│   │   ├── test_company_tools.py
│   │   ├── test_people_tools.py
│   │   ├── test_watcher_tools.py
│   │   ├── test_dataset_tools.py
│   │   ├── test_web_tools.py
│   │   ├── test_utility_tools.py
│   │   ├── test_resources.py
│   │   ├── test_auth.py
│   │   ├── test_client_factory.py
│   │   └── test_integration.py
│   ├── CLAUDE.md                     # Development documentation
│   ├── README.md                     # User documentation
│   ├── pyproject.toml
│   ├── uv.lock
│   └── PRD-MCP-SERVER.md
├── README.md                         # Repository root README
└── CLAUDE.md                         # Repository root CLAUDE.md
```

### 6.2 Core Components

#### Server Initialization (server.py)

```python
from fastmcp import FastMCP

mcp = FastMCP(
    name="Crustdata MCP Server",
    version="1.0.0",
    description="B2B data platform API access for AI assistants"
)

# Mount tool modules
from .tools import company, people, watcher, dataset, web, utility

mcp.include_router(company.router, prefix="company")
mcp.include_router(people.router, prefix="people")
mcp.include_router(watcher.router, prefix="watcher")
mcp.include_router(dataset.router, prefix="dataset")
mcp.include_router(web.router, prefix="web")
mcp.include_router(utility.router, prefix="util")
```

#### API Client (client.py)

```python
import httpx
from typing import Any

class CrustdataClient:
    BASE_URL = "https://api.crustdata.com"

    def __init__(self, api_token: str):
        self.api_token = api_token
        self._client = httpx.AsyncClient(
            base_url=self.BASE_URL,
            headers={"Authorization": f"Token {api_token}"},
            timeout=30.0
        )

    async def get(self, endpoint: str, params: dict = None) -> dict:
        response = await self._client.get(endpoint, params=params)
        response.raise_for_status()
        return response.json()

    async def post(self, endpoint: str, data: dict = None) -> dict:
        response = await self._client.post(endpoint, json=data)
        response.raise_for_status()
        return response.json()
```

### 6.3 Technology Stack

| Component | Technology | Rationale |
|-----------|------------|-----------|
| Framework | FastMCP 2.x | Pythonic, production-ready MCP framework |
| HTTP Client | httpx | Async-first, modern HTTP client |
| Validation | Pydantic v2 | Type-safe input/output validation |
| Testing | pytest + pytest-asyncio | Standard Python async testing |
| Linting | ruff | Fast, comprehensive linting |
| Type Checking | pyright | Static type analysis |

---

## 7. API Tool Mappings

### 7.1 Company Tools

#### `crustdata_company_identify`
**Endpoint**: `POST /screener/identify/`
**Purpose**: Match companies by domain or name
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| company_name | str | No* | Company name to match |
| company_website | str | No* | Company website/domain |
*At least one required

**Example Use Cases**:
- "Find the company that owns example.com"
- "Identify which company is 'Acme Corp'"

---

#### `crustdata_company_enrich`
**Endpoint**: `GET /screener/company`
**Purpose**: Get detailed company profile
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| company_domain | str | No* | Company domain |
| company_name | str | No* | Company name |
| company_linkedin_url | str | No* | LinkedIn company URL |
| fields | list[str] | No | Specific fields to return |
| enrich_realtime | bool | No | Force real-time enrichment |
*At least one identifier required

**Returns**: Comprehensive company data including:
- Basic info (name, website, founded date)
- Employee metrics (headcount, growth)
- Funding data (total raised, last round)
- Social profiles (LinkedIn, Twitter)
- Location data (HQ, offices)
- Industry classification

---

#### `crustdata_company_search_realtime`
**Endpoint**: `POST /screener/company/search`
**Purpose**: Search companies with filters (real-time)
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| filters | list[Filter] | Yes | Search filters |
| page | int | No | Page number (default: 1) |
| limit | int | No | Results per page (max: 100) |
| sort_by | str | No | Sort field |

**Filter Types**:
- Location (country, state, city)
- Industry/sector
- Employee count range
- Funding amount range
- Founded date range
- Technology stack
- Keywords

---

#### `crustdata_company_search_db`
**Endpoint**: `POST /screener/companydb/search`
**Purpose**: Search pre-indexed company database (faster)
**Parameters**: Same as realtime search

---

#### `crustdata_company_linkedin_posts`
**Endpoint**: `POST /screener/linkedin_posts/keyword_search/`
**Purpose**: Search company LinkedIn posts by keyword
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| keyword | str | Yes | Search keyword |
| date_from | str | No | Start date (ISO 8601) |
| date_to | str | No | End date (ISO 8601) |
| limit | int | No | Max results |

---

### 7.2 People Tools

#### `crustdata_people_enrich`
**Endpoint**: `GET /screener/person/enrich`
**Purpose**: Get detailed person profile
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| linkedin_url | str | No* | Person's LinkedIn URL |
| email | str | No* | Person's email |
| enrich_realtime | bool | No | Force real-time enrichment |
*At least one required

---

#### `crustdata_people_search_realtime`
**Endpoint**: `POST /screener/person/search`
**Purpose**: Search people with filters
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| filters | list[Filter] | Yes | Search filters |
| page | int | No | Page number |
| limit | int | No | Results per page |

**Filter Types**:
- Current company
- Job title/seniority
- Location
- Skills
- Education
- Previous companies

---

#### `crustdata_people_search_db`
**Endpoint**: `POST /screener/persondb/search`
**Purpose**: Search pre-indexed people database
**Parameters**: Same as realtime search

---

#### `crustdata_people_linkedin_posts`
**Endpoint**: `GET /screener/linkedin_posts`
**Purpose**: Get LinkedIn posts for person/company
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| linkedin_url | str | Yes | LinkedIn profile URL |
| date_from | str | No | Start date |
| date_to | str | No | End date |
| limit | int | No | Max results |

---

### 7.3 Watcher Tools

#### `crustdata_watcher_create`
**Endpoint**: `POST /watcher/watches`
**Purpose**: Create a new watch for monitoring
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| name | str | Yes | Watch name |
| type | str | Yes | Watch type (job_postings, linkedin_posts, funding) |
| filters | dict | Yes | Monitoring filters |
| notification_email | str | No | Email for notifications |
| webhook_url | str | No | Webhook for updates |

---

#### `crustdata_watcher_update`
**Endpoint**: `POST /watcher/watches/{watch_id}/update`
**Purpose**: Update existing watch
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| watch_id | str | Yes | Watch identifier |
| status | str | No | New status (active, paused, deleted) |
| filters | dict | No | Updated filters |

---

#### `crustdata_watcher_simulate`
**Endpoint**: `POST /watcher/simulation/watches`
**Purpose**: Test watch configuration
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| type | str | Yes | Watch type |
| filters | dict | Yes | Filters to test |

---

### 7.4 Dataset Tools

#### `crustdata_dataset_job_listings`
**Endpoint**: `POST /data_lab/job_listings/Table/`
**Purpose**: Query job listings dataset
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| company_ids | list[int] | No | Filter by company IDs |
| keywords | list[str] | No | Job title keywords |
| locations | list[str] | No | Location filter |
| date_from | str | No | Posted after date |
| date_to | str | No | Posted before date |
| limit | int | No | Max results |

---

### 7.5 Web Tools

#### `crustdata_web_search`
**Endpoint**: `POST /screener/web-search`
**Purpose**: Search the web with AI mode
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| query | str | Yes | Search query |
| ai_mode | bool | No | Enable AI summarization |
| sources | list[str] | No | Source filters (google, linkedin, scholar) |
| limit | int | No | Max results |

---

#### `crustdata_web_fetch`
**Endpoint**: `POST /screener/web-fetch`
**Purpose**: Fetch content from URLs
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| url | str | Yes | URL to fetch |
| extract_text | bool | No | Return extracted text |

---

### 7.6 Utility Tools

#### `crustdata_credits_check`
**Endpoint**: `GET /user/credits`
**Purpose**: Check remaining API credits
**Parameters**: None
**Returns**: Credit balance and tier info

---

#### `crustdata_autocomplete_company`
**Endpoint**: `POST /screener/companydb/autocomplete`
**Purpose**: Get autocomplete suggestions for company fields
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| field | str | Yes | Field name |
| prefix | str | Yes | Search prefix |
| limit | int | No | Max suggestions |

---

#### `crustdata_autocomplete_person`
**Endpoint**: `POST /screener/persondb/autocomplete`
**Purpose**: Get autocomplete suggestions for person fields
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| field | str | Yes | Field name |
| prefix | str | Yes | Search prefix |
| limit | int | No | Max suggestions |

---

#### `crustdata_autocomplete_linkedin_filter`
**Endpoint**: `GET /screener/linkedin_filter/autocomplete`
**Purpose**: Get LinkedIn filter suggestions
**Parameters**:
| Name | Type | Required | Description |
|------|------|----------|-------------|
| filter_type | str | Yes | Filter type |
| query | str | Yes | Search query |

---

## 8. Security Requirements

### 8.1 Authentication

The server supports two transport modes with different authentication approaches:

#### 8.1.1 STDIO Transport (Single-Tenant)

For local/CLI use, the API token is read from environment variables at startup:

```bash
# Required for STDIO transport
CRUSTDATA_TOKEN=your-token crustdata-mcp
```

- Token validated at server startup
- All tool calls use the same token
- Suitable for single-user/local development

#### 8.1.2 HTTP Transport (Multi-Tenant, Per-Request)

For HTTP transport, the server supports **per-request authentication**. Each client provides their own Crustdata API token:

```bash
# No token required at server startup
crustdata-mcp-http

# Clients provide their own token in each request
curl -X POST http://localhost:8000/mcp \
  -H "Authorization: Bearer <CLIENT_CRUSTDATA_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"method": "tools/list"}'
```

**Implementation:**

```python
class PassthroughTokenVerifier(TokenVerifier):
    """Accepts any valid-format bearer token and passes it through."""

    MIN_TOKEN_LENGTH = 10

    async def verify_token(self, token: str) -> AccessToken | None:
        if not token or len(token) < self.MIN_TOKEN_LENGTH:
            return None
        return AccessToken(
            token=token,  # Stores the original token for API calls
            client_id="crustdata-api-client",
            scopes=["read", "write"],
            expires_at=None,
        )
```

**Token Flow:**
1. Client sends request with `Authorization: Bearer <token>`
2. `PassthroughTokenVerifier` accepts any valid-format token
3. Token is stored in `AccessToken.token` field
4. Tools retrieve token via `get_access_token()` from MCP context
5. `CrustdataClient` is created with the request-specific token

**Multi-Tenant Benefits:**
- Different clients can use different Crustdata accounts
- No server-side token configuration required
- Token validation delegated to Crustdata API

### 8.2 Security Checklist

- [ ] Never log API tokens or sensitive data
- [ ] Validate all input parameters before API calls
- [ ] Sanitize error messages (no stack traces to clients)
- [ ] Implement request timeouts (30s default, 120s for exports)
- [ ] Rate limit tool calls to prevent abuse
- [ ] Use HTTPS for all API communication
- [ ] No hardcoded credentials in source code

---

## 9. Testing Strategy

### 9.1 Test Categories

| Category | Coverage Target | Description |
|----------|-----------------|-------------|
| Unit Tests | 95% | Individual tool functions |
| Integration Tests | 80% | Tool + API client interaction |
| E2E Tests | Key flows | Full MCP client → server → API |
| Security Tests | Critical paths | Auth, input validation |

### 9.2 Test Implementation

See [TEST-CASES.md](./TEST-CASES.md) for complete test specifications.

### 9.3 Mocking Strategy

```python
# conftest.py
import pytest
from unittest.mock import AsyncMock
from crustdata_mcp.client import CrustdataClient

@pytest.fixture
def mock_client():
    """Mock Crustdata API client"""
    client = AsyncMock(spec=CrustdataClient)
    return client

@pytest.fixture
def sample_company_response():
    """Sample company enrichment response"""
    return {
        "company_name": "Acme Corp",
        "linkedin_profile_url": "https://linkedin.com/company/acme",
        "hq_country": "US",
        "employee_count": 500,
        "year_founded": 2010,
        "total_funding_usd": 50000000
    }
```

---

## 10. Success Metrics

### 10.1 Technical Metrics

| Metric | Target | Measurement |
|--------|--------|-------------|
| Test Coverage | >95% | pytest-cov |
| Response Time (p95) | <2s | Tool execution time |
| Error Rate | <1% | Failed tool calls / total |
| Uptime | 99.9% | Server availability |

### 10.2 Usage Metrics

| Metric | Tracking Method |
|--------|-----------------|
| Tool invocations by type | Logging + analytics |
| Most common query patterns | Query analysis |
| Error types distribution | Error logging |
| Credit consumption patterns | API tracking |

---

## 11. Implementation Phases

### Phase 1: Core Foundation (Week 1-2)

**Goal**: Basic server with authentication and company tools

**Deliverables**:
- [ ] Project structure setup
- [ ] FastMCP server initialization
- [ ] Authentication implementation
- [ ] CrustdataClient class
- [ ] Company tools (identify, enrich, search)
- [ ] Unit tests for Phase 1 components

**Exit Criteria**:
- Server starts and accepts connections
- Auth flow works with valid/invalid tokens
- Company identify and enrich tools functional
- 90%+ test coverage for implemented tools

---

### Phase 2: Full Tool Coverage (Week 3-4)

**Goal**: Complete all tool implementations

**Deliverables**:
- [ ] People tools (enrich, search)
- [ ] Watcher tools (create, update, simulate)
- [ ] Dataset tools (job listings)
- [ ] Web tools (search, fetch)
- [ ] Utility tools (credits, autocomplete)
- [ ] Unit tests for all tools

**Exit Criteria**:
- All 20 API endpoints exposed as tools
- Input validation working for all tools
- Error handling comprehensive
- 95%+ test coverage

---

### Phase 3: Resources & Polish (Week 5)

**Goal**: Documentation resources and production hardening

**Deliverables**:
- [ ] MCP resources for documentation
- [ ] Response formatting optimization
- [ ] Integration tests
- [ ] Performance optimization
- [ ] Documentation (README, CLAUDE.md)

**Exit Criteria**:
- Resources serve API documentation
- Response format optimized for LLMs
- All tests passing
- README with usage examples

---

### Phase 4: Production Readiness (Week 6)

**Goal**: Deployment preparation

**Deliverables**:
- [ ] Docker configuration
- [ ] CI/CD pipeline
- [ ] Monitoring/logging setup
- [ ] Security audit
- [ ] Load testing

**Exit Criteria**:
- Docker image builds and runs
- CI pipeline passes all tests
- Security checklist complete
- Performance benchmarks documented

---

## Appendix A: Example Tool Implementations

### Company Enrichment Tool

```python
from fastmcp import FastMCP, ToolError
from pydantic import BaseModel, Field
from typing import Optional, List

class CompanyEnrichInput(BaseModel):
    """Input for company enrichment"""
    company_domain: Optional[str] = Field(None, description="Company website domain (e.g., 'acme.com')")
    company_name: Optional[str] = Field(None, description="Company name")
    company_linkedin_url: Optional[str] = Field(None, description="LinkedIn company page URL")
    fields: Optional[List[str]] = Field(None, description="Specific fields to return")
    enrich_realtime: bool = Field(False, description="Force real-time data enrichment")

@mcp.tool()
async def crustdata_company_enrich(
    input: CompanyEnrichInput,
    context: Context
) -> dict:
    """
    Get comprehensive company profile data.

    Returns detailed information including:
    - Company basics (name, website, founded date)
    - Employee metrics (headcount, growth trends)
    - Funding data (total raised, last round details)
    - Social profiles (LinkedIn, Twitter)
    - Location data (headquarters, offices)
    - Industry classification

    Example: Get details for a company by domain
    """
    # Validate at least one identifier provided
    if not any([input.company_domain, input.company_name, input.company_linkedin_url]):
        raise ToolError(
            "At least one identifier required: company_domain, company_name, or company_linkedin_url",
            is_retriable=False
        )

    # Build query params
    params = {}
    if input.company_domain:
        params["company_domain"] = input.company_domain
    if input.company_name:
        params["company_name"] = input.company_name
    if input.company_linkedin_url:
        params["company_linkedin_url"] = input.company_linkedin_url
    if input.fields:
        params["fields"] = ",".join(input.fields)
    if input.enrich_realtime:
        params["enrich_realtime"] = "true"

    # Get API client from context
    client = get_client(context)

    try:
        result = await client.get("/screener/company", params=params)
        return format_company_response(result)
    except httpx.HTTPStatusError as e:
        if e.response.status_code == 404:
            return {"found": False, "message": "Company not found in database"}
        raise ToolError(f"API error: {e.response.status_code}", is_retriable=e.response.status_code >= 500)
```

---

## Appendix B: Configuration

### Environment Variables

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| CRUSTDATA_TOKEN | STDIO only* | - | Crustdata API token |
| CRUSTDATA_API_URL | No | https://api.crustdata.com | API base URL |
| CRUSTDATA_HTTP_HOST | No | 127.0.0.1 | HTTP server bind host |
| CRUSTDATA_HTTP_PORT | No | 8000 | HTTP server port |
| CRUSTDATA_HTTP_PATH | No | /mcp | HTTP endpoint path |
| LOG_LEVEL | No | INFO | Logging level |
| REQUEST_TIMEOUT | No | 30 | API request timeout (seconds) |

*`CRUSTDATA_TOKEN` is required for STDIO transport but NOT for HTTP transport.
HTTP transport uses per-request authentication - clients provide their token in the Authorization header.

### pyproject.toml

```toml
[project]
name = "crustdata-mcp"
version = "1.0.0"
description = "MCP server for Crustdata B2B data platform"
requires-python = ">=3.11"
dependencies = [
    "fastmcp>=2.0,<3.0",
    "httpx>=0.27",
    "pydantic>=2.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-asyncio>=0.23",
    "pytest-cov>=4.0",
    "ruff>=0.3",
    "pyright>=1.1",
]

[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]

[tool.ruff]
line-length = 100
target-version = "py311"

[tool.pyright]
pythonVersion = "3.11"
typeCheckingMode = "standard"
```

---

## Appendix C: Error Codes

| Code | Name | Description | HTTP Status |
|------|------|-------------|-------------|
| E001 | INVALID_INPUT | Input validation failed | 400 |
| E002 | AUTH_FAILED | Authentication failed | 401 |
| E003 | NOT_FOUND | Resource not found | 404 |
| E004 | RATE_LIMITED | Rate limit exceeded | 429 |
| E005 | INSUFFICIENT_CREDITS | Not enough credits | 402 |
| E006 | API_ERROR | Backend API error | 500 |
| E007 | TIMEOUT | Request timed out | 504 |

---

---

## Appendix D: Detailed Input Specifications

This appendix provides comprehensive input validation rules derived from the official Crustdata API documentation. These rules MUST be enforced in Pydantic models to prevent API errors.

### D.1 Company Identification API

**Endpoint**: `POST /screener/identify/`

| Parameter | API Name | Type | Required | Constraints | Default |
|-----------|----------|------|----------|-------------|---------|
| company_name | `query_company_name` | string | No* | - | - |
| company_website | `query_company_website` | string | No* | Domain format (e.g., "acme.com") | - |
| company_linkedin_url | `query_company_linkedin_url` | string | No* | Full LinkedIn URL | - |
| company_crunchbase_url | `query_company_crunchbase_url` | string | No* | Full Crunchbase URL | - |
| company_id | `query_company_id` | int | No* | Crustdata company ID | - |
| exact_match | `exact_match` | bool | No | - | false |
| count | `count` | int | No | 1-25 | 10 |

*At least one identifier required

**Credit Cost**: 0 credits

---

### D.2 Company Enrichment API

**Endpoint**: `GET /screener/company`

| Parameter | API Name | Type | Required | Constraints | Default |
|-----------|----------|------|----------|-------------|---------|
| company_domain | `company_domain` | string | No* | Up to 25 comma-separated | - |
| company_name | `company_name` | string | No* | Up to 25 comma-separated | - |
| company_linkedin_url | `company_linkedin_url` | string | No* | Up to 25 comma-separated | - |
| company_id | `company_id` | int | No* | Up to 25 comma-separated | - |
| fields | `fields` | string | No | Comma-separated field names | All fields |
| enrich_realtime | `enrich_realtime` | bool | No | - | false |
| exact_match | `exact_match` | bool | No | - | false |

*At least one identifier required

**Credit Cost**: 1 credit (database), 4 credits (real-time)

---

### D.3 Company Search (Real-time via LinkedIn Sales Navigator)

**Endpoint**: `POST /screener/company/search`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| filters | array[Filter] | Yes | See filter types below | - |
| page | int | No | 1-65 (25 results per page) | 1 |

**Company Filter Types**:

| filter_type | type | value format | sub_filter |
|-------------|------|--------------|------------|
| `COMPANY_HEADCOUNT` | `in` / `not in` | Array: `["1-10", "11-50", "51-200", "201-500", "501-1,000", "1,001-5,000", "5,001-10,000", "10,001+"]` | - |
| `REGION` | `in` / `not in` | Array of regions (use Autocomplete API) | - |
| `INDUSTRY` | `in` / `not in` | Array of industries (use Autocomplete API) | - |
| `NUM_OF_FOLLOWERS` | `in` / `not in` | Array: `["1-50", "51-100", "101-1000", "1001-5000", "5001+"]` | - |
| `FORTUNE` | `in` / `not in` | Array: `["Fortune 50", "Fortune 51-100", "Fortune 101-250", "Fortune 251-500"]` | - |
| `ACCOUNT_ACTIVITIES` | `in` / `not in` | Array: `["Senior leadership changes in last 3 months", "Funding events in past 12 months"]` | - |
| `JOB_OPPORTUNITIES` | `in` | Array: `["Hiring on Linkedin"]` | - |
| `COMPANY_HEADCOUNT_GROWTH` | `between` | `{"min": float, "max": float}` | - |
| `ANNUAL_REVENUE` | `between` | `{"min": int, "max": int}` (in millions) | `currency`: `["USD", "EUR", "GBP", "JPY", "CAD", "AUD", "CHF", "INR", "CNY"]` |
| `DEPARTMENT_HEADCOUNT` | `between` | `{"min": int, "max": int}` | `department`: `["Sales", "Engineering", "Operations", "Finance", "HR", "Marketing", "IT", "R&D", "Support", "Legal", "Consulting", "Education", "Arts and Design", "Administrative", "Accounting", "Healthcare Services", "Media and Communication", "Community and Social Services", "Military and Protective Services", "Quality Assurance", "Real Estate", "Product Management", "Purchasing", "Business Development"]` |
| `KEYWORD` | `in` | Array of strings | - |

**Credit Cost**: 1 credit per company returned (minimum 0)

---

### D.4 Company Database Search

**Endpoint**: `POST /screener/company/Table`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| filters | object | Yes | `op` + `conditions` structure | - |
| limit | int | No | 1-1000 | 20 |
| cursor | string | No | Pagination token from previous response | - |
| sorts | array[Sort] | No | `[{"column": "field_name", "order": "asc/desc"}]` | - |

**Filter Condition Structure**:
```json
{
  "op": "and" | "or",
  "conditions": [
    {"filter_type": "field_name", "type": "operator", "value": "..."}
  ]
}
```

**Filter Operators**:
| Operator | Description | Example |
|----------|-------------|---------|
| `=` | Equals | `{"filter_type": "hq_country", "type": "=", "value": "US"}` |
| `!=` | Not equals | `{"filter_type": "hq_country", "type": "!=", "value": "US"}` |
| `>` | Greater than | `{"filter_type": "employee_count", "type": ">", "value": 100}` |
| `<` | Less than | `{"filter_type": "employee_count", "type": "<", "value": 1000}` |
| `>=` | Greater than or equal | - |
| `<=` | Less than or equal | - |
| `in` | In list | `{"filter_type": "hq_country", "type": "in", "value": ["US", "CA"]}` |
| `not_in` | Not in list | - |
| `(.)` | Fuzzy match | `{"filter_type": "company_name", "type": "(.)", "value": "Anthropic"}` |
| `[.]` | Regex match | `{"filter_type": "company_name", "type": "[.]", "value": "^Ant.*"}` |

---

### D.5 People Enrichment API

**Endpoint**: `GET /screener/person/enrich`

| Parameter | API Name | Type | Required | Constraints | Default |
|-----------|----------|------|----------|-------------|---------|
| linkedin_profile_url | `linkedin_profile_url` | string | No* | Up to 25 comma-separated | - |
| business_email | `business_email` | string | No* | Up to 25 comma-separated | - |
| fields | `fields` | string | No | Comma-separated field names | All fields |
| enrich_realtime | `enrich_realtime` | bool | No | - | false |
| preview | `preview` | bool | No | Access controlled | false |

*At least one identifier required

**Credit Cost**:
- Database enrichment: 3 credits per profile
- Real-time enrichment: 5 credits per profile
- Business email enrichment: +2 additional credits
- Preview mode: 0 credits

---

### D.6 People Search (Real-time via LinkedIn Sales Navigator)

**Endpoint**: `POST /screener/person/search`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| filters | array[Filter] | Yes | See filter types below | - |
| page | int | No | 1-100 (25 results per page) | 1 |
| limit | int | No | 1-10000 (for async only) | - |
| background_job | bool | No | Use for >500 results | false |
| preview | bool | No | Access controlled | false |

**People Filter Types**:

| filter_type | type | value format | Notes |
|-------------|------|--------------|-------|
| `CURRENT_COMPANY` | `in` / `not in` | Array of URLs, domains, or names | LinkedIn URL most accurate |
| `PAST_COMPANY` | `in` / `not in` | Array of URLs, domains, or names | - |
| `CURRENT_TITLE` | `in` / `not in` | Array of titles (use Autocomplete API) | LinkedIn predefined titles |
| `PAST_TITLE` | `in` / `not in` | Array of titles | - |
| `COMPANY_HEADQUARTERS` | `in` / `not in` | Array of locations (use Autocomplete API) | - |
| `COMPANY_HEADCOUNT` | `in` / `not in` | Array: `["Self-employed", "1-10", "11-50", "51-200", "201-500", "501-1,000", "1,001-5,000", "5,001-10,000", "10,001+"]` | - |
| `REGION` | `in` / `not in` | Array of regions (use Autocomplete API) | - |
| `INDUSTRY` | `in` / `not in` | Array of industries (use Autocomplete API) | - |
| `PROFILE_LANGUAGE` | `in` / `not in` | Array: `["Arabic", "English", "Spanish", "Portuguese", "Chinese", "French", "Italian", "Russian", "German", "Dutch", "Turkish", "Tagalog", "Polish", "Korean", "Japanese", "Malay", "Norwegian", "Danish", "Romanian", "Swedish", "Bahasa Indonesia", "Czech"]` | - |
| `SENIORITY_LEVEL` | `in` / `not in` | Array: `["Owner / Partner", "CXO", "Vice President", "Director", "Experienced Manager", "Entry Level Manager", "Strategic", "Senior", "Entry Level", "In Training"]` | - |
| `YEARS_AT_CURRENT_COMPANY` | `in` / `not in` | Array: `["Less than 1 year", "1 to 2 years", "3 to 5 years", "6 to 10 years", "More than 10 years"]` | - |
| `YEARS_IN_CURRENT_POSITION` | `in` / `not in` | Array (same as above) | - |
| `YEARS_OF_EXPERIENCE` | `in` / `not in` | Array (same as above) | - |
| `FUNCTION` | `in` / `not in` | Array: `["Accounting", "Administrative", "Arts and Design", "Business Development", "Community and Social Services", "Consulting", "Education", "Engineering", "Entrepreneurship", "Finance", "Healthcare Services", "Human Resources", "Information Technology", "Legal", "Marketing", "Media and Communication", "Military and Protective Services", "Operations", "Product Management", "Program and Project Management", "Purchasing", "Quality Assurance", "Real Estate", "Research", "Sales", "Customer Success and Support"]` | - |
| `FIRST_NAME` | `in` | Array (max length 1) | Single value only |
| `LAST_NAME` | `in` | Array (max length 1) | Single value only |
| `SCHOOL` | `in` / `not in` | Array of school names | - |
| `RECENTLY_CHANGED_JOBS` | - | No value needed | Boolean filter |
| `POSTED_ON_LINKEDIN` | - | No value needed | Boolean filter |
| `IN_THE_NEWS` | - | No value needed | Boolean filter |
| `KEYWORD` | `in` | Array of strings | - |

**Credit Cost**: 1 credit per profile (minimum 5), Preview: 5 credits

**Rate Limits**:
- Synchronous: 15 requests/min
- Background jobs: 3 concurrent max per account

---

### D.7 LinkedIn Posts APIs

#### D.7.1 LinkedIn Posts by Keyword Search

**Endpoint**: `POST /screener/linkedin_posts/keyword_search/`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| keyword | string | Yes | - | - |
| limit | int | No | 1-100 | 10 |

#### D.7.2 LinkedIn Posts by Profile

**Endpoint**: `GET /screener/linkedin_posts`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| person_linkedin_url | string | No* | LinkedIn profile URL | - |
| company_linkedin_url | string | No* | LinkedIn company URL | - |
| company_domain | string | No* | Company domain | - |
| company_name | string | No* | Company name | - |
| company_id | int | No* | Crustdata company ID | - |
| page | int | No | 1-20 (use `page` OR `limit`, not both) | - |
| limit | int | No | 1-100 (use `page` OR `limit`, not both) | - |
| fields | string | No | Add `"reactors"` or `"comments"` for engagement data | - |
| post_types | string | No | `"original"`, `"repost"`, or `"repost,original"` | `"repost,original"` |
| max_reactors | int | No | 0-5000 (when `fields` includes `"reactors"`) | 100 |
| max_comments | int | No | 0-5000 (when `fields` includes `"comments"`) | 100 |

*Exactly one identifier required (person OR company identifier)

---

### D.8 Watcher APIs

**Endpoint**: `POST /watcher/watches`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| event_type_slug | string | Yes | See event types below | - |
| event_filters | array[Filter] | Yes | Event-specific filters | - |
| lead_filters | array[Filter] | No | People filter criteria | - |
| account_filters | array[Filter] | No | Company filter criteria | - |
| notification_endpoint | string | Yes | Valid webhook URL (HTTPS) | - |
| frequency | int | Yes | Days between notifications (minimum 1) | - |
| expiration_date | string | No | ISO 8601 date (`YYYY-MM-DD`), must be future | - |
| approximate_notification_time | int | No | UTC hour (0-23) | - |
| max_notifications_per_execution | int | No | Multiple of 50 | - |

**Event Type Slugs**:
- `job-posting-with-keyword-and-location`
- `company-watch-linkedin-posts`
- `linkedin-person-post-updates`

**Account Filter Types** (for watchers):
- `INDUSTRY`
- `COMPANY_HEADCOUNT`
- `COMPANY_HQ_COUNTRY`
- `LAST_FUNDING_ROUND_TYPE`

**Credit Cost**:
- Creating watch: 0 credits
- Notifications: 5 credits per notification

---

### D.9 Dataset Job Listings API

**Endpoint**: `POST /data_lab/job_listings/Table/`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| filters | object | Yes | `op` + `conditions` structure | - |
| offset | int | Yes | Starting point | 0 |
| limit | int | Yes | 1-100 (sync), unlimited (async) | - |
| sorts | array[Sort] | No | Sort criteria | - |
| aggregations | array | No | Aggregate columns | - |
| functions | array | No | Functions to apply | - |
| groups | array | No | Group by columns | - |
| background_task | bool | No | Async processing for large sets | false |
| sync_from_source | bool | No | Real-time fetch (requires company_id, max 100) | false |

**Job Listings Filter Fields**:
- `company_website_domain` (string)
- `company_id` (int)
- `date_updated` (date: `YYYY-MM-DD`)
- `date_added` (date: `YYYY-MM-DD`)
- `title` (string)

**Credit Cost**:
- Standard: 1 credit per company returned
- Real-time sync: 5 credits per request
- Background task: 5 credits per request

---

### D.10 Web APIs

#### D.10.1 Web Search

**Endpoint**: `POST /screener/web-search`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| query | string | Yes | Max 1000 characters | - |
| geolocation | string | No | ISO 3166-1 alpha-2: `["US", "CA", "GB", "DE", "FR", "JP", "AU", "IN", "BR", "MX", ...]` | - |
| sources | array[string] | No | `["news", "web", "scholar-articles", "scholar-articles-enriched", "scholar-author", "ai", "social"]` | - |
| site | string | No | Domain restriction (e.g., `"github.com"`) | - |
| startDate | int | No | Unix timestamp (must be < endDate) | - |
| endDate | int | No | Unix timestamp (must be > startDate) | - |
| fetch_content | bool | No | Query param: fetch webpage HTML | false |

**Credit Cost**: 1 credit per search query (fixed)

#### D.10.2 Web Fetch

**Endpoint**: `POST /screener/web-fetch`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| urls | array[string] | Yes | Max 10 URLs, must include `http://` or `https://` | - |

**Credit Cost**: 1 credit per URL fetched

---

### D.11 Utility APIs

#### D.11.1 Credits Check

**Endpoint**: `GET /user/credits`

No parameters required.

**Returns**: `{"credits": <integer>}`

**Credit Cost**: 0 credits

#### D.11.2 LinkedIn Filter Autocomplete

**Endpoint**: `GET /screener/linkedin_filter/autocomplete`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| filter_type | string | Yes | `["region", "industry", "title", "school"]` | - |
| query | string | Yes | Search text | - |
| count | int | No | 1-50 | 10 |
| start | int | No | Pagination offset | 0 |

**Credit Cost**: 0 credits

#### D.11.3 Company Autocomplete

**Endpoint**: `POST /screener/company/autocomplete`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| field | string | Yes | Field name to autocomplete | - |
| query | string | Yes | Search prefix | - |
| limit | int | No | 1-100 | 20 |

#### D.11.4 Person Autocomplete

**Endpoint**: `POST /screener/person/autocomplete`

| Parameter | Type | Required | Constraints | Default |
|-----------|------|----------|-------------|---------|
| field | string | Yes | Field name to autocomplete | - |
| query | string | Yes | Search prefix | - |
| limit | int | No | 1-100 | 20 |

---

### D.12 Validation Rules Summary

#### Cross-Field Validation Rules

1. **At least one identifier required**:
   - Company Identify: `company_name` OR `company_website` OR `company_linkedin_url` OR `company_crunchbase_url` OR `company_id`
   - Company Enrich: `company_domain` OR `company_name` OR `company_linkedin_url` OR `company_id`
   - People Enrich: `linkedin_profile_url` OR `business_email`
   - LinkedIn Posts by Profile: Exactly ONE of person or company identifiers

2. **Mutual exclusivity**:
   - LinkedIn Posts: Use `page` OR `limit`, not both
   - People Search: `background_job=true` requires `limit` parameter

3. **Conditional requirements**:
   - `ANNUAL_REVENUE` filter requires `sub_filter.currency`
   - `DEPARTMENT_HEADCOUNT` filter requires `sub_filter.department`
   - `sync_from_source=true` in Job Listings requires `company_id` filter

#### Type Constraints

| Type | Validation |
|------|------------|
| LinkedIn URL | Must start with `https://linkedin.com/` or `https://www.linkedin.com/` |
| Domain | Lowercase, no protocol (e.g., `anthropic.com`) |
| ISO Date | `YYYY-MM-DD` format |
| Unix Timestamp | Integer seconds since epoch |
| Email | Valid email format |
| Webhook URL | Valid HTTPS URL |

#### Pagination Limits

| API | Max Page | Results Per Page | Max Total |
|-----|----------|------------------|-----------|
| Company Search (realtime) | 65 | 25 | 1,625 |
| People Search (sync) | 100 | 25 | 2,500 |
| People Search (async) | - | - | 10,000 |
| LinkedIn Posts | 20 | varies | - |
| Company DB Search | - | 1000 | - |

#### Rate Limits

| API | Limit |
|-----|-------|
| People Search (sync) | 15 requests/min |
| People Search (async) | 3 concurrent jobs |
| Web Search | 15 requests/min |

---

## Appendix E: Database Search Field Dictionaries

This appendix documents the valid field names for the **database search** APIs (`/screener/company/Table` and `/screener/person/Table`). These are different from the LinkedIn Sales Navigator filter types in Appendix D.

### E.1 Company Database Search Fields

**Endpoint**: `POST /screener/company/Table`

#### Core Company Fields
| Field | Type | Description |
|-------|------|-------------|
| `company_id` | number | Crustdata company ID |
| `company_name` | string | Company name |
| `year_founded` | number | Year founded |
| `company_type` | string | "Privately Held", "Public", etc. |
| `acquisition_status` | string | "acquired", etc. |

#### Location Fields
| Field | Type | Description |
|-------|------|-------------|
| `hq_country` | string | ISO 3-alpha format (USA, GBR, CAN, IND) |
| `hq_location` | string | Full HQ location string |
| `largest_headcount_country` | string | ISO 3-alpha format |

#### Employee Metrics (nested with dot notation)
| Field | Type | Description |
|-------|------|-------------|
| `employee_metrics.latest_count` | number | Current employee count |
| `employee_metrics.growth_1m` | number | 1-month growth (absolute) |
| `employee_metrics.growth_3m` | number | 3-month growth (absolute) |
| `employee_metrics.growth_6m` | number | 6-month growth (absolute) |
| `employee_metrics.growth_12m` | number | 12-month growth (absolute) |
| `employee_metrics.growth_1m_percent` | number | 1-month growth (%) |
| `employee_metrics.growth_6m_percent` | number | 6-month growth (%) |
| `employee_metrics.growth_12m_percent` | number | 12-month growth (%) |

#### Financial Fields
| Field | Type | Description |
|-------|------|-------------|
| `estimated_revenue_lower_bound_usd` | number | Revenue lower bound |
| `estimated_revenue_higher_bound_usd` | number | Revenue upper bound |
| `crunchbase_total_investment_usd` | number | Total funding |
| `last_funding_round_type` | string | "Series A", "Seed", etc. |
| `last_funding_date` | date | Date of last funding |

#### Industry & Categories
| Field | Type | Description |
|-------|------|-------------|
| `linkedin_industries` | array | LinkedIn industry tags |
| `linkedin_categories` | array | LinkedIn category tags |
| `crunchbase_categories` | array | Crunchbase categories |

#### Web & Social
| Field | Type | Description |
|-------|------|-------------|
| `company_website_domain` | string | Primary domain |
| `linkedin_profile_url` | string | LinkedIn company URL |
| `twitter_handle` | string | Twitter handle |

---

### E.2 People Database Search Fields

**Endpoint**: `POST /screener/person/Table`

#### Core Person Fields
| Field | Type | Description |
|-------|------|-------------|
| `person_id` | number | Crustdata person ID |
| `name` | string | Full name |
| `first_name` | string | First name |
| `last_name` | string | Last name |
| `headline` | string | LinkedIn headline |
| `summary` | string | LinkedIn summary |
| `region` | string | Location (e.g., "Bengaluru, Karnataka, India") |
| `num_of_connections` | number | LinkedIn connections count |
| `years_of_experience_raw` | number | Years of experience (numeric) |
| `years_of_experience` | string | Years of experience (range string) |
| `recently_changed_jobs` | boolean | Recently changed jobs flag |
| `profile_language` | string | Profile language |

#### Skills & Languages
| Field | Type | Description |
|-------|------|-------------|
| `skills` | array | List of skills |
| `languages` | array | List of languages |

#### Current Employers (nested with dot notation)
| Field | Type | Description |
|-------|------|-------------|
| `current_employers.name` | string | Company name |
| `current_employers.title` | string | Job title |
| `current_employers.company_id` | number | Crustdata company ID |
| `current_employers.company_website_domain` | string | Company domain |
| `current_employers.location` | string | Job location |
| `current_employers.start_date` | date | Start date |
| `current_employers.seniority_level` | string | Seniority level |
| `current_employers.function_category` | string | Job function |
| `current_employers.company_headcount_range` | string | Company size range |
| `current_employers.company_industries` | array | Company industries |
| `current_employers.company_headquarters_country` | string | HQ country (ISO 3-alpha) |

#### Past Employers (nested with dot notation)
| Field | Type | Description |
|-------|------|-------------|
| `past_employers.name` | string | Company name |
| `past_employers.title` | string | Job title |
| `past_employers.company_id` | number | Crustdata company ID |
| `past_employers.start_date` | date | Start date |
| `past_employers.end_date` | date | End date |

#### Education (nested with dot notation)
| Field | Type | Description |
|-------|------|-------------|
| `education_background.institute_name` | string | School/university name |
| `education_background.degree_name` | string | Degree name |
| `education_background.field_of_study` | string | Field of study |

---

### E.3 Filter Operators for Database Search

| Operator | Description | Applicable Types |
|----------|-------------|------------------|
| `=` | Equals | string, number |
| `!=` | Not equals | string, number |
| `>` | Greater than | number, date |
| `<` | Less than | number, date |
| `>=` | Greater than or equal | number, date |
| `<=` | Less than or equal | number, date |
| `in` | In list | string, number (array) |
| `not_in` | Not in list | string, number (array) |
| `(.)` | Fuzzy match | string |
| `[.]` | Regex/substring match | string |

---

### E.4 Example Database Search Filters

**Simple filter (fuzzy match on headline):**
```json
{"filter_type": "headline", "type": "(.)", "value": "golang developer bangalore"}
```

**Filter by region:**
```json
{"filter_type": "region", "type": "(.)", "value": "Bengaluru, Karnataka, India"}
```

**Filter by current employer title:**
```json
{"filter_type": "current_employers.title", "type": "(.)", "value": "Software Engineer"}
```

**Complex AND filter:**
```json
{
  "op": "and",
  "conditions": [
    {"filter_type": "headline", "type": "(.)", "value": "golang"},
    {"filter_type": "region", "type": "(.)", "value": "Bengaluru"}
  ]
}
```

**Numeric comparison:**
```json
{"filter_type": "employee_metrics.latest_count", "type": ">", "value": 100}
```

---

## Appendix F: Next Steps & Improvements

### F.1 Current Gaps Identified

| Gap | Impact | Priority | Status |
|-----|--------|----------|--------|
| Database search field names not in MCP tool descriptions | LLMs guess wrong field names, causing 400 errors | High | ✅ Resolved |
| PersonDB uses `column` but MCP describes `filter_type` | API returns 400 errors when LLMs use `filter_type` for PersonDB | Critical | ✅ Resolved |
| **Company Enrichment `fields` parameter has no valid field list** | **LLMs guess field names like `employee_count`, `short_description`** | **Critical** | ✅ **FIXED** |
| No field discovery tool | Users can't find valid field names | Medium | Open |
| Complex nested AND/OR filters may not work for all DB fields | Unexpected API errors | Medium | Open |
| No example filters in tool descriptions | LLMs don't know correct filter format | High | ✅ Resolved |
| Operator case sensitivity: `and`/`or` vs `AND`/`OR` | API may reject uppercase operators | High | ✅ Resolved |

### F.2 Recommended Improvements

#### Phase 1: Improve Tool Descriptions (High Priority)
1. **Add valid field names to tool descriptions** - Include top 10-15 most common fields directly in the MCP tool docstrings for `crustdata_company_search_db` and `crustdata_people_search_db`
2. **Add example filters** - Include working filter examples in descriptions
3. **Document simple vs complex filter syntax** - Clarify when to use simple filters vs nested AND/OR

#### Phase 2: Add Field Discovery (Medium Priority)
1. **Create `crustdata_list_company_fields` tool** - Returns all valid field names for company DB search
2. **Create `crustdata_list_people_fields` tool** - Returns all valid field names for people DB search
3. **Enhance autocomplete tools** - Make them easier to use for field discovery

#### Phase 3: Improve Error Handling (Medium Priority)
1. **Validate filter field names before API call** - Check against known field list
2. **Provide helpful error messages** - Suggest valid field names when invalid ones used
3. **Add retry logic with simpler filters** - Fallback to fuzzy headline search if complex filter fails

### F.3 Implementation Priority

| Task | Effort | Impact | Priority |
|------|--------|--------|----------|
| Add field names to tool descriptions | Low | High | P0 |
| Add example filters to descriptions | Low | High | P0 |
| Create field discovery tools | Medium | Medium | P1 |
| Add filter validation in Pydantic models | Medium | Medium | P1 |
| Add helpful error messages | Low | Medium | P2 |

---

### F.4 PersonDB vs CompanyDB Filter Schema Differences

**Critical Issue Discovered**: The PersonDB and CompanyDB search APIs have different filter schemas.

#### PersonDB Search (`/screener/persondb/search`)

Uses `column` field name (per OpenAPI spec):
```json
{
  "filters": {
    "op": "and",
    "conditions": [
      {"column": "name", "type": "(.)", "value": "John Smith"},
      {"column": "current_employers.company_name", "type": "(.)", "value": "Google"}
    ]
  }
}
```

#### CompanyDB Search (`/screener/companydb/search`)

Uses `filter_type` field name:
```json
{
  "filters": {
    "op": "and",
    "conditions": [
      {"filter_type": "company_name", "type": "(.)", "value": "Anthropic"},
      {"filter_type": "hq_country", "type": "=", "value": "USA"}
    ]
  }
}
```

#### Key Differences

| Aspect | PersonDB | CompanyDB |
|--------|----------|-----------|
| Field name key | `column` | `filter_type` |
| Operator case | lowercase (`and`, `or`) | lowercase (`and`, `or`) |
| Endpoint | `/screener/persondb/search` | `/screener/companydb/search` |

#### MCP Implementation Fix Required

The MCP tool description for `crustdata_people_search_db` must:
1. Use `column` instead of `filter_type` in examples
2. Explicitly document this difference from company search
3. Validate filter structure before sending to API

---

### F.5 Company Enrichment Field Validation (IMPLEMENTED)

**Problem**: LLMs would guess invalid field names when calling `crustdata_company_enrich`, causing API errors.

**Example Error**:
```
Error: Invalid fields provided: long_description, employee_count, linkedin_company_url, short_description
```

**Common Hallucinated Fields → Correct Fields**:
| Invalid (Guessed) | Valid (Actual) |
|-------------------|----------------|
| `employee_count` | `headcount` or `employee_count_range` |
| `short_description` | `linkedin_company_description` |
| `long_description` | `linkedin_company_description` |
| `linkedin_company_url` | `linkedin_profile_url` |
| `description` | `linkedin_company_description` |
| `employees` | `headcount` |
| `funding` | `funding_and_investment` |

**Solution Implemented**:

1. **Created `CompanyEnrichField` enum** in `models/common.py` with 36 valid top-level field names
2. **Added `validate_company_enrich_fields()` helper** to check field names against enum
3. **Updated `CompanyEnrichInput.fields` field validator** to reject invalid fields with helpful error messages
4. **Improved field description** in tool definition to list valid field names by category

**Code Location**: `src/crustdata_mcp/models/common.py` (lines 444-555)

**Valid Fields** (36 total):
- **Basic**: `company_id`, `company_name`, `linkedin_profile_url`, `linkedin_id`, `company_website_domain`, `company_website`, `domains`
- **Location**: `hq_country`, `headquarters`, `largest_headcount_country`, `hq_street_address`, `all_office_addresses`
- **Firmographics**: `year_founded`, `company_type`, `linkedin_company_description`, `employee_count_range`, `acquisition_status`, `ipo_date`
- **Financial**: `estimated_revenue_lower_bound_usd`, `estimated_revenue_higher_bound_usd`, `funding_and_investment`
- **Metrics**: `headcount`, `linkedin_followers`, `web_traffic`, `seo`, `job_openings`
- **Reviews**: `glassdoor`, `g2`, `gartner`, `producthunt`
- **People**: `founders`, `cxos`, `decision_makers`
- **Other**: `taxonomy`, `competitors`, `news_articles`, `markets`, `stock_symbols`

---

### F.6 PR Review Issues (✅ IMPLEMENTED)

The following issues were identified during PR review of the per-request HTTP authentication implementation. All issues have been resolved.

#### F.6.1 HTTP Mode Silent Fallback to Env Token (Medium Priority) ✅ RESOLVED

**Location**: `client_factory.py` lines 50-55

**Problem**: If an HTTP request reaches tool code without an `AccessToken` (auth middleware bug/misconfiguration), the code silently falls back to `CRUSTDATA_TOKEN` env var. This contradicts the documented behavior that HTTP mode does NOT require the env token.

**Current Code**:
```python
access_token = get_access_token()
if access_token and access_token.token:
    return self._get_or_create_client(access_token.token)

# Fall back to STDIO mode (env var token)  <-- PROBLEM: should fail explicitly in HTTP mode
return self._get_stdio_client()
```

**Recommended Fix**:
1. Add `_transport_mode: Optional[Literal["stdio", "http"]]` attribute to `ClientFactory`
2. Add `set_transport_mode(mode)` method
3. Call `factory.set_transport_mode("stdio")` in `app_lifespan_stdio()`
4. Call `factory.set_transport_mode("http")` in `app_lifespan_http()`
5. In `get_client_for_request()`: if HTTP mode and no token, raise `ValueError` explicitly

**Impact**: Medium - affects security guarantees of multi-tenant HTTP mode

---

#### F.6.2 WeakValueDictionary Misleading Docstring (Low Priority) ✅ RESOLVED

**Location**: `client_factory.py` lines 25-27

**Current**:
```python
"""...while still providing connection pooling benefits."""
```

**Problem**: The docstring claims "connection pooling benefits" but clients are eligible for garbage collection immediately after each request ends (no strong references held). The actual benefit is memory management, not connection pooling.

**Recommended Fix**: Update docstring to:
```python
"""Clients are cached using weak references for automatic cleanup when no
longer referenced. Note: Clients become eligible for garbage collection
immediately after each request ends; httpx handles underlying connection
cleanup via __del__."""
```

**Impact**: Low - documentation accuracy only

---

#### F.6.3 Outdated CLAUDE.md Documentation (Low Priority) ✅ RESOLVED

**Location**: `src/crustdata_mcp/CLAUDE.md` lines 74, 139, 151

**Problem**: Still references the old pattern `ctx.request_context.lifespan_context.client` which was refactored to `get_client_for_request()`.

**Recommended Fix**: Update all references to use the new pattern:
```python
from ..client_factory import get_client_for_request

@mcp.tool(name="crustdata_example_tool", description="...")
async def crustdata_example_tool(params: ExampleInput, ctx: Context) -> dict:
    client = get_client_for_request()  # NEW pattern
    # ...
```

**Impact**: Low - affects developer onboarding/documentation only

---

#### F.6.4 Security and Resource Management Documentation (Documentation) ✅ RESOLVED

The following design decisions should be documented clearly:

1. **PassthroughTokenVerifier as "Proxy Mode"**
   - Bearer tokens are passed directly to Crustdata API
   - No token validation performed by MCP server (validates format only, 10+ chars)
   - Actual validation delegated to Crustdata API
   - Intentional design for multi-tenant support

2. **Thread Safety Assumptions**
   - Global `_factory` singleton is safe for single-process async (no threading)
   - All tool calls run in the same event loop
   - No concurrent access protection needed

3. **Client Cleanup Behavior**
   - Clients in `WeakValueDictionary` rely on httpx cleanup via `__del__`
   - Explicit `close_all()` called on server shutdown
   - GC'd clients (evicted from weak dict) won't be explicitly closed

---

## Document History

| Version | Date | Author | Changes |
|---------|------|--------|---------|
| 1.0 | 2026-01-06 | Claude | Initial PRD |
| 1.1 | 2026-01-07 | Claude | Added Appendix D: Detailed Input Specifications |
| 1.2 | 2026-01-07 | Claude | Added Appendix E: Database Search Field Dictionaries |
| 1.3 | 2026-01-07 | Claude | Added Appendix F: Next Steps & Improvements |
| 1.4 | 2026-01-07 | Claude | Documented PersonDB `column` vs `filter_type` issue (F.1, F.4) |
| 1.5 | 2026-01-07 | Claude | Added CompanyEnrichField enum validation (F.5), updated gap status |
| 1.6 | 2026-01-12 | Claude | Added F.6: PR Review Issues (pending implementation) |
| 1.7 | 2026-01-12 | Claude | Implemented F.6: Transport mode tracking, docstring fix, CLAUDE.md update, design docs |
| 1.8 | 2026-01-12 | Claude | BREAKING: Removed CRUSTDATA_API_TOKEN legacy env var support |
