# Full-Text Search HTTP API Server

A FastAPI-based HTTP server that provides REST API endpoints for full-text search functionality. This server runs search operations in a thread pool to avoid blocking and provides async support for high-performance concurrent requests.

## Features

- **Async/Non-blocking**: Search operations run in a thread pool executor to prevent blocking the FastAPI server
- **REST API**: Standard HTTP endpoints with JSON request/response
- **Auto-generated docs**: Interactive API documentation at `/docs`
- **Multiple indices**: Support for multiple search indices via YAML configuration
- **Batch operations**: Execute multiple search/fetch operations in parallel
- **CORS enabled**: Ready for cross-origin requests

## Installation

The server dependencies are included in the main package:

```bash
pip install -e .
```

## Quick Start

1. Create a YAML configuration file (e.g., `config.yaml`):

```yaml
server:
  name: "my_search_api"
  description: "My full-text search API"
  host: "0.0.0.0"
  port: 8000

indices:
  - name: "documents"
    description: "Document collection"
    data_file: "data/documents.jsonl"
    id_column: "id"
    text_column: "content"
    searchable_columns: ["title", "content"]
```

2. Run the server:

```bash
# Using the CLI command
fts-api --config config.yaml

# Or directly with Python
python -m src.full_text_search.server.api --config config.yaml

# With custom port and worker threads
python -m src.full_text_search.server.api --config config.yaml --port 8080 --workers 8

# With auto-reload for development
python -m src.full_text_search.server.api --config config.yaml --reload
```

3. Access the API documentation at `http://localhost:8000/docs`

## API Endpoints

### Health Check
```bash
GET /health
```

Returns server status and list of available indices.

### List Indices
```bash
GET /indices
```

Returns detailed information about all configured indices.

### Search
```bash
POST /search
Content-Type: application/json

{
  "index": "documents",
  "queries": ["search term", "another term"],
  "limit": 5,
  "metadata_filters": {
    "agency_name": ["CalPERS", "CalABLE"],
    "document_type": ["strategic_plan"]
  }
}
```

Response:
```json
{
  "index": "documents",
  "queries": ["search term"],
  "results": [
    {
      "id": "doc123",
      "score": 15.5,
      "preview": "First 200 characters of content...",
      "content": {
        "title": "Document Title",
        "content": "Full document content...",
        "id": "doc123"
      }
    }
  ],
  "total": 1
}
```

### Fetch Documents
```bash
POST /fetch
Content-Type: application/json

{
  "index": "documents",
  "document_ids": ["doc123", "doc456"]
}
```

Response:
```json
{
  "index": "documents",
  "documents": [
    {
      "id": "doc123",
      "title": "Document Title",
      "content": "Full content..."
    }
  ],
  "found_ids": ["doc123"],
  "missing_ids": ["doc456"]
}
```

### Batch Operations
```bash
POST /batch
Content-Type: application/json

{
  "commands": [
    {
      "name": "search",
      "index": "documents",
      "queries": ["search term"],
      "limit": 5
    },
    {
      "name": "fetch",
      "index": "documents",
      "document_ids": ["doc123"]
    }
  ]
}
```

All commands execute in parallel and results are returned in the same order.

### Get Citation
```bash
GET /citation/{index}/{doc_id}
```

Returns citation information (title, url) for a specific document.

## Configuration

### Server Configuration

```yaml
server:
  name: "server_name"              # Required: Server name
  description: "Server description" # Required: Server description
  host: "0.0.0.0"                  # Optional: Host to bind (default: from CLI)
  port: 8000                       # Optional: Port to bind (default: from CLI)
```

### Index Configuration

```yaml
indices:
  - name: "index_name"                        # Required: Unique index name
    description: "Index description"           # Required: Index description
    data_file: "path/to/data.jsonl"           # Required: JSONL data file
    id_column: "id"                           # Required: ID column name
    text_column: "content"                    # Required: Main text column
    searchable_columns: ["title", "content"]  # Required: Columns to search
    metadata_columns: ["agency_name"]         # Optional: Exact-match filterable fields
    index_path: "path/to/index"               # Optional: Index storage path
```

## CLI Options

```
--config PATH        Path to YAML configuration file (required)
--host HOST         Host to bind to (default: 0.0.0.0)
--port PORT         Port to bind to (default: 8000)
--workers N         Number of worker threads for search operations (default: 4)
--reload            Enable auto-reload for development
```

## Architecture

### Async Worker Pool

The server uses Python's `ThreadPoolExecutor` to run blocking Tantivy search operations in a thread pool, preventing them from blocking the FastAPI event loop. This allows the server to handle many concurrent requests efficiently.

Key components:
- `asyncio.run_in_executor()`: Runs blocking operations in thread pool
- `asyncio.gather()`: Executes batch operations concurrently
- Configurable worker pool size via `--workers` flag

### Performance

- **Non-blocking**: FastAPI remains responsive during slow searches
- **Concurrent requests**: Multiple requests processed simultaneously
- **Batch operations**: Execute multiple searches/fetches in parallel
- **Thread pool**: Configurable number of worker threads for CPU-bound operations

## Example Usage

### Python Client

```python
import requests

# Search
response = requests.post(
    "http://localhost:8000/search",
    json={
        "index": "documents",
        "queries": ["python programming"],
        "limit": 10
    }
)
results = response.json()

# Fetch
response = requests.post(
    "http://localhost:8000/fetch",
    json={
        "index": "documents",
        "document_ids": ["doc1", "doc2"]
    }
)
documents = response.json()

# Batch
response = requests.post(
    "http://localhost:8000/batch",
    json={
        "commands": [
            {"name": "search", "index": "documents", "queries": ["python"], "limit": 5},
            {"name": "fetch", "index": "documents", "document_ids": ["doc1"]}
        ]
    }
)
batch_results = response.json()
```

### cURL Examples

```bash
# Search
curl -X POST http://localhost:8000/search \
  -H 'Content-Type: application/json' \
  -d '{"index":"documents","queries":["python"],"limit":5}'

# Fetch
curl -X POST http://localhost:8000/fetch \
  -H 'Content-Type: application/json' \
  -d '{"index":"documents","document_ids":["doc1"]}'

# Health check
curl http://localhost:8000/health

# List indices
curl http://localhost:8000/indices
```

## Comparison with MCP Server

| Feature | HTTP API Server | MCP Server |
|---------|----------------|------------|
| Protocol | REST/HTTP | Model Context Protocol |
| Format | JSON | MCP messages |
| Async | Yes (thread pool) | No |
| Use case | General web APIs | AI/LLM integration |
| Docs | Auto-generated OpenAPI | MCP tools |
| Client | Any HTTP client | MCP client library |

Use the HTTP API server when you need a standard REST API that can be consumed by web apps, scripts, or any HTTP client. Use the MCP server when integrating with AI systems that support the Model Context Protocol.

## Development

To run in development mode with auto-reload:

```bash
python -m src.full_text_search.server.api --config config.yaml --reload
```

The server will automatically restart when you modify the code.

## Testing

Run the test suite:

```bash
# Start test server
python -m src.full_text_search.server.api --config test_api_config.yaml --port 8765 &

# Run tests
curl http://localhost:8765/health
curl -X POST http://localhost:8765/search -H 'Content-Type: application/json' \
  -d '{"index":"ccr","queries":["tourism"],"limit":2}'

# Stop server
kill %1
```
