# LLM Integration Guide

neuraparse supports multiple LLM providers for generating QA pairs, summaries, and other LLM-powered datasets.

## Supported Providers

- **OpenAI** (GPT-4, GPT-3.5-turbo, etc.)
- **Anthropic** (Claude 3.5 Sonnet, Claude 3 Opus, etc.)
- **Ollama** (Local models: Llama 3.2, Mistral, Phi-3, etc.)
- **Dummy** (For testing, returns deterministic responses)

## Installation

```bash
# OpenAI
pip install neuraparse[llm-openai]

# Anthropic
pip install neuraparse[llm-anthropic]

# Ollama
pip install neuraparse[llm-ollama]

# All providers
pip install neuraparse[llm-all]
```

## Configuration

### OpenAI

**Environment variable**:
```bash
export OPENAI_API_KEY="sk-..."
```

**Recipe config**:
```json
{
  "kind": "basic_qa",
  "params": {
    "llm": {
      "provider": "openai",
      "model": "gpt-4",
      "temperature": 0.7,
      "max_tokens": 512
    }
  }
}
```

**Available models**:
- `gpt-4` (recommended for quality)
- `gpt-3.5-turbo` (faster, cheaper)
- `gpt-4-turbo`
- `gpt-4o`

---

### Anthropic (Claude)

**Environment variable**:
```bash
export ANTHROPIC_API_KEY="sk-ant-..."
```

**Recipe config**:
```json
{
  "kind": "outline_summary",
  "params": {
    "llm": {
      "provider": "anthropic",
      "model": "claude-3-5-sonnet-20241022",
      "temperature": 0.5,
      "max_tokens": 1024
    }
  }
}
```

**Available models**:
- `claude-3-5-sonnet-20241022` (recommended, latest)
- `claude-3-opus-20240229` (highest quality)
- `claude-3-sonnet-20240229`
- `claude-3-haiku-20240307` (fastest)

---

### Ollama (Local Models)

**Prerequisites**:
1. Install Ollama: https://ollama.ai
2. Pull a model: `ollama pull llama3.2`
3. Start Ollama server: `ollama serve` (usually runs on http://localhost:11434)

**Recipe config**:
```json
{
  "kind": "basic_qa",
  "params": {
    "llm": {
      "provider": "ollama",
      "model": "llama3.2",
      "base_url": "http://localhost:11434",
      "temperature": 0.6,
      "max_tokens": 800
    }
  }
}
```

**Available models** (after pulling):
- `llama3.2` (recommended, 3B/1B)
- `llama3.1` (8B/70B/405B)
- `mistral` (7B)
- `phi3` (3.8B, fast)
- `codellama` (code-focused)

---

### Dummy (Testing)

**Recipe config**:
```json
{
  "kind": "basic_qa",
  "params": {
    "llm": {
      "provider": "dummy",
      "prefix": "LLM"
    }
  }
}
```

Returns deterministic responses like `"LLM: <prompt>"`. Useful for testing without API costs.

---

## Python API

### Using LLM Clients Directly

```python
from neuraparse.llm import OpenAILLMClient, AnthropicLLMClient, OllamaLLMClient

# OpenAI
openai_client = OpenAILLMClient(
    model="gpt-4",
    api_key="sk-...",  # or set OPENAI_API_KEY env var
    temperature=0.7,
    max_tokens=512
)
response = openai_client.generate("What is Python?")

# Anthropic
anthropic_client = AnthropicLLMClient(
    model="claude-3-5-sonnet-20241022",
    temperature=0.5
)
response = anthropic_client.generate("Summarize this text: ...")

# Ollama
ollama_client = OllamaLLMClient(
    model="llama3.2",
    base_url="http://localhost:11434"
)
response = ollama_client.generate("Generate a question about: ...")
```

### Using LLM in Recipes

```python
from neuraparse.llm.base import build_llm_from_recipe_params

params = {
    "llm": {
        "provider": "openai",
        "model": "gpt-4",
        "temperature": 0.7
    }
}

llm = build_llm_from_recipe_params(params)
if llm:
    response = llm.generate("Your prompt here")
```

---

## Best Practices

### 1. **Choose the Right Model**

- **Quality-critical tasks** (QA, summaries): GPT-4, Claude 3.5 Sonnet
- **Cost-sensitive tasks**: GPT-3.5-turbo, Claude 3 Haiku
- **Privacy/local**: Ollama (Llama 3.2, Mistral)
- **Testing**: Dummy provider

### 2. **Temperature Settings**

- **Factual tasks** (QA, entity extraction): `temperature=0.3-0.5`
- **Creative tasks** (summaries, paraphrasing): `temperature=0.7-0.9`
- **Deterministic output**: `temperature=0.0`

### 3. **Token Limits**

- **Short answers** (QA): `max_tokens=256-512`
- **Summaries**: `max_tokens=512-1024`
- **Long-form**: `max_tokens=2048+`

### 4. **Error Handling**

```python
from neuraparse.llm import OpenAILLMClient

try:
    client = OpenAILLMClient(model="gpt-4")
    response = client.generate("Your prompt")
except ValueError as e:
    print(f"API key not set: {e}")
except RuntimeError as e:
    print(f"Package not installed: {e}")
```

### 5. **Cost Management**

- Use `gpt-3.5-turbo` for development/testing
- Switch to `gpt-4` for production
- Consider Ollama for high-volume, low-cost scenarios
- Monitor token usage with OpenAI/Anthropic dashboards

---

## Recipes That Use LLMs

| Recipe | LLM Usage | Recommended Provider |
|--------|-----------|---------------------|
| `basic_qa` | Generate questions from paragraphs | OpenAI GPT-4 |
| `outline_summary` | Generate section summaries | Anthropic Claude 3.5 |
| `entity_knowledge` | Optional: enrich entity descriptions | GPT-3.5-turbo |

---

## Troubleshooting

### "OpenAI API key not provided"

**Solution**:
```bash
export OPENAI_API_KEY="sk-..."
```

Or pass it in the recipe config:
```json
{
  "llm": {
    "provider": "openai",
    "api_key": "sk-..."
  }
}
```

### "Anthropic provider requires 'anthropic' package"

**Solution**:
```bash
pip install neuraparse[llm-anthropic]
```

### "Ollama connection refused"

**Solution**:
1. Start Ollama server: `ollama serve`
2. Check it's running: `curl http://localhost:11434`
3. Pull a model: `ollama pull llama3.2`

---

## Next Steps

- [Recipe Guide](recipes.md) - Learn about all recipes
- [Examples](../examples/) - See real LLM configs
- [Testing](../tests/test_llm_providers.py) - See mock-based tests

