# LLM Analytics Python SDK

Python SDK for collecting LLM app traces (chat, RAG, vision, structured output), standardizing them, and sending them to an Analytics Bank API for scoring.

## Features
- Pydantic v2 strict payload models.
- Sync and async clients via `httpx`.
- Built-in scoring tags via `ScoringTag`.
- Automatic UUID generation for `trace_id`.

## Installation
```bash
pip install gaunt
```

For local development:
```bash
pip install -e ".[dev]"
```

## Quickstart
```python
from gaunt import Client, ScoringTag

client = Client(
    api_key="sk_live_xxx",
    base_url="https://analytics-bank.example.com",
)

result = client.log(
    project_id="project_123",
    inputs={
        "messages": [
            {"role": "system", "content": "You are a concise assistant."},
            {"role": "user", "content": "What is RAG?"},
        ],
        "rag_context": [
            {"content": "RAG combines retrieval and generation.", "source_id": "doc-1"},
        ],
    },
    outputs={
        "raw": "RAG uses retrieved knowledge to ground generations.",
    },
    expected={"topic": "rag"},
    metadata={"env": "prod"},
    scoring_tags=[
        ScoringTag.RAG_FAITHFULNESS,
        ScoringTag.ANSWER_CORRECTNESS,
    ],
)

print(result)
client.close()
```

## Scoring Tags
- `ScoringTag.SCHEMA_VALIDITY` -> `score:schema_validity`
- `ScoringTag.RAG_FAITHFULNESS` -> `score:rag_faithfulness`
- `ScoringTag.VISION_HALLUCINATION` -> `score:vision_hallucination`
- `ScoringTag.TOXICITY` -> `score:toxicity`
- `ScoringTag.NUMERIC_SCORING` -> `score:numeric_scoring`
- `ScoringTag.ANSWER_CORRECTNESS` -> `score:answer_correctness`

## Payload Shape
The SDK emits payloads in this form:

```json
{
  "project_id": "project_123",
  "trace_id": "uuid...",
  "inputs": {
    "messages": [],
    "images": [],
    "rag_context": [],
    "json_schema": {}
  },
  "outputs": {
    "raw": "...",
    "parsed": {}
  },
  "expected": {},
  "metadata": {},
  "config": {
    "scoring_tags": ["score:rag_faithfulness", "score:schema_validity"]
  }
}
```

## Testing
```bash
pytest -q
```

## Publishing
See `docs/publishing.md` for a release checklist and TestPyPI/PyPI upload commands.
