# TrainTrack

**Training-time evaluation and win-rate tracking for LLMs.**

TrainTrack helps you monitor model behavior during training by running automated LLM-as-a-Judge evaluations on every checkpoint.

## Features

- 🚀 **Real-time Metrics**: Get immediate feedback on conciseness, helpfulness, and reasoning quality.
- 📊 **Win-rate Tracking**: Automatically track win-rates against an anchor checkpoint (baseline) or the previous step.
- 📚 **Built-in Benchmarks**: Integrated support for GPQA, MMLU-Pro, IFEval, and TruthfulQA.
- 🛠️ **Seamless Integration**: Works with standard PyTorch loops and HuggingFace Trainer.

## Quick Installation

```bash
pip install traintrack-ai
```

## Minimal Example

```python
from traintrack import TrainTrackHook

# 1. Initialize the hook
hook = TrainTrackHook(
    model=model,
    tokenizer=tokenizer,
    run_name="my-first-run",
    datasets=["reasoning", "helpfulness"]
)

# 2. Capture a baseline (optional)
hook.capture_anchor()

# 3. Add to your training loop
for step, batch in enumerate(train_dataloader):
    # ... training logic ...

    hook.step(step)
```

## Documentation

For full documentation and advanced configuration (custom metrics, rubrics, and category-based evaluation), visit:
[github.com/traintrack/traintrack](https://github.com/traintrack/traintrack)
