# guido-inference

Run [Guido](https://huggingface.co/northsea-ai) models locally. Model code with **Titans memory update during inference**, HuggingFace loader, cartridge support, and CLI.

Guido's memory evolves as you talk to it. Save the memory state as a `.cart` cartridge file and reload it later — the model picks up where it left off.

## Installation

```bash
pip install guido-inference
```

For Flash Linear Attention kernel support (GPU):
```bash
pip install guido-inference[fla]
```

## Quick Start

### CLI

```bash
guido run northsea-ai/Guido-3B --prompt "What is the capital of the Netherlands?"
guido run northsea-ai/Guido-3B                          # Interactive mode
guido run northsea-ai/Guido-3B --effort high             # Adaptive compute
guido run northsea-ai/Guido-3B --cartridge ./john.cart   # Resume with saved memory
guido info northsea-ai/Guido-3B                          # Model info (no GPU)
```

### Python

```python
from guido_inference import load_model

model, tokenizer = load_model("northsea-ai/Guido-3B")
input_ids = tokenizer.encode("Hello!", return_tensors="pt").to(model.device)
output = model.generate(input_ids)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

### Cartridges — Create, Save, Reload

Memory updates happen automatically during inference (Titans architecture). Save and reload the evolved state:

```python
# Conversation evolves the model's memory
for msg in ["I'm a researcher at TU Delft.", "My focus is renewable energy."]:
    ids = tokenizer.encode(msg, return_tensors="pt").to(model.device)
    model.generate(ids)

# Save evolved memory as portable cartridge (~250KB-1MB)
model.save_memory("./researcher.cart")

# Later: reload and continue where you left off
model.load_memory("./researcher.cart")
output = model.generate(tokenizer.encode("What papers should I read?", return_tensors="pt").to(model.device))

# Reset memory to blank slate
model.reset_memory()
```

## Model Downloads

| Model | Params (Total) | Params (Active) | HuggingFace |
|-------|---------------|-----------------|-------------|
| Guido-300M | 0.42B | 0.29B | [northsea-ai/Guido-300M](https://huggingface.co/northsea-ai/Guido-300M) |
| Guido-3B | 3.33B | 0.84B | [northsea-ai/Guido-3B](https://huggingface.co/northsea-ai/Guido-3B) |
| Guido-7B | 7.36B | 1.32B | [northsea-ai/Guido-7B](https://huggingface.co/northsea-ai/Guido-7B) |

For fine-tuning, use third-party training harnesses like [Axolotl](https://github.com/axolotl-ai-cloud/axolotl) or [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) with a LoRA/QLoRA adapter.

See the [monorepo README](../../README.md) for architecture details and full documentation.
