# Training on TPU

Welcome to the 🤗 Optimum-TPU training guide! This section covers how to fine-tune models using Google Cloud TPUs.

## Currently Supported Models

The following models have been tested and validated for fine-tuning on TPU v5e:

- 🦙 LLaMA Family
  - LLaMA-2 7B
  - LLaMA-3 8B
- 💎 Gemma Family
  - Gemma 2B
  - Gemma 7B

## Getting Started

### Prerequisites

Before starting the training process, ensure you have:

1. A configured Google Cloud TPU instance (see [Deployment Guide](./deploy))
2. Optimum-TPU installed with PyTorch/XLA support:
```bash
pip install optimum-tpu -f https://storage.googleapis.com/libtpu-releases/index.html
export PJRT_DEVICE=TPU
```

### Example Training Scripts

We provide several example scripts to help you get started:

1. Gemma Fine-tuning:
   - See our [Gemma fine-tuning notebook](https://github.com/huggingface/optimum-tpu/blob/main/examples/language-modeling/gemma_tuning.ipynb) for a step-by-step guide

2. LLaMA Fine-tuning:
   - Check our [LLaMA fine-tuning script](https://github.com/huggingface/optimum-tpu/blob/main/examples/language-modeling/llama_tuning.md) for detailed instructions