---
authors:
  - Zhiyuan Chen
date: 2024-05-04
---

# tokenisers

`tokenisers` provide a collection of pre-defined tokenizers.

A tokenizer is a class that converts a sequence of nucleotides or amino acids into a sequence of indices. It is used to pre-process the input sequence before feeding it into a model.

Please refer to [Tokenizer](https://huggingface.co/docs/transformers/main/en/main_classes/tokenizer) for more details.

## Available Tokenizers

- [DnaTokenizer](tokenisers/dna.md)
- [RnaTokenizer](tokenisers/rna.md)
- [ProteinTokenizer](tokenisers/protein.md)
