[![Multi-Modality](agorabanner.png)](https://discord.gg/qUtxnK2NMf)

# Pali3
![pali](pali.png)

"Figure 1: Overview of the PaLI-3 (5B) model: images are encoded into visual tokens individually
by the contrastively pretrained 2B SigLIP vision model. Along with a query, these visual tokens
are passed to an 3B encoder-decoder UL2 Transformer which produces the desired answer."



## Installation

`pip install pali3`


## Usage:



# License
MIT

# Todo

- [ ] Implement sig_lip vit model with training recipe
- [ ] Implement the text tokenizer, maybe use token monster 
- [ ] Implement the UL2 Transformer Encoder and Decoder
- [ ] Implement training scripts
- [ ] 