<img src="./fig4.png" width="400px"></img>

## UltraMem (wip)

Implementation of [UltraMem](https://arxiv.org/abs/2411.12364v1), improved Product Key Memory design, from Bytedance AI labs

## Citations

```bibtex
@misc{huang2025ultrasparsememorynetwork,
    title   = {Ultra-Sparse Memory Network}, 
    author  = {Zihao Huang and Qiyang Min and Hongzhi Huang and Defa Zhu and Yutao Zeng and Ran Guo and Xun Zhou},
    year    = {2025},
    eprint  = {2411.12364},
    archivePrefix = {arXiv},
    primaryClass = {cs.LG},
    url     = {https://arxiv.org/abs/2411.12364}, 
}
```

```bibtex
@inproceedings{anonymous2025continual,
    title   = {Continual Learning via Sparse Memory Finetuning},
    author  = {Anonymous},
    booktitle = {Submitted to The Fourteenth International Conference on Learning Representations},
    year    = {2025},
    url     = {https://openreview.net/forum?id=LGo7U1m24L},
    note    = {under review}
}
```
