Metadata-Version: 2.1
Name: retentive-network
Version: 0.0.1
Summary: Unofficial codebase for the "Retentive Network: A Successor to Transformer for Large Language Models" paper [https://arxiv.org/pdf/2307.08621.pdf]
License: MIT
Author: Zach Bloss
Author-email: zacharybloss@gmail.com
Requires-Python: >=3.9,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: numpy (>=1.25.1,<2.0.0)
Requires-Dist: torch (>=2.0.1,<3.0.0)
Description-Content-Type: text/markdown

# RetentiveNetwork
Unofficial codebase for the "Retentive Network: A Successor to Transformer for Large Language Models" paper [https://arxiv.org/pdf/2307.08621.pdf]

The official codebase for RetNet should be made available roughly August 1st, 2023 according to Microsoft here: 

* [https://github.com/microsoft/unilm/tree/master/retnet](https://github.com/microsoft/unilm/tree/master/retnet)
* [https://github.com/microsoft/torchscale](https://github.com/microsoft/torchscale)
