Metadata-Version: 2.2
Name: turnkeyml
Version: 6.0.2
Summary: TurnkeyML Tools and Models
Author-email: turnkeyml@amd.com
Requires-Python: >=3.8, <3.12
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE.md
Requires-Dist: invoke>=2.0.0
Requires-Dist: onnx>=1.11.0
Requires-Dist: onnxmltools==1.10.0
Requires-Dist: torch>=1.12.1
Requires-Dist: pyyaml>=5.4
Requires-Dist: typeguard>=2.3.13
Requires-Dist: packaging>=20.9
Requires-Dist: numpy<2.0.0
Requires-Dist: pandas>=1.5.3
Requires-Dist: fasteners
Requires-Dist: GitPython>=3.1.40
Requires-Dist: psutil
Requires-Dist: wmi
Requires-Dist: pytz
Requires-Dist: tqdm
Requires-Dist: zstandard
Requires-Dist: matplotlib
Requires-Dist: tabulate
Requires-Dist: onnxruntime>=1.10.1; platform_system == "Linux" and extra != "llm-oga-cuda"
Requires-Dist: onnxruntime-directml>=1.19.0; platform_system == "Windows" and extra != "llm-oga-cuda"
Requires-Dist: onnxruntime-gpu>=1.19.1; extra == "llm-oga-cuda"
Provides-Extra: llm
Requires-Dist: torch>=2.0.0; extra == "llm"
Requires-Dist: transformers; extra == "llm"
Requires-Dist: accelerate; extra == "llm"
Requires-Dist: py-cpuinfo; extra == "llm"
Requires-Dist: sentencepiece; extra == "llm"
Requires-Dist: datasets; extra == "llm"
Requires-Dist: human-eval-windows==1.0.4; extra == "llm"
Requires-Dist: fastapi; extra == "llm"
Requires-Dist: uvicorn[standard]; extra == "llm"
Requires-Dist: openai; extra == "llm"
Provides-Extra: llm-oga-cpu
Requires-Dist: onnxruntime-genai>=0.5.2; extra == "llm-oga-cpu"
Requires-Dist: torch<2.4,>=2.0.0; extra == "llm-oga-cpu"
Requires-Dist: transformers<4.45.0; extra == "llm-oga-cpu"
Requires-Dist: turnkeyml[llm]; extra == "llm-oga-cpu"
Provides-Extra: llm-oga-igpu
Requires-Dist: onnxruntime-genai-directml>=0.4.0; extra == "llm-oga-igpu"
Requires-Dist: torch<2.4,>=2.0.0; extra == "llm-oga-igpu"
Requires-Dist: transformers<4.45.0; extra == "llm-oga-igpu"
Requires-Dist: turnkeyml[llm]; extra == "llm-oga-igpu"
Provides-Extra: llm-oga-cuda
Requires-Dist: onnxruntime-genai-cuda>=0.4.0; extra == "llm-oga-cuda"
Requires-Dist: torch<2.4,>=2.0.0; extra == "llm-oga-cuda"
Requires-Dist: transformers<4.45.0; extra == "llm-oga-cuda"
Requires-Dist: turnkeyml[llm]; extra == "llm-oga-cuda"
Provides-Extra: llm-oga-npu
Requires-Dist: onnx==1.16.0; extra == "llm-oga-npu"
Requires-Dist: onnxruntime==1.18.0; extra == "llm-oga-npu"
Requires-Dist: numpy==1.26.4; extra == "llm-oga-npu"
Requires-Dist: turnkeyml[llm]; extra == "llm-oga-npu"
Provides-Extra: llm-oga-hybrid
Requires-Dist: onnx==1.16.1; extra == "llm-oga-hybrid"
Requires-Dist: numpy==1.26.4; extra == "llm-oga-hybrid"
Requires-Dist: turnkeyml[llm]; extra == "llm-oga-hybrid"
Dynamic: author-email
Dynamic: description
Dynamic: description-content-type
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# Welcome to ONNX TurnkeyML

[![Turnkey tests](https://github.com/onnx/turnkeyml/actions/workflows/test_turnkey.yml/badge.svg)](https://github.com/onnx/turnkeyml/tree/main/test "Check out our tests")
[![Lemonade tests](https://github.com/onnx/turnkeyml/actions/workflows/test_lemonade.yml/badge.svg)](https://github.com/onnx/turnkeyml/tree/main/test "Check out our tests")
[![OS - Windows | Linux](https://img.shields.io/badge/OS-windows%20%7C%20linux-blue)](https://github.com/onnx/turnkeyml/blob/main/docs/install.md "Check out our instructions")
[![Made with Python](https://img.shields.io/badge/Python-3.8,3.10-blue?logo=python&logoColor=white)](https://github.com/onnx/turnkeyml/blob/main/docs/install.md "Check out our instructions")

We are on a mission to make it easy to use the most important tools in the ONNX ecosystem. TurnkeyML accomplishes this by providing no-code CLIs and low-code APIs for both general ONNX workflows with `turnkey` as well as LLMs with `lemonade`.

|                     [**Lemonade SDK**](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/getting_started.md)                    	|                            [**Turnkey**](https://github.com/onnx/turnkeyml/blob/main/docs/turnkey/getting_started.md)                                	|
|:----------------------------------------------:	|:-----------------------------------------------------------------:	|
| Serve and benchmark LLMs on CPU, GPU, and NPU. <br/>	[Click here to get started with `lemonade`.](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/getting_started.md) | Export and optimize ONNX models for CNNs and Transformers. <br/>	[Click here to get started with `turnkey`.](https://github.com/onnx/turnkeyml/blob/main/docs/turnkey/getting_started.md)	|
| <img src="https://github.com/onnx/turnkeyml/blob/main/img/llm_demo.png?raw=true"/> | <img src="https://github.com/onnx/turnkeyml/blob/main/img/classic_demo.png?raw=true"/> |


## How It Works

The `turnkey` (CNNs and transformers) and `lemonade` (LLMs) CLIs provide a set of `Tools` that users can invoke in a `Sequence`. The first `Tool` takes the input (`-i`), performs some action, and passes its state to the next `Tool` in the `Sequence`.

You can read the `Sequence` out like a sentence. For example, the demo command above was:

```
> turnkey -i bert.py discover export-pytorch optimize-ort convert-fp16
```

Which you can read like:

> Use `turnkey` on `bert.py` to `discover` the model, `export` the `pytorch` to ONNX, `optimize` the ONNX with `ort`, and `convert` the ONNX to `fp16`.

You can configure each `Tool` by passing it arguments. For example, `export-pytorch --opset 18` would set the opset of the resulting ONNX model to 18.

A full command with an argument looks like:

```
> turnkey -i bert.py discover export-pytorch --opset 18 optimize-ort convert-fp16
```


## Contributing

We are actively seeking collaborators from across the industry. If you would like to contribute to this project, please check out our [contribution guide](https://github.com/onnx/turnkeyml/blob/main/docs/contribute.md).

## Maintainers

This project is sponsored by the [ONNX Model Zoo](https://github.com/onnx/models) special interest group (SIG). It is maintained by @danielholanda @jeremyfowers @ramkrishna @vgodsoe in equal measure. You can reach us by filing an [issue](https://github.com/onnx/turnkeyml/issues) or emailing `turnkeyml at amd dot com`.

## License

This project is licensed under the [Apache 2.0 License](https://github.com/onnx/turnkeyml/blob/main/LICENSE).

## Attribution

TurnkeyML used code from other open source projects as a starting point (see [NOTICE.md](NOTICE.md)). Thank you Philip Colangelo, Derek Elkins, Jeremy Fowers, Dan Gard, Victoria Godsoe, Mark Heaps, Daniel Holanda, Brian Kurtz, Mariah Larwood, Philip Lassen, Andrew Ling, Adrian Macias, Gary Malik, Sarah Massengill, Ashwin Murthy, Hatice Ozen, Tim Sears, Sean Settle, Krishna Sivakumar, Aviv Weinstein, Xueli Xao, Bill Xing, and Lev Zlotnik for your contributions to that work.

