Metadata-Version: 2.1
Name: optimum
Version: 1.25.0
Summary: Optimum Library is an extension of the Hugging Face Transformers library, providing a framework to integrate third-party libraries from Hardware Partners and interface with their specific functionality.
Home-page: https://github.com/huggingface/optimum
Author: HuggingFace Inc. Special Ops Team
Author-email: hardware@huggingface.co
License: Apache
Keywords: transformers,quantization,pruning,optimization,training,inference,onnx,onnx runtime,intel,habana,graphcore,neural compressor,ipu,hpu
Classifier: Development Status :: 5 - Production/Stable
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9.0
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: transformers >=4.29
Requires-Dist: torch >=1.11
Requires-Dist: packaging
Requires-Dist: numpy
Requires-Dist: huggingface-hub >=0.8.0
Provides-Extra: amd
Requires-Dist: optimum-amd ; extra == 'amd'
Provides-Extra: benchmark
Requires-Dist: optuna ; extra == 'benchmark'
Requires-Dist: tqdm ; extra == 'benchmark'
Requires-Dist: scikit-learn ; extra == 'benchmark'
Requires-Dist: seqeval ; extra == 'benchmark'
Requires-Dist: torchvision ; extra == 'benchmark'
Requires-Dist: evaluate >=0.2.0 ; extra == 'benchmark'
Provides-Extra: dev
Requires-Dist: accelerate ; extra == 'dev'
Requires-Dist: pytest <=8.0.0 ; extra == 'dev'
Requires-Dist: requests ; extra == 'dev'
Requires-Dist: parameterized ; extra == 'dev'
Requires-Dist: pytest-xdist ; extra == 'dev'
Requires-Dist: Pillow ; extra == 'dev'
Requires-Dist: sacremoses ; extra == 'dev'
Requires-Dist: torchvision ; extra == 'dev'
Requires-Dist: torchaudio ; extra == 'dev'
Requires-Dist: einops ; extra == 'dev'
Requires-Dist: timm ; extra == 'dev'
Requires-Dist: scikit-learn ; extra == 'dev'
Requires-Dist: sentencepiece ; extra == 'dev'
Requires-Dist: rjieba ; extra == 'dev'
Requires-Dist: hf-xet ; extra == 'dev'
Requires-Dist: black ~=23.1 ; extra == 'dev'
Requires-Dist: ruff ==0.1.5 ; extra == 'dev'
Provides-Extra: doc-build
Requires-Dist: accelerate ; extra == 'doc-build'
Provides-Extra: exporters
Requires-Dist: onnx ; extra == 'exporters'
Requires-Dist: timm ; extra == 'exporters'
Requires-Dist: onnxruntime ; extra == 'exporters'
Requires-Dist: protobuf >=3.20.1 ; extra == 'exporters'
Requires-Dist: transformers <4.52.0,>=4.36 ; extra == 'exporters'
Provides-Extra: exporters-gpu
Requires-Dist: onnx ; extra == 'exporters-gpu'
Requires-Dist: timm ; extra == 'exporters-gpu'
Requires-Dist: onnxruntime-gpu ; extra == 'exporters-gpu'
Requires-Dist: protobuf >=3.20.1 ; extra == 'exporters-gpu'
Requires-Dist: transformers <4.52.0,>=4.36 ; extra == 'exporters-gpu'
Provides-Extra: exporters-tf
Requires-Dist: onnx ; extra == 'exporters-tf'
Requires-Dist: timm ; extra == 'exporters-tf'
Requires-Dist: h5py ; extra == 'exporters-tf'
Requires-Dist: tf2onnx ; extra == 'exporters-tf'
Requires-Dist: onnxruntime ; extra == 'exporters-tf'
Requires-Dist: numpy <1.24.0 ; extra == 'exporters-tf'
Requires-Dist: datasets <=2.16 ; extra == 'exporters-tf'
Requires-Dist: tensorflow <=2.12.1,>=2.4 ; extra == 'exporters-tf'
Requires-Dist: transformers <4.38,>=4.36 ; extra == 'exporters-tf'
Provides-Extra: furiosa
Requires-Dist: optimum-furiosa ; extra == 'furiosa'
Provides-Extra: graphcore
Requires-Dist: optimum-graphcore ; extra == 'graphcore'
Provides-Extra: habana
Requires-Dist: optimum-habana ; extra == 'habana'
Requires-Dist: transformers <4.46.0,>=4.45.0 ; extra == 'habana'
Provides-Extra: intel
Requires-Dist: optimum-intel >=1.18.0 ; extra == 'intel'
Provides-Extra: ipex
Requires-Dist: optimum-intel[ipex] >=1.18.0 ; extra == 'ipex'
Provides-Extra: neural-compressor
Requires-Dist: optimum-intel[neural-compressor] >=1.18.0 ; extra == 'neural-compressor'
Provides-Extra: neuronx
Requires-Dist: optimum-neuron[neuronx] >=0.0.28 ; extra == 'neuronx'
Provides-Extra: nncf
Requires-Dist: optimum-intel[nncf] >=1.18.0 ; extra == 'nncf'
Provides-Extra: onnxruntime
Requires-Dist: onnx ; extra == 'onnxruntime'
Requires-Dist: datasets >=1.2.1 ; extra == 'onnxruntime'
Requires-Dist: protobuf >=3.20.1 ; extra == 'onnxruntime'
Requires-Dist: onnxruntime >=1.11.0 ; extra == 'onnxruntime'
Requires-Dist: transformers <4.52.0,>=4.36 ; extra == 'onnxruntime'
Provides-Extra: onnxruntime-gpu
Requires-Dist: onnx ; extra == 'onnxruntime-gpu'
Requires-Dist: datasets >=1.2.1 ; extra == 'onnxruntime-gpu'
Requires-Dist: protobuf >=3.20.1 ; extra == 'onnxruntime-gpu'
Requires-Dist: onnxruntime-gpu >=1.11.0 ; extra == 'onnxruntime-gpu'
Requires-Dist: transformers <4.52.0,>=4.36 ; extra == 'onnxruntime-gpu'
Provides-Extra: onnxruntime-training
Requires-Dist: evaluate ; extra == 'onnxruntime-training'
Requires-Dist: torch-ort ; extra == 'onnxruntime-training'
Requires-Dist: accelerate ; extra == 'onnxruntime-training'
Requires-Dist: datasets >=1.2.1 ; extra == 'onnxruntime-training'
Requires-Dist: protobuf >=3.20.1 ; extra == 'onnxruntime-training'
Requires-Dist: transformers <4.52.0,>=4.36 ; extra == 'onnxruntime-training'
Requires-Dist: onnxruntime-training >=1.11.0 ; extra == 'onnxruntime-training'
Provides-Extra: openvino
Requires-Dist: optimum-intel[openvino] >=1.18.0 ; extra == 'openvino'
Provides-Extra: quality
Requires-Dist: black ~=23.1 ; extra == 'quality'
Requires-Dist: ruff ==0.1.5 ; extra == 'quality'
Provides-Extra: quanto
Requires-Dist: optimum-quanto >=0.2.4 ; extra == 'quanto'
Provides-Extra: tests
Requires-Dist: accelerate ; extra == 'tests'
Requires-Dist: pytest <=8.0.0 ; extra == 'tests'
Requires-Dist: requests ; extra == 'tests'
Requires-Dist: parameterized ; extra == 'tests'
Requires-Dist: pytest-xdist ; extra == 'tests'
Requires-Dist: Pillow ; extra == 'tests'
Requires-Dist: sacremoses ; extra == 'tests'
Requires-Dist: torchvision ; extra == 'tests'
Requires-Dist: torchaudio ; extra == 'tests'
Requires-Dist: einops ; extra == 'tests'
Requires-Dist: timm ; extra == 'tests'
Requires-Dist: scikit-learn ; extra == 'tests'
Requires-Dist: sentencepiece ; extra == 'tests'
Requires-Dist: rjieba ; extra == 'tests'
Requires-Dist: hf-xet ; extra == 'tests'

[![ONNX Runtime](https://github.com/huggingface/optimum/actions/workflows/test_onnxruntime.yml/badge.svg)](https://github.com/huggingface/optimum/actions/workflows/test_onnxruntime.yml)

# Hugging Face Optimum

🤗 Optimum is an extension of 🤗 Transformers and Diffusers, providing a set of optimization tools enabling maximum efficiency to train and run models on targeted hardware, while keeping things easy to use.

## Installation

🤗 Optimum can be installed using `pip` as follows:

```bash
python -m pip install optimum
```

If you'd like to use the accelerator-specific features of 🤗 Optimum, you can install the required dependencies according to the table below:

| Accelerator                                                                                                            | Installation                                                      |
|:-----------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------|
| [ONNX Runtime](https://huggingface.co/docs/optimum/onnxruntime/overview)                                               | `pip install --upgrade --upgrade-strategy eager optimum[onnxruntime]`      |
| [ExecuTorch](https://github.com/huggingface/optimum-executorch)                                                         | `pip install --upgrade --upgrade-strategy eager optimum[executorch]`
| [Intel Neural Compressor](https://huggingface.co/docs/optimum/intel/index)                                             | `pip install --upgrade --upgrade-strategy eager optimum[neural-compressor]`|
| [OpenVINO](https://huggingface.co/docs/optimum/intel/index)                                                            | `pip install --upgrade --upgrade-strategy eager optimum[openvino]`         |
| [NVIDIA TensorRT-LLM](https://huggingface.co/docs/optimum/main/en/nvidia_overview)                                     | `docker run -it --gpus all --ipc host huggingface/optimum-nvidia`          |
| [AMD Instinct GPUs and Ryzen AI NPU](https://huggingface.co/docs/optimum/amd/index)                                    | `pip install --upgrade --upgrade-strategy eager optimum[amd]`              |
| [AWS Trainum & Inferentia](https://huggingface.co/docs/optimum-neuron/index)                                           | `pip install --upgrade --upgrade-strategy eager optimum[neuronx]`          |
| [Habana Gaudi Processor (HPU)](https://huggingface.co/docs/optimum/habana/index)                                       | `pip install --upgrade --upgrade-strategy eager optimum[habana]`           |
| [FuriosaAI](https://huggingface.co/docs/optimum/furiosa/index)                                                         | `pip install --upgrade --upgrade-strategy eager optimum[furiosa]`          |

The `--upgrade --upgrade-strategy eager` option is needed to ensure the different packages are upgraded to the latest possible version.

To install from source:

```bash
python -m pip install git+https://github.com/huggingface/optimum.git
```

For the accelerator-specific features, append `optimum[accelerator_type]` to the above command:

```bash
python -m pip install optimum[onnxruntime]@git+https://github.com/huggingface/optimum.git
```

## Accelerated Inference

🤗 Optimum provides multiple tools to export and run optimized models on various ecosystems:

- [ONNX](https://huggingface.co/docs/optimum/exporters/onnx/usage_guides/export_a_model) / [ONNX Runtime](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models)
- [ExecuTorch](https://huggingface.co/docs/optimum-executorch/guides/export), PyTorch’s native solution to inference on the Edge, more details [here](https://pytorch.org/executorch/stable/)
- TensorFlow Lite
- [OpenVINO](https://huggingface.co/docs/optimum/intel/inference)
- Habana first-gen Gaudi / Gaudi2, more details [here](https://huggingface.co/docs/optimum/main/en/habana/usage_guides/accelerate_inference)
- AWS Inferentia 2 / Inferentia 1, more details [here](https://huggingface.co/docs/optimum-neuron/en/guides/models)
- NVIDIA TensorRT-LLM , more details [here](https://huggingface.co/blog/optimum-nvidia)

The [export](https://huggingface.co/docs/optimum/exporters/overview) and optimizations can be done both programmatically and with a command line.


### ONNX + ONNX Runtime

Before you begin, make sure you have all the necessary libraries installed :

```bash
pip install optimum[exporters,onnxruntime]
```

It is possible to export 🤗 Transformers and Diffusers models to the [ONNX](https://onnx.ai/) format and perform graph optimization as well as quantization easily.

For more information on the ONNX export, please check the [documentation](https://huggingface.co/docs/optimum/exporters/onnx/usage_guides/export_a_model).

Once the model is exported to the ONNX format, we provide Python classes enabling you to run the exported ONNX model in a seemless manner using [ONNX Runtime](https://onnxruntime.ai/) in the backend.

More details on how to run ONNX models with `ORTModelForXXX` classes [here](https://huggingface.co/docs/optimum/main/en/onnxruntime/usage_guides/models).


### ExecuTorch

Before you begin, make sure you have all the necessary libraries installed :

```bash
pip install optimum[exporters-executorch]
```

Users can export 🤗 Transformers models to [ExecuTorch](https://github.com/pytorch/executorch) and run inference on edge devices within PyTorch's ecosystem.

For more information about export 🤗 Transformers to ExecuTorch, please check the doc for [Optimum-ExecuTorch](https://huggingface.co/docs/optimum-executorch/guides/export).


### TensorFlow Lite

Before you begin, make sure you have all the necessary libraries installed :

```bash
pip install optimum[exporters-tf]
```

Just as for ONNX, it is possible to export models to [TensorFlow Lite](https://www.tensorflow.org/lite) and quantize them.
You can find more information in our [documentation](https://huggingface.co/docs/optimum/main/exporters/tflite/usage_guides/export_a_model).

### Intel (OpenVINO + Neural Compressor + IPEX)

Before you begin, make sure you have all the necessary [libraries installed](https://huggingface.co/docs/optimum/main/en/intel/installation).

You can find more information on the different integration in our [documentation](https://huggingface.co/docs/optimum/main/en/intel/index) and in the examples of [`optimum-intel`](https://github.com/huggingface/optimum-intel).


### Quanto

[Quanto](https://github.com/huggingface/optimum-quanto) is a pytorch quantization backenb which allowss you to quantize a model either using the python API or the `optimum-cli`.

You can see more details and [examples](https://github.com/huggingface/optimum-quanto/tree/main/examples) in the [Quanto](https://github.com/huggingface/optimum-quanto) repository.

## Accelerated training

🤗 Optimum provides wrappers around the original 🤗 Transformers [Trainer](https://huggingface.co/docs/transformers/main_classes/trainer) to enable training on powerful hardware easily.
We support many providers:

- Habana's Gaudi processors
- AWS Trainium instances, check [here](https://huggingface.co/docs/optimum-neuron/en/guides/distributed_training)
- ONNX Runtime (optimized for GPUs)

### Habana

Before you begin, make sure you have all the necessary libraries installed :

```bash
pip install --upgrade --upgrade-strategy eager optimum[habana]
```

You can find examples in the [documentation](https://huggingface.co/docs/optimum/habana/quickstart) and in the [examples](https://github.com/huggingface/optimum-habana/tree/main/examples).

### ONNX Runtime


Before you begin, make sure you have all the necessary libraries installed :

```bash
pip install optimum[onnxruntime-training]
```

You can find examples in the [documentation](https://huggingface.co/docs/optimum/onnxruntime/usage_guides/trainer) and in the [examples](https://github.com/huggingface/optimum/tree/main/examples/onnxruntime/training).
