Metadata-Version: 2.1
Name: tritony
Version: 0.0.9
Summary: Tiny configuration for Triton Inference Server
Home-page: https://github.com/rtzr/tritony
Author: Arthur Kim, RTZR team
Author-email: arthur@rtzr.ai
License: BSD
Project-URL: Download, https://pypi.org/project/tritony/#files
Project-URL: Source, https://github.com/rtzr/tritony
Project-URL: Tracker, https://github.com/rtzr/tritony/issues
Keywords: grpc,http,triton,tensorrt,inference,server,service,client,nvidia,rtzr
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Information Technology
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Utilities
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Environment :: Console
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown; charset=UTF-8
Provides-Extra: tests
Provides-Extra: display
License-File: LICENSE

# tritony - Tiny configuration for Triton Inference Server

![CI](https://github.com/rtzr/tritony/actions/workflows/pre-commit_pytest.yml/badge.svg)

## What is this?

If you see [the official example](https://github.com/triton-inference-server/client/tree/main/src/python/examples), it is really confusing to use where to start.

Use tritony! You will get really short lines of code like example below.

```python
import argparse
import os
from glob import glob
import numpy as np
from PIL import Image

from tritony import InferenceClient


def preprocess(img, dtype=np.float32, h=224, w=224, scaling="INCEPTION"):
    sample_img = img.convert("RGB")

    resized_img = sample_img.resize((w, h), Image.Resampling.BILINEAR)
    resized = np.array(resized_img)
    if resized.ndim == 2:
        resized = resized[:, :, np.newaxis]

    scaled = (resized / 127.5) - 1
    ordered = np.transpose(scaled, (2, 0, 1))
    
    return ordered.astype(dtype)


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--image_folder", type=str, help="Input folder.")
    FLAGS = parser.parse_args()

    client = InferenceClient.create_with("densenet_onnx", "0.0.0.0:8001", input_dims=3, protocol="grpc")
    client.output_kwargs = {"class_count": 1}

    image_data = []
    for filename in glob(os.path.join(FLAGS.image_folder, "*")):
        image_data.append(preprocess(Image.open(filename)))

    result = client(np.asarray(image_data))

    for output in result:
        max_value, arg_max, class_name = output[0].decode("utf-8").split(":")
        print(f"{max_value} ({arg_max}) = {class_name}")
```

## Key Features

- [x] Simple configuration. Only `$host:$port` and `$model_name` are required.
- [x] Generating asynchronous requests with `asyncio.Queue`
- [x] Simple Model switching
- [ ] Support async tritonclient

## Requirements

    $ pip install tritonclient[all]

## Install

    $ pip install tritony

## Test

### With Triton

```bash
docker run --rm \
    -v ${PWD}:/models \
    nvcr.io/nvidia/tritonserver:22.01-pyt-python-py3 \
    tritonserver --model-repo=/models
```

```bash
pytest -m -s tests/test_tritony.py
```

### Example with image_client.py

- Follow steps
  in [the official triton server documentation](https://github.com/triton-inference-server/server#serve-a-model-in-3-easy-steps)

```bash
# Download Images from https://github.com/triton-inference-server/server.git
python ./example/image_client.py --image_folder "./server/qa/images"
```
