Metadata-Version: 2.1
Name: truss
Version: 0.6.1
Summary: A seamless bridge from model development to model delivery
Home-page: https://github.com/basetenlabs/truss
License: MIT
Keywords: MLOps,AI,Model Serving,Model Deployment,Machine Learning
Author: Pankaj Gupta
Author-email: pankaj@baseten.co
Requires-Python: >=3.8,<3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: Jinja2 (>=3.1.2,<4.0.0)
Requires-Dist: PyYAML (>=6.0,<7.0)
Requires-Dist: blake3 (>=0.3.3,<0.4.0)
Requires-Dist: boto3 (>=1.26.157,<2.0.0)
Requires-Dist: cloudpickle (>=2.2.0,<3.0.0)
Requires-Dist: fastapi (>=0.95.0,<0.96.0)
Requires-Dist: httpx (>=0.24.1,<0.25.0)
Requires-Dist: huggingface_hub (>=0.16.4,<0.17.0)
Requires-Dist: inquirerpy (>=0.3.4,<0.4.0)
Requires-Dist: joblib (>=1.2.0,<2.0.0)
Requires-Dist: msgpack (>=1.0.2)
Requires-Dist: msgpack-numpy (>=0.4.7.1)
Requires-Dist: numpy (==1.23.5)
Requires-Dist: packaging (>=20.9)
Requires-Dist: psutil (>=5.9.4,<6.0.0)
Requires-Dist: pydantic (>=1.10.9,<2.0.0)
Requires-Dist: python-json-logger (>=2.0.2)
Requires-Dist: python-on-whales (>=0.46.0,<0.47.0)
Requires-Dist: rich (>=13.4.2,<14.0.0)
Requires-Dist: rich-click (>=1.6.1,<2.0.0)
Requires-Dist: single-source (>=0.3.0,<0.4.0)
Requires-Dist: tenacity (>=8.0.1,<9.0.0)
Requires-Dist: uvicorn (>=0.21.1,<0.22.0)
Requires-Dist: watchfiles (>=0.19.0,<0.20.0)
Project-URL: Bug Reports, https://github.com/basetenlabs/truss/issues
Project-URL: Baseten, https://baseten.co
Project-URL: Documentation, https://truss.baseten.co
Project-URL: Homepage, https://truss.baseten.co
Project-URL: Repository, https://github.com/basetenlabs/truss
Description-Content-Type: text/markdown

# Truss

**The simplest way to serve AI/ML models in production**

[![PyPI version](https://badge.fury.io/py/truss.svg)](https://badge.fury.io/py/truss)
[![ci_status](https://github.com/basetenlabs/truss/actions/workflows/release.yml/badge.svg)](https://github.com/basetenlabs/truss/actions/workflows/release.yml)

## Why Truss?

* **Write once, run anywhere:** Package and test model code, weights, and dependencies with a model server that behaves the same in development and production.
* **Fast developer loop:** Implement your model with fast feedback from a live reload server, and skip Docker and Kubernetes configuration with a batteries-included model serving environment.
* **Support for all Python frameworks**: From `transformers` and `diffusors` to `PyTorch` and `Tensorflow` to `XGBoost` and `sklearn`, Truss supports models created with any framework, even entirely custom models.

See Trusses for popular models including:

* 🦅 [Falcon 40B](https://github.com/basetenlabs/falcon-40b-truss)
* 🧙 [WizardLM](https://github.com/basetenlabs/wizardlm-truss)
* 🎨 [Stable Diffusion](https://github.com/basetenlabs/stable-diffusion-truss)
* 🗣 [Whisper](https://github.com/basetenlabs/whisper-truss)

and [dozens more examples](examples/).

## Installation

Install Truss with:

```
pip install --upgrade truss
```

## Quickstart

As a quick example, we'll package a [text classification pipeline](https://huggingface.co/docs/transformers/main_classes/pipelines) from the open-source [`transformers` package](https://github.com/huggingface/transformers).

### Create a Truss

To get started, create a Truss with the following terminal command:

```
truss init text-classification
```

This will create an empty Truss at `./text-classification`.

### Implement the model

The model serving code goes in `./text-classification/model/model.py` in your newly created Truss.

```python
from typing import List
from transformers import pipeline


class Model:
    def __init__(self, **kwargs) -> None:
        self._model = None

    def load(self):
        self._model = pipeline("text-classification")

    def predict(self, model_input: str) -> List:
        return self._model(model_input)
```

There are two functions to implement:

* `load()` runs once when the model is spun up and is responsible for initializing `self._model`
* `predict()` runs each time the model is invoked and handles the inference. It can use any JSON-serializable type as input and output.

### Add model dependencies

The pipeline model relies on Transformers and PyTorch. These dependencies must be specified in the Truss config.

In `./text-classification/config.yaml`, find the line `requirements`. Replace the empty list with:

```yaml
requirements:
  - torch==2.0.1
  - transformers==4.30.0
```

No other configuration needs to be changed.

## Deployment

You can deploy a Truss to your [Baseten](https://baseten.co) account with:

```
cd ./text-classification
truss push
```

Truss will support other remotes soon, starting with AWS SageMaker.

## Truss contributors

Truss is backed by Baseten and built in collaboration with ML engineers worldwide. Special thanks to [Stephan Auerhahn](https://github.com/palp) @ [stability.ai](https://stability.ai/) and [Daniel Sarfati](https://github.com/dsarfati) @ [Salad Technologies](https://salad.com/) for their contributions.

We enthusiastically welcome contributions in accordance with our [contributors' guide](CONTRIBUTING.md) and [code of conduct](CODE_OF_CONDUCT.md).

