Metadata-Version: 2.1
Name: torchserve-client
Version: 0.0.1
Summary: Python Client for TorchServe APIs
Home-page: https://github.com/Ankur-singh/torchserve_client
Author: Ankur Singh
Author-email: as.ankursingh3.1@gmail.com
License: Apache Software License 2.0
Keywords: nbdev jupyter notebook python
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: License :: OSI Approved :: Apache Software License
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Provides-Extra: dev
License-File: LICENSE

# TorchServe Python Client

<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

## Install

``` sh
pip install torchserve_client
```

## Usage

Using `torchserve_client` is a breeze! To get started, simply initialize
a `TorchServeClient` object as shown below:

``` python
from torchserve_client import TorchServeClient

# Initialize the TorchServeClient object
ts_client = TorchServeClient()
ts_client
```

    TorchServeClient(base_url=http://localhost, management_port=8081, inference_port=8080)

If you wish to customize the *base URL*, *management port*, or
*inference port* of your TorchServe server, you can pass them as
arguments during initialization:

``` python
from torchserve_client import TorchServeClient

# Customize the base URL, management port, and inference port
ts_client = TorchServeClient(base_url='http://your-torchserve-server.com', 
                             management_port=8081, inference_port=8080)
ts_client
```

    TorchServeClient(base_url=http://your-torchserve-server.com, management_port=8081, inference_port=8080)

Alternatively, if you don’t provide a base URL during initialization,
the client will check for the presence of `TORCHSERVE_URL` in the
environment variables. If the variable is not found, it will gracefully
fall back to using *localhost* as the default. This way, you have the
flexibility to tailor your TorchServeClient to your needs effortlessly!
Happy serving! 🍿🔥

### Management APIs

With TorchServe Management APIs, you can effortlessly manage your models
at runtime. Here’s a quick rundown of the actions you can perform using
our `TorchServeClient` SDK:

1.  **Register a Model**: Easily register a model with TorchServe using
    the `ts_client.management.register_model()` method.

``` python
ts_client.management.register_model('https://torchserve.pytorch.org/mar_files/squeezenet1_1.mar')
```

    {'status': 'Model "squeezenet1_1" Version: 1.0 registered with 0 initial workers. Use scale workers API to add workers for the model.'}

2.  **Increase/Decrease Workers**: Scale the number of workers for a
    specific model with simplicity using
    `ts_client.management.scale_workers()`.

``` python
ts_client.management.scale_workers('squeezenet1_1', min_worker=1, max_worker=2)
```

    {'status': 'Processing worker updates...'}

3.  **Model Status**: Curious about a model’s status? Fetch all the
    details you need using `ts_client.management.describe_model()`.

``` python
ts_client.management.describe_model('squeezenet1_1')
```

    [{'modelName': 'squeezenet1_1',
      'modelVersion': '1.0',
      'modelUrl': 'https://torchserve.pytorch.org/mar_files/squeezenet1_1.mar',
      'runtime': 'python',
      'minWorkers': 1,
      'maxWorkers': 1,
      'batchSize': 1,
      'maxBatchDelay': 100,
      'loadedAtStartup': False,
      'workers': [{'id': '9001',
        'startTime': '2023-07-17T22:55:40.155Z',
        'status': 'UNLOADING',
        'memoryUsage': 0,
        'pid': -1,
        'gpu': False,
        'gpuUsage': 'N/A'}]}]

4.  **List Registered Models**: Quickly fetch a list of all registered
    models using `ts_client.management.list_models()`.

``` python
# List all models
ts_client.management.list_models()
```

    {'models': [{'modelName': 'squeezenet1_1',
       'modelUrl': 'https://torchserve.pytorch.org/mar_files/squeezenet1_1.mar'}]}

5.  **Set Default Model Version**: Ensure the desired version of a model
    is the default choice with the
    `ts_client.management.set_model_version()` method.

``` python
ts_client.management.set_default_version('squeezenet1_1', '1.0')
```

    {'status': 'Default vesion succsesfully updated for model "squeezenet1_1" to "1.0"'}

6.  **Unregister a Model**: If you need to bid farewell to a model, use
    the `ts_client.management.unregister_model()` function to gracefully
    remove it from TorchServe.

``` python
ts_client.management.unregister_model('squeezenet1_1')
```

    {'status': 'Model "squeezenet1_1" unregistered'}

7.  **API Description**: view a full list of Managment APIs.

``` python
ts_client.management.api_description()
```

Remember, all these management APIs can be accessed conveniently under
the namespace `ts_client.management`.

### Inference APIs

TorchServeClient allows you to interact with the Inference API, which
listens on port 8080, enabling you to run inference on your samples
effortlessly. Here are the available APIs under the
`ts_client.inference` namespace:

1.  **API Description**: Want to explore what APIs and options are
    available? Use `ts_client.inference.api_description()` to get a
    comprehensive list.

``` python
ts_client.inference.api_description()
```

    {'openapi': '3.0.1',
     'info': {'title': 'TorchServe APIs',
      'description': 'TorchServe is a flexible and easy to use tool for serving deep learning models',
      'version': '0.8.1'},
     'paths': {'/': {'options': {'description': 'Get openapi description.',
        'operationId': 'apiDescription',
        'parameters': [],
        'responses': {'200': {'description': 'A openapi 3.0.1 descriptor',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['openapi', 'info', 'paths'],
             'properties': {'openapi': {'type': 'string'},
              'info': {'type': 'object'},
              'paths': {'type': 'object'}}}}}},
         '500': {'description': 'Internal Server Error',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string',
               'description': 'Error message.'}}}}}}}}},
      '/ping': {'get': {'description': 'Get TorchServe status.',
        'operationId': 'ping',
        'parameters': [],
        'responses': {'200': {'description': 'TorchServe status',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['status'],
             'properties': {'status': {'type': 'string',
               'description': 'Overall status of the TorchServe.'}}}}}},
         '500': {'description': 'Internal Server Error',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string',
               'description': 'Error message.'}}}}}}}}},
      '/v1/models/{model_name}:predict': {'post': {'description': 'Predictions entry point to get inference using default model version.',
        'operationId': 'predictions',
        'parameters': [{'in': 'path',
          'name': 'model_name',
          'description': 'Name of model.',
          'required': True,
          'schema': {'type': 'string'}}],
        'requestBody': {'description': 'Input data format is defined by each model.',
         'content': {'*/*': {'schema': {'type': 'string', 'format': 'binary'}}},
         'required': True},
        'responses': {'200': {'description': 'Output data format is defined by each model.',
          'content': {'*/*': {'schema': {'type': 'string', 'format': 'binary'}}}},
         '404': {'description': 'Model not found or Model Version not found',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}},
         '500': {'description': 'Internal Server Error',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}},
         '503': {'description': 'No worker is available to serve request',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string',
               'description': 'Error message.'}}}}}}}}},
      '/v2/models/{model_name}/infer': {'post': {'description': 'Predictions entry point to get inference using default model version.',
        'operationId': 'predictions',
        'parameters': [{'in': 'path',
          'name': 'model_name',
          'description': 'Name of model.',
          'required': True,
          'schema': {'type': 'string'}}],
        'requestBody': {'description': 'Input data format is defined by each model.',
         'content': {'*/*': {'schema': {'type': 'string', 'format': 'binary'}}},
         'required': True},
        'responses': {'200': {'description': 'Output data format is defined by each model.',
          'content': {'*/*': {'schema': {'type': 'string', 'format': 'binary'}}}},
         '404': {'description': 'Model not found or Model Version not found',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}},
         '500': {'description': 'Internal Server Error',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}},
         '503': {'description': 'No worker is available to serve request',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string',
               'description': 'Error message.'}}}}}}}}},
      '/predictions/{model_name}': {'post': {'description': 'Predictions entry point to get inference using default model version.',
        'operationId': 'predictions',
        'parameters': [{'in': 'path',
          'name': 'model_name',
          'description': 'Name of model.',
          'required': True,
          'schema': {'type': 'string'}}],
        'requestBody': {'description': 'Input data format is defined by each model.',
         'content': {'*/*': {'schema': {'type': 'string', 'format': 'binary'}}},
         'required': True},
        'responses': {'200': {'description': 'Output data format is defined by each model.',
          'content': {'*/*': {'schema': {'type': 'string', 'format': 'binary'}}}},
         '404': {'description': 'Model not found or Model Version not found',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}},
         '500': {'description': 'Internal Server Error',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}},
         '503': {'description': 'No worker is available to serve request',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string',
               'description': 'Error message.'}}}}}}}}},
      '/predictions/{model_name}/{model_version}': {'post': {'description': 'Predictions entry point to get inference using specific model version.',
        'operationId': 'version_predictions',
        'parameters': [{'in': 'path',
          'name': 'model_name',
          'description': 'Name of model.',
          'required': True,
          'schema': {'type': 'string'}},
         {'in': 'path',
          'name': 'model_version',
          'description': 'Name of model version.',
          'required': True,
          'schema': {'type': 'string'}}],
        'requestBody': {'description': 'Input data format is defined by each model.',
         'content': {'*/*': {'schema': {'type': 'string', 'format': 'binary'}}},
         'required': True},
        'responses': {'200': {'description': 'Output data format is defined by each model.',
          'content': {'*/*': {'schema': {'type': 'string', 'format': 'binary'}}}},
         '404': {'description': 'Model not found or Model Version not found',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}},
         '500': {'description': 'Internal Server Error',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}},
         '503': {'description': 'No worker is available to serve request',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string',
               'description': 'Error message.'}}}}}}}}},
      '/api-description': {'get': {'description': 'Get openapi description.',
        'operationId': 'api-description',
        'parameters': [],
        'responses': {'200': {'description': 'A openapi 3.0.1 descriptor',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['openapi', 'info', 'paths'],
             'properties': {'openapi': {'type': 'string'},
              'info': {'type': 'object'},
              'paths': {'type': 'object'}}}}}},
         '500': {'description': 'Internal Server Error',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string', 'description': 'Error message.'}}}}}}},
        'deprecated': True}},
      '/metrics': {'get': {'description': 'Get TorchServe application metrics in prometheus format.',
        'operationId': 'metrics',
        'parameters': [{'in': 'query',
          'name': 'name[]',
          'description': 'Names of metrics to filter',
          'required': False,
          'schema': {'type': 'string'}}],
        'responses': {'200': {'description': 'TorchServe application metrics',
          'content': {'text/plain; version=0.0.4; charset=utf-8': {'schema': {'type': 'object',
             'required': ['# HELP', '# TYPE', 'metric'],
             'properties': {'# HELP': {'type': 'string',
               'description': 'Help text for TorchServe metric.'},
              '# TYPE': {'type': 'string',
               'description': 'Type of TorchServe metric.'},
              'metric': {'type': 'string',
               'description': 'TorchServe application metric.'}}}}}},
         '500': {'description': 'Internal Server Error',
          'content': {'application/json': {'schema': {'type': 'object',
             'required': ['code', 'type', 'message'],
             'properties': {'code': {'type': 'integer',
               'description': 'Error code.'},
              'type': {'type': 'string', 'description': 'Error type.'},
              'message': {'type': 'string',
               'description': 'Error message.'}}}}}}}}}}}

2.  **Health Check API**: Ensure the health of the running server with
    the `ts_client.inference.health_check()` method.

``` python
ts_client.inference.health_check()
```

    {'status': 'Healthy'}

3.  **Predictions API**: Get predictions from the served model using
    `ts_client.inference.predictions()`.

``` python
ts_client.inference.prediction('squeezenet1_1', data={'data': open('/Users/ankursingh/Downloads/kitten_small.jpg', 'rb')})
```

    {'lynx': 0.5455798506736755,
     'tabby': 0.2794159948825836,
     'Egyptian_cat': 0.10391879826784134,
     'tiger_cat': 0.06263326108455658,
     'leopard': 0.0050191376358270645}

4.  **Explanations API**: Dive into the served model’s explanations with
    ease using `ts_client.inference.explanations()`.

``` python
ts_client.inference.explaination('squeezenet1_1', data={'data': open('/Users/ankursingh/Downloads/kitten_small.jpg', 'rb')})
```

With these intuitive APIs at your disposal, you can harness the full
power of the Management and Inference API and take your application to
next level. Happy inferencing! 🚀🔥
