# Mobilint Model Zoo

<div align="center">
<p>
<a href="https://www.mobilint.com/" target="_blank">
<img src="https://raw.githubusercontent.com/mobilint/mblt-model-zoo/master/assets/Mobilint_Logo_Primary.png" alt="Mobilint Logo" width="60%">
</a>
</p>
</div>

**mblt-model-zoo** is a curated collection of AI models optimized by [Mobilint](https://www.mobilint.com/)’s Neural Processing Units (NPUs).

Designed to help developers accelerate deployment, Mobilint's Model Zoo offers access to public, pre-trained, and pre-quantized models for vision, language, and multimodal tasks. Along with performance results, we provide pre- and post-processing tools to help developers evaluate, fine-tune, and integrate the models with ease.

## Installation

[![PyPI - Version](https://img.shields.io/pypi/v/mblt-model-zoo?logo=pypi&logoColor=white)](https://pypi.org/project/mblt-model-zoo/)
[![PyPI Downloads](https://static.pepy.tech/badge/mblt-model-zoo?period=total&units=INTERNATIONAL_SYSTEM&left_color=BLACK&right_color=GREEN&left_text=downloads)](https://clickpy.clickhouse.com/dashboard/mblt-model-zoo)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mblt-model-zoo?logo=python&logoColor=gold)](https://pypi.org/project/mblt-model-zoo/)

- Prepare environment equipped with Mobilint's NPU. In case you are not a Mobilint customer, please contact [us](mailto:tech-support@mobilint.com).
- Install **mblt-model-zoo** using pip:

```bash
pip install mblt-model-zoo
```

- If you want to install the latest version from the source, clone the repository and install it:

```bash
git clone https://github.com/mobilint/mblt-model-zoo.git
cd mblt-model-zoo
pip install -e .
```

## Quick Start Guide

### Initializing Quantized Model Class

**mblt-model-zoo** provides a quantized model with associated pre- and post-processing tools. The following code snippet shows how to use the pre-trained model for inference.

```python
from mblt_model_zoo.vision import ResNet50

# Load the pre-trained model. 
# Automatically download the model if not found in the local cache.
resnet50 = ResNet50() 

# Load the model trained with a different recipe
# Currently, the default is "DEFAULT", or "IMAGENET1K_V1".
resnet50 = ResNet50(model_type = "IMAGENET1K_V2")

# Download the model to local directory and load it
resnet50 = ResNet50(local_path = "path/to/local/") # the file will be downloaded to "path/to/local/model.mxq"

# Load the model from a local path or download as filename and file path you want
resnet50 = ResNet50(local_path = "path/to/local/model.mxq")

# Set inference mode for better performance
# ARIES supports "single", "multi", "global4", and "global8" inference mode. Default is "global8"
resnet50 = ResNet50(infer_mode = "global8")

# (Beta) If you are holding a model compiled for REGULUS, enable inference on the REGULUS device.
resnet50 = ResNet50(product = "regulus")

# In summary, the model can be loaded with the following arguments. 
# You may customize those arguments to work with Mobilint's NPU.
resnet50 = ResNet50(
    local_path = None,
    model_type = "DEFAULT",
    infer_mode = "global8",
    product = "aries",
)

```

### Working with Quantized Model

With the image given as a path, PIL image, numpy array, or torch tensor, you can perform inference with the quantized model. The following code snippet shows how to use the quantized model for inference:

```python
image_path = "path/to/image.jpg"

input_img = resnet50.preprocess(image_path) # Preprocess the input image
output = resnet50(input_img) # Perform inference with the quantized model
result = resnet50.postprocess(output) # Postprocess the output

result.plot(
    source_path=image_path,
    save_path="path/to/save/result.jpg",
)
```

### Listing Available Models

**mblt-model-zoo** offers a function to list all available models. You can use the following code snippet to list the models for a specific task (e.g., image classification, object detection, etc.):

```python
from mblt_model_zoo.vision import list_models
from pprint import pprint

available_models = list_models()
pprint(available_models)
```

## Model List

We provide the models that are quantized with our advanced quantization techniques. A list of available vision models is [here](mblt_model_zoo/vision/README.md).

## Optional Extras

When working with tasks other than vision, extra dependencies may be required. Those options can be installed via `pip install mblt-model-zoo[NAME]` or `pip install -e .[NAME]`.

Currently, these optional functions are only available on environment equipped with Mobilint's [ARIES](https://www.mobilint.com/aries).

|Name|Use|Details|
|-------|------|------|
|transformers|For using HuggingFace transformers related models|[README.md](mblt_model_zoo/hf_transformers/README.md)|
|MeloTTS|For using MeloTTS models|[README.md](mblt_model_zoo/MeloTTS/README.md)|

For the `transformers` extra, the repository also includes:
- functional test instructions in [tests/transformers/TEST.md](tests/transformers/TEST.md)
- benchmark script usage in [benchmark/transformers/README.md](benchmark/transformers/README.md)

> Note: The `MeloTTS` extra includes `unidic`, which requires an additional dictionary download step. Python packaging (PEP 517/518) does not support running arbitrary post-install commands automatically, so run `mblt-unidic-download` (or `python -m unidic download`) after installing the extra when needed.

## Verbose Option

By default, model initialization stays quiet. To print the model file size and MD5 hash whenever an MXQ model loads, set the environment variable `MBLT_MODEL_ZOO_VERBOSE` to a truthy value before running your script:

```bash
export MBLT_MODEL_ZOO_VERBOSE=true  # accepted values: true/1/yes/on (case-insensitive)
python your_script.py
```

### Example Verbose Output

```bash
Model Initialized
Model Size: 216.94 MB
Model Hash: 23c262c43b4c1c453dd0326e249480a0
Device Number: 0
Core Mode: single
Target Cores: [CoreId(cluster=Cluster.Cluster0, core=Core.Core0)]
Model Variant 0
        Input Shape: [(1, 200, 96), (1, 200, 96), (2, 200, 200)]
        Output Shape: [(1, 102400, 1)]
Model Variant 1
        Input Shape: [(1, 300, 96), (1, 300, 96), (2, 300, 300)]
        Output Shape: [(1, 153600, 1)]
Model Variant 2
        Input Shape: [(1, 400, 96), (1, 400, 96), (2, 400, 400)]
        Output Shape: [(1, 204800, 1)]
Model Variant 3
        Input Shape: [(1, 500, 96), (1, 500, 96), (2, 500, 500)]
        Output Shape: [(1, 256000, 1)]
Model Variant 4
        Input Shape: [(1, 600, 96), (1, 600, 96), (2, 600, 600)]
        Output Shape: [(1, 307200, 1)]
Model Variant 5
        Input Shape: [(1, 900, 96), (1, 900, 96), (2, 900, 900)]
        Output Shape: [(1, 460800, 1)]
```

Unset or set the variable to any other value to suppress these messages.

## License

The Mobilint Model Zoo is released under BSD 3-Clause License. Please see the [LICENSE](https://github.com/mobilint/mblt-model-zoo/blob/master/LICENSE) file for more details.

Additionally, the license for each model provided in this package follows the terms specified in the source link provided with it.

## Support & Issues

If you encounter any problems with this package, please feel free to contact [us](mailto:tech-support@mobilint.com).
