ml_type - Infernet ML
Description: Infernet ML Documentation

Skip to content

# Module: `ml_type`

## `MLType`

Bases: `StrEnum`

supported by infernet.

Attributes:

Name | Type | Description
--- | --- | ---
`CSS` |  | Closed Source Models: i.e. OpenAI's GPT-4, etc.
`ONNX` |  | Models with ONNX format
`TORCH` |  | PyTorch models
`LLAMA_CPP` |  | LLM models with llama.cpp backend
`TGI_CLIENT` |  | Services that act as a client to a TGI server
`HF_CLIENT` |  | Services that can run huggignface's inference client library & all of its related tasks

Source code in `src/infernet_ml/utils/specs/ml_type.py`

     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    class MLType(StrEnum):
        """
        MLType: Enum for the different types of machine learning models that can be
        supported by infernet.

        Attributes:
            CSS: Closed Source Models: i.e. OpenAI's GPT-4, etc.
            ONNX: Models with ONNX format
            TORCH: PyTorch models
            LLAMA_CPP: LLM models with llama.cpp backend
            TGI_CLIENT: Services that act as a client to a TGI server
            HF_CLIENT: Services that can run huggignface's inference client
                library & all of its related tasks
        """

        CSS = "css"
        ONNX = "onnx"
        TORCH = "torch"
        LLAMA_CPP = "llama_cpp"
        TGI_CLIENT = "tgi_client"
        HF_CLIENT = "hf_client"