hf_types - Infernet ML
Description: Infernet ML Documentation

Skip to content

# Module: `hf_types`

## `HFClassificationInferenceInput`

Bases: `HFInferenceBaseInput`

Input data for classification models

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`text` | `str` | Text to classify | required

Source code in `src/infernet_ml/utils/hf_types.py`

    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    class HFClassificationInferenceInput(HFInferenceBaseInput):
        """Input data for classification models

        Args:
            text (str): Text to classify
        """

        task_id: HFTaskId = HFTaskId.TEXT_CLASSIFICATION

        text: str

## `HFDiffusionInferenceInput`

Bases: `HFInferenceBaseInput`

Input data for diffusion models

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`prompt` | `str` | Text prompt for image generation | required
`negative_prompt` | `Optional[str]` | Negative text prompt for the model | required
`height` | `Optional[int]` | Height in pixels of the image to generate. Default 512. | required
`width` | `Optional[int]` | Width in pixels of the image to generate. Default 512. | required
`num_inference_steps` | `Optional[int]` | Number of denoising steps. More steps --> higher quality but slower inference. | required
`guidance_scale` | `Optional[float]` | Guidance scale for the model to control the influence of the prompt on the generated image. Higher values --> more influence of the prompt on the generated image but may lead to lower image quality. Default values are model dependent but usually between 7 and 8. | required

Source code in `src/infernet_ml/utils/hf_types.py`

    162
    163
    164
    165
    166
    167
    168
    169
    170
    171
    172
    173
    174
    175
    176
    177
    178
    179
    180
    181
    182
    183
    184
    185
    186
    187
    class HFDiffusionInferenceInput(HFInferenceBaseInput):
        """Input data for diffusion models

        Args:
            prompt (str): Text prompt for image generation
            negative_prompt (Optional[str]): Negative text prompt for the model
            height (Optional[int]): Height in pixels of the image to generate.
                Default 512.
            width (Optional[int]): Width in pixels of the image to generate.
                Default 512.
            num_inference_steps (Optional[int]): Number of denoising steps.
                More steps --> higher quality but slower inference.
            guidance_scale (Optional[float]): Guidance scale for the model to
                control the influence of the prompt on the generated image.
                Higher values --> more influence of the prompt on the generated
                image but may lead to lower image quality. Default values are
                model dependent but usually between 7 and 8.
        """

        task_id: HFTaskId = HFTaskId.TEXT_TO_IMAGE
        prompt: str
        negative_prompt: Optional[str] = None
        height: Optional[int] = None
        width: Optional[int] = None
        num_inference_steps: Optional[int] = None
        guidance_scale: Optional[float] = None

## `HFInferenceBaseInput`

Bases: `BaseModel`

Base class for input data

Source code in `src/infernet_ml/utils/hf_types.py`

    28
    29
    30
    31
    32
    33
    34
    35
    class HFInferenceBaseInput(BaseModel):
        """Base class for input data"""

        model_config = ConfigDict(protected_namespaces=())

        model: Optional[str] = None

        task_id: HFTaskId

## `HFSummarizationConfig`

Bases: `TypedDict`

Summarization model configuration

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`model` | `str` | Model name | required
`max_length` | `int` | Maximum length in tokens of the generated summary | required
`min_length` | `int` | Minimum length in tokens of the generated summary | required
`top_k` | `int` | Number of top tokens to sample from | required
`top_p` | `float` | Cumulative probability for top-k sampling | required
`temperature` | `float` | Temperature for sampling. Default 1.0 | required
`repetition_penalty` | `float` | Repetition penalty for beam search | required
`num_return_sequences` | `int` | Number of sequences to return | required
`use_cache` | `bool` | Whether to use cache during inference | required

Source code in `src/infernet_ml/utils/hf_types.py`

    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    134
    135
    136
    137
    138
    class HFSummarizationConfig(TypedDict):
        """Summarization model configuration

        Args:
            model (str): Model name
            max_length (int): Maximum length in tokens of the generated summary
            min_length (int): Minimum length in tokens of the generated summary
            top_k (int): Number of top tokens to sample from
            top_p (float): Cumulative probability for top-k sampling
            temperature (float): Temperature for sampling. Default 1.0
            repetition_penalty (float): Repetition penalty for beam search
            num_return_sequences (int): Number of sequences to return
            use_cache (bool): Whether to use cache during inference
        """

        max_length: NotRequired[int]
        min_length: NotRequired[int]
        top_k: NotRequired[int]
        top_p: NotRequired[float]
        temperature: NotRequired[float]
        repetition_penalty: NotRequired[float]
        max_time: NotRequired[float]

## `HFSummarizationInferenceInput`

Bases: `HFInferenceBaseInput`

Input data for summarization models

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`text` | `str` | Text to summarize | required
`parameters` | `Optional[HFSummarizationConfig]` | Summarization model | required

Source code in `src/infernet_ml/utils/hf_types.py`

    141
    142
    143
    144
    145
    146
    147
    148
    149
    150
    151
    class HFSummarizationInferenceInput(HFInferenceBaseInput):
        """Input data for summarization models

        Args:
            text (str): Text to summarize
            parameters (Optional[HFSummarizationConfig]): Summarization model
        """

        task_id: HFTaskId = HFTaskId.SUMMARIZATION
        text: str
        parameters: Optional[HFSummarizationConfig] = None

## `HFTaskId`

Bases: `IntEnum`

Hugging Face task types

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`UNSET` | `int` | Unset task | required
`TEXT_GENERATION` | `int` | Text generation task | required
`TEXT_CLASSIFICATION` | `int` | Text classification task | required
`TOKEN_CLASSIFICATION` | `int` | Token classification task | required
`SUMMARIZATION` | `int` | Summarization task | required
`TEXT_TO_IMAGE` | `int` | Text to image task | required

Source code in `src/infernet_ml/utils/hf_types.py`

     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    class HFTaskId(IntEnum):
        """Hugging Face task types

        Args:
            UNSET (int): Unset task
            TEXT_GENERATION (int): Text generation task
            TEXT_CLASSIFICATION (int): Text classification task
            TOKEN_CLASSIFICATION (int): Token classification task
            SUMMARIZATION (int): Summarization task
            TEXT_TO_IMAGE (int): Text to image task
        """

        UNSET = 0
        TEXT_GENERATION = 1
        TEXT_CLASSIFICATION = 2
        TOKEN_CLASSIFICATION = 3
        SUMMARIZATION = 4
        TEXT_TO_IMAGE = 5

## `HFTextGenerationInferenceInput`

Bases: `HFInferenceBaseInput`

Input data for text generation models

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`prompt` | `str` | Prompt for text generation | required
`details` | `bool` | Whether to return detailed output (tokens, probabilities, seed, finish reason, etc.) | required
`stream` | `bool` | Whether to stream output. Only available for models running with the `text-generation-interface` backend. | required
`do_sample` | `bool` | Whether to use logits sampling | required
`max_new_tokens` | `int` | Maximum number of tokens to generate | required
`best_of` | `int` | Number of best sequences to generate and return with highest token logprobs | required
`repetition_penalty` | `float` | Repetition penalty for greedy decoding. 1.0 is no penalty | required
`return_full_text` | `bool` | Whether to preprend the prompt to the generated text | required
`seed` | `int` | Random seed for generation sampling | required
`stop_sequences` | `str` | Sequence to stop generation if a member of `stop_sequences` is generated | required
`temperature` | `float` | Sampling temperature for logits sampling | required
`top_k` | `int` | Number of highest probability vocabulary tokens to keep for top-k sampling | required
`top_p` | `float` | If <1, only the most probable tokens with probabilities that add up to `top_p` or higher are kept for top-p sampling | required
`truncate` | `int` | Truncate input to this length if set | required
`typical_p` | `float` | Typical decoding mass. | required
`watermark` | `bool` | Whether to add a watermark to the generated text Defaults to False. | required
`decoder_input_details` | `bool` | Whether to return the decoder input token logprobs and ids. Requires `details` to be set to True as well. Defaults to False. | required

Source code in `src/infernet_ml/utils/hf_types.py`

     62
     63
     64
     65
     66
     67
     68
     69
     70
     71
     72
     73
     74
     75
     76
     77
     78
     79
     80
     81
     82
     83
     84
     85
     86
     87
     88
     89
     90
     91
     92
     93
     94
     95
     96
     97
     98
     99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    class HFTextGenerationInferenceInput(HFInferenceBaseInput):
        """Input data for text generation models

        Args:
            prompt (str): Prompt for text generation
            details (bool): Whether to return detailed output (tokens, probabilities,
                seed, finish reason, etc.)
            stream (bool): Whether to stream output. Only available for models
                running with the `text-generation-interface` backend.
            do_sample (bool): Whether to use logits sampling
            max_new_tokens (int): Maximum number of tokens to generate
            best_of (int): Number of best sequences to generate and return
                with highest token logprobs
            repetition_penalty (float): Repetition penalty for greedy decoding.
                1.0 is no penalty
            return_full_text (bool): Whether to preprend the prompt to
                the generated text
            seed (int): Random seed for generation sampling
            stop_sequences (str): Sequence to stop generation if a member of
              `stop_sequences` is generated
            temperature (float): Sampling temperature for logits sampling
            top_k (int): Number of highest probability vocabulary tokens to keep for top-k
                sampling
            top_p (float): If <1, only the most probable tokens with probabilities that add
                up to `top_p` or higher are kept for top-p sampling
            truncate (int): Truncate input to this length if set
            typical_p (float): Typical decoding mass.
            watermark (bool): Whether to add a watermark to the generated text
                Defaults to False.
            decoder_input_details (bool): Whether to return the decoder input token
                logprobs and ids. Requires `details` to be set to True as well.
                Defaults to False.

        """

        task_id: HFTaskId = HFTaskId.TEXT_GENERATION
        prompt: str
        details: bool = Field(default=False)
        stream: bool = Field(default=False)
        do_sample: bool = Field(default=False)
        max_new_tokens: int = Field(default=20)
        best_of: Optional[int] = None
        repetition_penalty: Optional[float] = None
        return_full_text: bool = Field(default=False)
        seed: Optional[int] = None
        stop_sequences: Optional[str] = None
        temperature: Optional[float] = None
        top_k: Optional[int] = None
        top_p: Optional[float] = None
        truncate: Optional[int] = None
        typical_p: Optional[float] = None
        watermark: bool = Field(default=False)
        decoder_input_details: bool = Field(default=False)

## `HFTokenClassificationInferenceInput`

Bases: `HFInferenceBaseInput`

Input data for token classification models

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`text` | `str` | Text to classify | required

Source code in `src/infernet_ml/utils/hf_types.py`

    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    class HFTokenClassificationInferenceInput(HFInferenceBaseInput):
        """Input data for token classification models

        Args:
            text (str): Text to classify
        """

        task_id: HFTaskId = HFTaskId.TOKEN_CLASSIFICATION

        text: str

## `parse_hf_inference_input_from_dict(r)`

Parse input data from dictionary

Source code in `src/infernet_ml/utils/hf_types.py`

    190
    191
    192
    193
    194
    195
    196
    197
    198
    199
    200
    def parse_hf_inference_input_from_dict(r: Dict[str, Any]) -> HFInferenceClientInput:
        """Parse input data from dictionary"""
        if r["task_id"] == HFTaskId.TEXT_CLASSIFICATION:
            return HFClassificationInferenceInput(**r)
        if r["task_id"] == HFTaskId.TOKEN_CLASSIFICATION:
            return HFTokenClassificationInferenceInput(**r)
        if r["task_id"] == HFTaskId.TEXT_GENERATION:
            return HFTextGenerationInferenceInput(**r)
        if r["task_id"] == HFTaskId.SUMMARIZATION:
            return HFSummarizationInferenceInput(**r)
        raise ValueError(f"Unknown task_id: {r['task_id']}")