Metadata-Version: 2.2
Name: openvino-openai-api
Version: 0.1.1
Summary: OpenAI Compatible API Server using OpenVINO GenAI
Author-email: Fabio Matricardi <fabio.matricardi@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/fabiomatricardi/openvino-openai-api
Project-URL: Bug Tracker, https://github.com/fabiomatricardi/openvino-openai-api/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: about-time==4.2.1
Requires-Dist: aiofiles==23.2.1
Requires-Dist: aiohappyeyeballs==2.4.6
Requires-Dist: aiohttp==3.11.13
Requires-Dist: aiosignal==1.3.2
Requires-Dist: alive-progress==3.2.0
Requires-Dist: annotated-types==0.7.0
Requires-Dist: anyio==4.8.0
Requires-Dist: attrs==25.1.0
Requires-Dist: autograd==1.7.0
Requires-Dist: certifi==2025.1.31
Requires-Dist: charset-normalizer==3.4.1
Requires-Dist: click==8.1.8
Requires-Dist: cma==3.2.2
Requires-Dist: colorama==0.4.6
Requires-Dist: contourpy==1.3.1
Requires-Dist: cycler==0.12.1
Requires-Dist: datasets==3.3.2
Requires-Dist: Deprecated==1.2.18
Requires-Dist: dill==0.3.8
Requires-Dist: distro==1.9.0
Requires-Dist: evaluate==0.4.3
Requires-Dist: fastapi==0.115.10
Requires-Dist: ffmpy==0.5.0
Requires-Dist: filelock==3.17.0
Requires-Dist: fonttools==4.56.0
Requires-Dist: frozenlist==1.5.0
Requires-Dist: fsspec==2024.12.0
Requires-Dist: gradio==5.20.0
Requires-Dist: gradio_client==1.7.2
Requires-Dist: grapheme==0.6.0
Requires-Dist: groovy==0.1.2
Requires-Dist: h11==0.14.0
Requires-Dist: httpcore==1.0.7
Requires-Dist: httpx==0.28.1
Requires-Dist: huggingface-hub==0.29.1
Requires-Dist: idna==3.10
Requires-Dist: Jinja2==3.1.5
Requires-Dist: jiter==0.8.2
Requires-Dist: joblib==1.4.2
Requires-Dist: jsonschema==4.23.0
Requires-Dist: jsonschema-specifications==2024.10.1
Requires-Dist: jstyleson==0.0.2
Requires-Dist: kiwisolver==1.4.8
Requires-Dist: markdown-it-py==3.0.0
Requires-Dist: MarkupSafe==2.1.5
Requires-Dist: matplotlib==3.10.1
Requires-Dist: mdurl==0.1.2
Requires-Dist: mpmath==1.3.0
Requires-Dist: multidict==6.1.0
Requires-Dist: multiprocess==0.70.16
Requires-Dist: natsort==8.4.0
Requires-Dist: networkx==3.3
Requires-Dist: ninja==1.11.1.3
Requires-Dist: nncf==2.15.0
Requires-Dist: numpy==2.1.3
Requires-Dist: onnx==1.17.0
Requires-Dist: openai==1.65.1
Requires-Dist: openvino==2025.0.0
Requires-Dist: openvino-genai==2025.0.0.0
Requires-Dist: openvino-telemetry==2025.0.1
Requires-Dist: openvino-tokenizers==2025.0.0.0
Requires-Dist: optimum==1.24.0
Requires-Dist: optimum-intel==1.22.0
Requires-Dist: orjson==3.10.15
Requires-Dist: packaging==24.2
Requires-Dist: pandas==2.2.3
Requires-Dist: pillow==11.1.0
Requires-Dist: propcache==0.3.0
Requires-Dist: protobuf==5.29.3
Requires-Dist: psutil==7.0.0
Requires-Dist: py-cpuinfo==9.0.0
Requires-Dist: pyarrow==19.0.1
Requires-Dist: pydantic==2.10.6
Requires-Dist: pydantic_core==2.27.2
Requires-Dist: pydot==2.0.0
Requires-Dist: pydub==0.25.1
Requires-Dist: Pygments==2.19.1
Requires-Dist: pymoo==0.6.1.3
Requires-Dist: pyparsing==3.2.1
Requires-Dist: python-dateutil==2.9.0.post0
Requires-Dist: python-multipart==0.0.20
Requires-Dist: pytz==2025.1
Requires-Dist: PyYAML==6.0.2
Requires-Dist: referencing==0.36.2
Requires-Dist: regex==2024.11.6
Requires-Dist: requests==2.32.3
Requires-Dist: rich==13.9.4
Requires-Dist: rpds-py==0.23.1
Requires-Dist: ruff==0.9.9
Requires-Dist: safehttpx==0.1.6
Requires-Dist: safetensors==0.5.3
Requires-Dist: scikit-learn==1.6.1
Requires-Dist: scipy==1.15.2
Requires-Dist: semantic-version==2.10.0
Requires-Dist: sentencepiece==0.2.0
Requires-Dist: shellingham==1.5.4
Requires-Dist: six==1.17.0
Requires-Dist: sniffio==1.3.1
Requires-Dist: starlette==0.46.0
Requires-Dist: sympy==1.13.1
Requires-Dist: tabulate==0.9.0
Requires-Dist: threadpoolctl==3.5.0
Requires-Dist: tiktoken==0.9.0
Requires-Dist: tokenizers==0.21.0
Requires-Dist: tomlkit==0.13.2
Requires-Dist: torch==2.6.0
Requires-Dist: tqdm==4.67.1
Requires-Dist: transformers==4.48.3
Requires-Dist: typer==0.15.2
Requires-Dist: typing_extensions==4.12.2
Requires-Dist: tzdata==2025.1
Requires-Dist: urllib3==2.3.0
Requires-Dist: uvicorn==0.34.0
Requires-Dist: websockets==15.0
Requires-Dist: wrapt==1.17.2
Requires-Dist: xxhash==3.5.0
Requires-Dist: yarl==1.18.3
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"

# OpenVINO OpenAI API

An OpenAI-compatible API server powered by OpenVINO GenAI for efficient inference on Intel hardware.

## Features

- OpenAI API compatibility for easy integration with existing applications
- Powered by OpenVINO for optimized inference on Intel CPUs and GPUs
- Support for both streaming and non-streaming responses
- Simple command-line interface for launching the server

## Installation

```bash
pip install openvino-openai-api
```

## Requirements

- Python 3.11 (due to dependency issues, only python 3.11 is supported)
- OpenVINO GenAI
- FastAPI
- Uvicorn

## Usage

### Starting the server

```bash
# Launch with default settings
openvino-openai-server --model-path /path/to/your/model

# Custom configuration
openvino-openai-server --model-path /path/to/your/model --device CPU --host 0.0.0.0 --port 8000
```

### Sending requests

The API is compatible with OpenAI's chat completions endpoint:

```python
import requests
import json

url = "http://localhost:8000/v1/chat/completions"
headers = {"Content-Type": "application/json"}
data = {
    "model": "local-model",
    "messages": [
        {"role": "user", "content": "Hello, how are you?"}
    ],
    "max_tokens": 500
}

response = requests.post(url, headers=headers, data=json.dumps(data))
print(response.json())
```

### Streaming responses

For streaming responses, set `stream=True` in your request and handle the server-sent events:

```python
import requests
import json

url = "http://localhost:8000/v1/chat/completions"
headers = {"Content-Type": "application/json"}
data = {
    "model": "local-model",
    "messages": [
        {"role": "user", "content": "Tell me a story"}
    ],
    "max_tokens": 500,
    "stream": True
}

response = requests.post(url, headers=headers, data=json.dumps(data), stream=True)
for line in response.iter_lines():
    if line:
        line = line.decode('utf-8')
        if line.startswith('data: ') and not line.endswith('[DONE]'):
            json_str = line[6:]  # Remove 'data: ' prefix
            try:
                chunk = json.loads(json_str)
                content = chunk['choices'][0]['delta'].get('content', '')
                if content:
                    print(content, end='', flush=True)
            except json.JSONDecodeError:
                pass
print()
```

## Model Requirements

The model directory should contain the following files:
- `openvino_model.bin`
- `openvino_tokenizer.bin`
- `openvino_detokenizer.bin`
- `tokenizer_config.json` with a valid `chat_template` defined

## Development

### Setup development environment

```bash
git clone https://github.com/yourusername/openvino-openai-api.git
cd openvino-openai-api
pip install -e ".[dev]"
```

### Running tests

```bash
pytest
```

## License

This project is licensed under the terms of the MIT license.
