Metadata-Version: 2.1
Name: ollama-proxy
Version: 0.2.1
Summary: ollama proxy to other cloud model service
License: Apache 2.0
Author: lipi
Author-email: lipicoder@qq.com
Requires-Python: >=3.12,<4.0
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: aiohttp (>=3.10.8,<4.0.0)
Requires-Dist: asyncio (>=3.4.3,<4.0.0)
Requires-Dist: click (>=8.1.7,<9.0.0)
Requires-Dist: fastapi (>=0.115.0,<0.116.0)
Requires-Dist: httpx (>=0.27.2,<0.28.0)
Requires-Dist: hypercorn (>=0.17.3,<0.18.0)
Requires-Dist: pydantic (>=2.9.2,<3.0.0)
Requires-Dist: pytest (>=8.3.3,<9.0.0)
Requires-Dist: setuptools (>=75.1.0,<76.0.0)
Requires-Dist: toml (>=0.10.2,<0.11.0)
Requires-Dist: uvicorn (>=0.31.0,<0.32.0)
Requires-Dist: watchfiles (>=0.24.0,<0.25.0)
Requires-Dist: zhipuai (>=2.1.5.20230904,<3.0.0.0)
Description-Content-Type: text/markdown

# ollama-proxy

**ollama-proxy** is a service that adheres to the Ollama external interface protocol. It enables connections to various large model services, such as **glm-4-plus** and **deepseek-coder-v2**, through the Ollama interface protocol. The primary goal of this project is to facilitate integration with **zed.dev**, allowing it to utilize additional large model services beyond the official configuration.

## Installation

To install **ollama-proxy**, a command-line tool, you can use the following command:

```bash
pip install ollama-proxy
```

## Command Line Usage

The `ollama-proxy` command-line interface (CLI) allows you to run the proxy server with specific configurations. Below are the details on how to use the CLI, including available options and their descriptions.

## Command Structure

To run the `ollama-proxy`, use the following command structure:

```bash
ollama-proxy <model_name> --config <config_path> --host <host> --port <port>
```

### Arguments

- **`model_name`**: 
  - **Type**: String
  - **Description**: A required argument that specifies the name of the model you want to run.

- **`config`**: 
  - **Type**: String
  - **Default**: `keys.toml`
  - **Description**: The path to the TOML configuration file.

### Options

- **`--host`**: 
  - **Type**: String
  - **Default**: `127.0.0.1`
  - **Description**: The host address for the server. For **zed.dev**, only `localhost` is allowed.

- **`--port`**: 
  - **Type**: Integer
  - **Default**: `11434`
  - **Description**: For **zed.dev**, the port number must be `11434`.

## Example Usage

To start the `ollama-proxy` with a specific model and configuration, you can run:

```bash
ollama-proxy glm-4-plus --config path/to/config.toml --host localhost --port 11434 --reload
```

In this example:
- `glm-4-plus` is the name of the model you want to run.
- The configuration file is located at `path/to/config.toml`.
- The server will listen on `localhost` at port `11434`.
- Hot reloading is enabled.

## Notes

- Ensure that the specified configuration file exists and is correctly formatted in TOML.
- The server will print logs to the console, indicating its status and any errors that may occur.

## Configuration TOML File

To configure **ollama-proxy**, create a configuration file in TOML format. This file allows you to specify various settings for the proxy.

### Example Configuration (config.toml)

```toml
[glm-4-plus]
provider = "zhipu"
url = "https://open.bigmodel.cn/api/paas/v4/chat/completions"
api_key = "your_api_key_here"

[deepseek-coder-v2]
provider = "deepseek"
url = "https://api.deepseek.com/chat/completions"
api_key = "your_api_key_here"
```

- The section name corresponds to the model name, such as `glm-4-plus` or `deepseek-coder-v2`.
- The `provider` specifies the name of the large model service provider.
- The `url` is the endpoint of the large model service provider.
- The `api_key` is the API key for the large model service provider.

When you run `ollama-proxy`, it will load the configuration file. For example, running `ollama-proxy glm-4-plus` will use `glm-4-plus` as the running model name. If `glm-4-plus` is not specified in the configuration file, an error will be raised.

## TODO

- [ ] Support more large model providers
- [ ] Add xinference as a local model provider
- [ ] Support cursor
