Providers — Swival
Description: Providers — Swival documentation

# Providers

Swival supports three providers: LM Studio (local), HuggingFace Inference API (hosted), and OpenRouter (multi-provider API). Under the hood, all LLM calls go through LiteLLM, which normalizes the API differences.

## LM Studio

This is the default and requires no configuration beyond having LM Studio running with a model loaded.

### Auto-discovery

Swival queries `http://127.0.0.1:1234/api/v1/models` at startup to find the loaded model. It looks for the first entry with `type: "llm"` and a non-empty `loaded_instances` array. The model identifier and context length are extracted automatically.

If no model is loaded, Swival exits with an error telling you to load one.

### Custom base URL

If LM Studio is running on a different host or port:

    swival --base-url http://192.168.1.100:1234 "task"

### Manual model selection

If auto-discovery doesn't find the right model (e.g., multiple models loaded), you can specify it:

    swival --model "qwen3-coder-next" "task"

### Context size configuration

You can request a specific context length, which may trigger LM Studio to reload the model:

    swival --max-context-tokens 131072 "task"

Swival calls LM Studio's `/api/v1/models/load` endpoint with the new context size. If the requested size matches what's already loaded, no reload happens. Reloads can be slow depending on the model and hardware.

### How the LiteLLM call works

For LM Studio, Swival prefixes the model identifier with `openai/` and sets `api_base` to `{base_url}/v1`. The API key is set to `"lm-studio"` (LM Studio doesn't require a real key). This tells LiteLLM to use the OpenAI-compatible API format.

## HuggingFace Inference API

For hosted inference without running a local model.

### Basic usage

    export HF_TOKEN=hf_your_token_here
    swival --provider huggingface --model zai-org/GLM-5 "task"

The `--model` flag is required and must be in `org/model` format. Authentication comes from `HF_TOKEN` in the environment or `--api-key` on the command line (which takes precedence).

Serverless endpoints typically have a 32k token context limit, which can be restrictive for agentic workloads that accumulate tool calls and file contents over many turns. If you're hitting context limits, consider a dedicated endpoint instead.

### Dedicated endpoints

For HuggingFace dedicated inference endpoints (private deployments):

    swival --provider huggingface \
        --model zai-org/GLM-5 \
        --base-url https://xyz.endpoints.huggingface.cloud \
        --api-key hf_your_key \
        "task"

The `--base-url` points to your endpoint. The model identifier still needs to match what's deployed there. Dedicated endpoints don't have the context size limits of serverless -- you control the deployment, so the full model context window is available.

### How the LiteLLM call works

For HuggingFace, Swival prefixes the model with `huggingface/` (stripping any existing prefix first) and passes the API key directly. If `--base-url` is set, it's passed as `api_base` to LiteLLM.

## OpenRouter

For access to dozens of models from different providers (OpenAI, Anthropic, Meta, Google, and others) through a single API.

### Basic usage

    export OPENROUTER_API_KEY=sk_or_your_token_here
    swival --provider openrouter --model openrouter/free "task"

The `--model` flag is required. Use the model identifier from OpenRouter's catalog (e.g., `openrouter/free`, `anthropic/claude-3-5-sonnet`, `openrouter/free`). Authentication comes from `OPENROUTER_API_KEY` in the environment or `--api-key` on the command line (which takes precedence).

### Custom base URL

For custom OpenRouter-compatible endpoints:

    swival --provider openrouter \
        --model openrouter/free \
        --base-url https://custom.openrouter.endpoint \
        --api-key sk_or_key \
        "task"

### Context size

OpenRouter models vary widely in context size (8K to 200K+ tokens). Swival defaults to whatever `--max-context-tokens` is set to; use it to match your model's actual limit:

    swival --provider openrouter --model openrouter/free \
        --max-context-tokens 131072 "task"

### How the LiteLLM call works

For OpenRouter, Swival prefixes the model with `openrouter/` (stripping any existing prefix first) and passes the API key directly. If `--base-url` is set, it's passed as `api_base` to LiteLLM.

## Future providers

Since Swival uses LiteLLM for the actual API call, adding new providers is straightforward -- it's mostly a matter of building the right model string and passing the right credentials. The provider-specific logic in `call_llm()` is about 10 lines per provider.

OpenRouter already provides access to dozens of models from different providers through a single API, so it's a good option if you need flexibility without configuring multiple credentials.