# Minimal AI Agent (LiteLLM + LM Studio) Implementation Plan

## Goal
Build a minimal Python CLI agent that:
- Accepts a user question as a positional CLI argument.
- Uses LiteLLM to call a model through local LM Studio at `http://127.0.0.1:1234`.
- Automatically discovers the currently loaded LLM and uses it by default.
- Returns only the model response on stdout.
- Supports configurable max context size and max output size.

## Scope
- Single-turn request/response CLI.
- Local LM Studio only.
- No conversation memory, no tools, no streaming.
- Keep dependencies minimal.

## Proposed Files
- `agent.py`: CLI entrypoint and runtime logic.
- `pyproject.toml`: Project metadata and dependencies (use `uv` workflow).
- `README.md`: Setup and usage docs.

## CLI Contract
1. Positional input:
- `question`

2. Options:
- `--base-url` (default: `http://127.0.0.1:1234`)
- `--max-context-tokens` (optional; if set, requested context length for the selected loaded model)
- `--max-output-tokens` (default: `512`)
- `--temperature` (default: `0.2`)
- `--system-prompt` (optional string)
- `--no-system-prompt` (omit system message entirely)
- `--model` (optional override; bypass auto-discovered loaded model)
- `-v, --verbose` (diagnostics to stderr)

3. Validation rules:
- `--system-prompt` and `--no-system-prompt` are mutually exclusive (`argparse` mutually exclusive group).
- If both `--max-context-tokens` and `--max-output-tokens` are provided, require `max_output_tokens <= max_context_tokens`.

## Implementation Steps
1. Create project/dependency setup with `uv`.
- Add `pyproject.toml` with `litellm` dependency only.
- Add `[project.scripts]` entry: `agent = "agent:main"`.
- Use stdlib `urllib.request` for REST calls (no `requests` dependency).

2. Implement loaded-model discovery.
- Query native LM Studio endpoint `GET {base_url}/api/v1/models` (not OpenAI-compatible `/v1/models`) to access loaded-instance metadata.
- Select first `type=="llm"` entry with a non-empty `loaded_instances`.
- Use `loaded_instances[0].id` as the default inference model identifier.
- If `--model` is set, use it instead of auto-discovered model.
- If no loaded LLM is found and no override is provided, fail with a clear error.

3. Implement optional context-size configuration.
- If `--max-context-tokens` is provided:
  - First inspect current loaded instance `config.context_length` from `/api/v1/models`.
  - Only call `POST {base_url}/api/v1/models/load` when requested context differs from current context.
  - Request body: `{ "model": <model key>, "context_length": <value> }`.
- Document and surface that `/models/load` can trigger a model reload and may take noticeable time.
- Do not do client-side token truncation/estimation in v1.

4. Wire LiteLLM chat completion.
- Call LiteLLM with OpenAI-compatible endpoint:
  - `model=f"openai/{model_id}"`
  - `api_base=f"{base_url}/v1"`
  - `api_key="lm-studio"`
- Build `messages`:
  - Include system message only when `--system-prompt` is provided and `--no-system-prompt` is not set.
  - Always append user `question`.
- Set `max_tokens=max_output_tokens`.

5. Output/error behavior.
- Print assistant response text to stdout only.
- Print diagnostics (selected model, context reload details, debug info) to stderr only, gated by `--verbose`.
- Return non-zero exit code on failure.

## Documentation Plan
1. `uv` setup:
- `uv sync`

2. Example runs:
- `uv run agent "What is the capital of France?"`
- `uv run agent "Explain TLS in 5 bullets" --max-output-tokens 256`
- `uv run agent "Summarize this in one sentence" --system-prompt "Be concise."`
- `uv run agent "Hello" --no-system-prompt -v`
- `uv run agent "Hi" --model my-loaded-model-id`
- `uv run agent "Long-context test" --max-context-tokens 8192` (note: may reload model if current context differs)

## Verification Plan
1. Positive path:
- Start LM Studio server, load an LLM, run:
  - `uv run agent "Say HELLO in one line."`
- Confirm:
  - Loaded model is auto-discovered.
  - Response text is printed to stdout.
  - `--max-output-tokens` affects output length tendency.

2. Context configuration path:
- Run with `--max-context-tokens` and verify request succeeds and model responds.
- Run with `--max-context-tokens 256 --max-output-tokens 512` and verify validation error.
- Run with `--max-context-tokens` equal to current context and verify no reload call is made.

3. Negative paths:
- LM Studio not running: clear connection error + non-zero exit.
- No loaded model and no `--model` override: clear error + non-zero exit.

4. Output channel correctness:
- Verify `uv run agent "question" | some_consumer` receives only model text (no metadata).

## Acceptance Criteria
- CLI accepts positional question and returns a model response.
- Default model selection comes from currently loaded LLM in LM Studio.
- `--max-context-tokens` and `--max-output-tokens` are configurable.
- System prompt is configurable and can be omitted.
- `--system-prompt` and `--no-system-prompt` conflict is rejected clearly.
- Diagnostics never pollute stdout.
- `uv`-based setup is documented and works.
