Metadata-Version: 2.4
Name: agent-status-sdk
Version: 1.0.0
Summary: Agent Status SDK - Outside-in monitoring for AI agents
Project-URL: Homepage, https://agentstatus.dev
Project-URL: Dashboard, https://platform.agentstatus.dev
Project-URL: Repository, https://github.com/Carmel-Labs-Inc/agent-status-sdk
Author-email: Carmel Labs <dev@carmel.so>
License-Expression: MIT
License-File: LICENSE
Keywords: agent-status,agents,ai,llm,monitoring,validation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.8
Requires-Dist: httpx>=0.24.0
Requires-Dist: pydantic>=2.0.0
Provides-Extra: all
Requires-Dist: anthropic>=0.18.0; extra == 'all'
Requires-Dist: crewai>=0.1.0; extra == 'all'
Requires-Dist: langchain-core>=0.1.0; extra == 'all'
Requires-Dist: mcp>=0.1.0; extra == 'all'
Requires-Dist: openai>=1.0.0; extra == 'all'
Requires-Dist: pyautogen>=0.2.0; extra == 'all'
Requires-Dist: websockets>=11.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.18.0; extra == 'anthropic'
Provides-Extra: autogen
Requires-Dist: pyautogen>=0.2.0; extra == 'autogen'
Requires-Dist: websockets>=11.0; extra == 'autogen'
Provides-Extra: crewai
Requires-Dist: crewai>=0.1.0; extra == 'crewai'
Requires-Dist: websockets>=11.0; extra == 'crewai'
Provides-Extra: dev
Requires-Dist: langchain-core>=0.1.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Requires-Dist: websockets>=11.0; extra == 'dev'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.1.0; extra == 'langchain'
Requires-Dist: websockets>=11.0; extra == 'langchain'
Provides-Extra: mcp
Requires-Dist: mcp>=0.1.0; extra == 'mcp'
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == 'openai'
Description-Content-Type: text/markdown

# Agent Status SDK

Outside-in monitoring for AI agents. Monitor any HTTP-accessible AI agent from distributed residential endpoints worldwide.

## Installation

```bash
pip install agent-status-sdk

# With LangChain integration
pip install agent-status-sdk[langchain]

# All integrations
pip install agent-status-sdk[all]
```

## Quick Start

```python
import agent_status

# Initialize with your API key
agent_status.init(api_key="rora_xxx")

# Register an agent for continuous monitoring
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Support Bot",
    interval_minutes=60,  # Check every hour
)

print(f"Registered: {agent.id}")

# Check current status
status = agent_status.status(agent.id)
print(f"Verdict: {status.verdict}")  # UP, DEGRADED, DOWN
print(f"Uptime: {status.uptime_24h}%")
print(f"Latency: {status.latency_p95}ms")
```

## One-Off Validation

Run a quick validation without registering for continuous monitoring:

```python
result = agent_status.run(
    endpoint="https://api.example.com/chat",
    prompts=["What is 2+2?", "Hello!"],
)

print(f"Verdict: {result.verdict}")
print(f"P95 Latency: {result.latency_p95}ms")
print(f"Pass Rate: {result.pass_rate}")
```

## Authentication

For agents requiring authentication:

```python
# Bearer token
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Private Bot",
    auth={"type": "bearer", "token": "sk-xxx"},
)

# API key in header
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="API Bot",
    auth={"type": "api_key", "header": "X-API-Key", "value": "xxx"},
)
```

## Advanced Options

```python
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Enterprise Bot",

    # Probing configuration
    interval_minutes=60,       # How often to probe
    max_nodes_per_run=10,      # Distributed nodes per check
    geos=["us", "eu", "ap"],   # Geographic regions
    timeout_ms=30000,          # Request timeout

    # Validation options
    eval_type="llm_judge",     # "basic", "llm_judge", or "all"
    gold_prompt_profile="search_agent",  # Specialized prompts
    inject_geo_context=True,   # Add location to prompts

    # Response handling
    streaming=True,            # SSE streaming responses
)
```

## LangChain Integration

### Expose for Outside-In Monitoring (Recommended)

The easiest way to monitor a LangChain agent - no infrastructure needed:

```python
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from agent_status.integrations.langchain import expose

# Build your chain
prompt = ChatPromptTemplate.from_template("You are a helpful assistant. {input}")
llm = ChatOpenAI()
chain = prompt | llm

# Expose to the distributed monitoring network - one line!
expose(chain, api_key="fb_live_xxx", agent_name="My Support Bot")

# That's it! Nodes around the world will now probe your agent.
# Output:
# ============================================================
#   Agent Status - Agent Exposed!
#   Name: My Support Bot
#   URL:  https://rora-tunnel.carmel.so/probe/agent-xxx
#   Press Ctrl+C to stop
# ============================================================
```

This works even for agents running locally on your laptop - no need to deploy anywhere!

### Non-Blocking Mode

Run in background while your app does other things:

```python
url = expose(
    chain,
    api_key="fb_live_xxx",
    agent_name="Background Bot",
    blocking=False,  # Returns immediately
)
print(f"Agent exposed at: {url}")

# Your app continues running...
```

### Local Callback Handler

For tracking metrics locally without outside-in monitoring:

```python
from langchain_openai import ChatOpenAI
from agent_status.integrations.langchain import AgentStatusCallbackHandler

handler = AgentStatusCallbackHandler(
    api_key="fb_live_xxx",
    agent_name="My LangChain Agent",
)

llm = ChatOpenAI(callbacks=[handler])
result = llm.invoke("Hello!")

print(handler.metrics)
# {'total_calls': 1, 'llm_calls': 1, 'total_latency_ms': 523, ...}
```

### Tracked Metrics (Callback Handler)

The callback handler tracks:
- LLM calls (latency, tokens, model)
- Chain executions
- Tool/function calls
- Errors and exceptions

## CLI Usage

```bash
# Set your API key
export RORA_API_KEY=rora_xxx

# Check agent status
agent-status status <agent_id>

# Run one-off validation
agent-status run https://api.example.com/chat --prompts "Hello,How are you?"

# List all agents
agent-status list

# Register a new agent
agent-status register https://api.mycompany.com/chat --name "My Bot"

# Delete an agent
agent-status delete <agent_id>
```

## Gold Prompt Profiles

Agent Status uses "gold prompts" - carefully crafted test prompts for different agent types:

| Profile | Description |
|---------|-------------|
| `general` | Generic conversational prompts |
| `search_agent` | Web search and information retrieval |
| `code_generator` | Code generation and debugging |
| `data_retriever` | Database and API queries |
| `customer_support` | Support and FAQ handling |
| `creative_writer` | Content generation |

## Evaluation Types

| Type | Description |
|------|-------------|
| `basic` | Response format and latency checks |
| `llm_judge` | GPT-4 evaluates response quality |
| `all` | Both basic and LLM evaluation |

## Response Models

### Agent
```python
agent.id              # UUID
agent.name            # Display name
agent.endpoint_url    # HTTP endpoint
agent.status          # active, paused, deleted
agent.last_status     # UP, DEGRADED, DOWN
```

### AgentStatus
```python
status.verdict       # UP, DEGRADED, DOWN, UNKNOWN
status.uptime_24h    # 24-hour uptime percentage
status.uptime_7d     # 7-day uptime percentage
status.latency_p50   # P50 latency (ms)
status.latency_p95   # P95 latency (ms)
status.pass_rate     # Pass rate (0-1)
status.total_checks  # Total probes run
```

### RunResult
```python
result.verdict          # UP, DEGRADED, DOWN
result.latency_p50      # P50 latency (ms)
result.latency_p95      # P95 latency (ms)
result.pass_rate        # Pass rate (0-1)
result.total_probes     # Probes sent
result.successful_probes  # Successful probes
result.by_region        # Per-region breakdown
result.judge_result     # LLM evaluation (if enabled)
```

## Error Handling

```python
from agent_status.client import AgentStatusError, AgentStatusAuthError, AgentStatusNotFoundError

try:
    status = agent_status.status("invalid-id")
except AgentStatusNotFoundError:
    print("Agent not found")
except AgentStatusAuthError:
    print("Invalid API key")
except AgentStatusError as e:
    print(f"Error: {e}")
```

## Environment Variables

| Variable | Description |
|----------|-------------|
| `RORA_API_KEY` | Your API key (required for CLI) |
| `RORA_BASE_URL` | API base URL (optional, for testing) |

> **Note:** Environment variables retain the `RORA_` prefix for backward compatibility.

## Links

- [Homepage](https://agentstatus.dev)
- [Dashboard](https://platform.agentstatus.dev)
- [GitHub](https://github.com/Carmel-Labs-Inc/rora-sdk)

## License

MIT
