# SDK Enhancements

## Decorator API

Add `@bandito.traced` as an alternative to the context manager. Auto-captures function input/output and dispatches to mappers.

```python
@bandito.traced("my-project")
def run_agent(query):
    return agent.run_sync(query)

result = run_agent("What's the weather?")
```

Equivalent to:

```python
with bandito.trace("my-project", input="What's the weather?") as t:
    result = agent.run_sync("What's the weather?")
    t.done(result)
```

The decorator should:
- Capture function args as trace input
- Pass the return value through the mapper dispatch
- Support `storage=False`
- Return the original return value (transparent to the caller)

## Offline Mode — External Observability Connectors

Two usage modes for Bandito:

### Online mode (current)

User integrates our SDK at runtime. `bandito.trace()` captures traces, mappers auto-extract spans from frameworks. User grades through Bandito. We optimize.

### Offline mode

User already has an observability platform (Langfuse, Braintrust, etc.) and doesn't want our SDK in their runtime. Bandito connects to their platform as an offline grading and analysis tool.

**How it works:**

- Event-driven connector syncs traces from the external platform (e.g. Langfuse API)
- Scheduled sync — pulls new traces periodically, tracks what's been graded vs not
- User grades through Bandito's grading tools against the synced traces
- Evaluations live in Bandito, full trace data stays in the external platform
- The external platform's trace ID is the bridge between systems

**Langfuse connector (first target):**

- Langfuse exposes a REST API for listing/fetching traces
- Connector pulls traces for a given project, stores minimal metadata (trace ID, input, output, timestamp) for grading
- Evaluations reference the Langfuse trace ID — user can click through to full span detail in Langfuse
- Sync tracks a cursor/timestamp to only fetch new traces on each run

**Product boundary:**

- Langfuse (or similar) owns trace storage and observability
- Bandito owns evaluations, grading workflow, and optimization
- No runtime SDK needed — Bandito is purely an offline tool in this mode

## Code Scoring (SDK)

Code evaluators that run at execution time as part of the SDK. Deterministic checks on trace output — the unit test equivalent for LLM applications.

**Terminology:**

- **Score** — code evaluator result, runs in SDK at runtime
- **Grade** — human or LLM-as-judge result, binary 0/1 with optional comment and span selection. Lives in TUI/CLI, not SDK.

**Possible API shapes (needs discussion):**

```python
# Inline after a trace
with bandito.trace("my-project", input=query) as t:
    result = agent.run_sync(query)
    t.done(result)
    t.score("contains_answer", 1 if "Tokyo" in str(result.output) else 0)

# Or as a registered scorer
@bandito.scorer("json_valid")
def check_json(trace):
    try:
        json.loads(trace.output)
        return 1
    except:
        return 0

# Or assertion-style
t.score("not_empty", int(bool(result.output)))
```

**Open questions:**

- Score schema: just a name + numeric value, or richer (category, threshold, metadata)?
- Where do scores get written — same evaluation store as grades, or separate?
- Can scores trigger actions (fail a CI run, flag for human review)?
- Relationship to grades — does a low score auto-queue a trace for human grading?

## Storage Reliability & Performance

### Background writes for network backends

`write_trace` currently blocks in `TraceContext.__exit__`. For local storage this is <1ms. For S3 (50-200ms) and PostgreSQL (10-50ms), this adds perceptible latency to every traced operation.

**Fix:** Queue traces in memory, flush asynchronously in a background thread. `write_trace` returns immediately. Transient failures become retriable instead of data loss.

Pattern from the old Bandito SDK: `ThreadPoolExecutor` with periodic flush + flush-on-close. The store interface stays sync — the async behavior is internal to the network backends.

### Atomic evaluation merge for PostgreSQL

The current merge-on-write pattern (SELECT → merge in Python → INSERT/UPDATE) has a race condition. Two concurrent `write_evaluation` calls can clobber each other.

**Fix:** Use PostgreSQL's JSONB merge operator:
```sql
INSERT INTO bandito_evaluations (trace_id, data)
VALUES (%s, %s)
ON CONFLICT (trace_id) DO UPDATE SET
    data = bandito_evaluations.data || EXCLUDED.data,
    updated_at = NOW()
```

Single atomic statement, no read-modify-write cycle. S3 has the same issue but is harder to fix (no server-side merge) — background writes with local buffering mitigate it.

### Retry logic for network backends

S3 and PostgreSQL writes currently fail silently on transient errors (timeouts, connection resets). A single failed PUT loses a trace permanently.

**Fix:** Add retry with exponential backoff (3 attempts, 1s/2s/4s). Only for transient errors (5xx, connection reset, timeout). Permanent errors (auth failure, bucket not found) fail immediately.

### PostgreSQL connection resilience

The current `PostgresStore` opens one connection at init and holds it forever. If the connection drops, all subsequent calls fail.

**Fix:**
- Reconnect on failure (detect closed connection, re-establish)
- Connection pooling via `psycopg_pool.ConnectionPool` for concurrent access
- Health check on `validate()`

### S3 evaluation merge race condition

Same race as PostgreSQL — GET + merge + PUT is not atomic. Two concurrent writers can lose data.

**Mitigation:** Background write buffer (see above) serializes writes per trace_id on the client side. Not a full fix for multi-instance deployments, but sufficient for single-SDK usage.

## Storage Security

### Credential handling

S3 access keys and PostgreSQL passwords are stored in plaintext in `~/.bandito/config.toml`. Standard for dev tools (AWS CLI does the same) but should be improved.

**Improvements:**
- Prefer provider credential chains (AWS `~/.aws/credentials`, IAM roles, env vars) over storing keys in our config
- Update `_setup_project` to suggest env vars instead of eagerly prompting for keys
- `chmod 600` the config file after writing
- Document security considerations for production deployments
- Future: support secrets managers (AWS Secrets Manager, Vault) for enterprise
