# E2E Test Run - Backend Log Analysis Report

**Date**: 2026-02-17
**Duration**: 18:41 - 18:58 UTC (~17 minutes of active monitoring)
**Total log lines**: 6,900
**Log file**: `docs/logs/e2e-test-run.log`

## Executive Summary

The E2E test run revealed **one critical backend bug** (Pydantic model_rebuild) and confirmed several **known pre-existing issues**. No new security vulnerabilities, no PII leakage, and no unexpected errors were found during testing.

**Verdict: CONDITIONAL PASS** - The critical `/openapi.json` 500 error needs fixing, but it does not affect core API functionality (auth, keys, costs, billing). All other endpoints responded correctly.

## HTTP Response Summary

| Status | Count | Description |
|--------|-------|-------------|
| 200    | 94    | Successful responses |
| 404    | 115   | Not found (mostly expected - unimplemented endpoints) |
| 500    | 12    | Server errors (all `/openapi.json`) |

## Error Analysis

### CRITICAL: `/openapi.json` -> 500 (12 occurrences)

**Root Cause**: `PydanticUserError` - The model `llmhosts.proxy.dashboard_models.RequestLogResponse` is not fully defined due to `from __future__ import annotations` deferred type evaluation. The model needs `.model_rebuild()` called after definition.

**Impact**: The `/docs` page loads (returns 200) but the Swagger UI cannot render because `/openapi.json` fails. This affects developer documentation but NOT the actual API functionality.

**Fix**: Add `RequestLogResponse.model_rebuild()` (and any other models in that module) at the bottom of `llmhosts/proxy/dashboard_models.py`, or in a startup event. This is the same pattern already used in `tests/conftest.py`.

**Stack trace summary**:
```
FastAPI -> get_openapi() -> get_definitions() -> Pydantic json_schema ->
  generate_inner() -> 'ref' in schema -> PydanticUserError:
  RequestLogResponse is not fully defined, call .rebuild()
```

### MEDIUM: Frontend Auth CredentialsSignin (5 occurrences)

**Source**: Auth.js credential provider
**Assessment**: Expected behavior during E2E login testing. Auth.js logs failed login attempts at error level by default. These appear to be from test scenarios testing invalid credentials.

### LOW: Cache API 404s (64 frontend + 30+ backend occurrences)

**Endpoints**: `/api/cache/stats`, `/api/cache/entries`, `/api/cache/namespaces`
**Assessment**: Pre-existing known issue. The cache management API endpoints are not yet exposed on the Python backend. The frontend dashboard cache UI polls these periodically.

### LOW: LAN Nodes 404 (12 occurrences)

**Endpoint**: `/api/lan-nodes`
**Assessment**: Pre-existing. The LAN discovery endpoint is not yet exposed via the proxy API.

### LOW: Metrics 404 (multiple occurrences)

**Endpoint**: `/metrics` (root path)
**Assessment**: Prometheus metrics are available at `/api/metrics` (which returns 200), but the monitoring stack was also trying `/metrics` (root path) which doesn't exist.

### LOW: Stripe Checkout Error (2 occurrences)

**Error**: `No such price: 'price_test_123'`
**Assessment**: Pre-existing known blocker. Stripe is configured with placeholder test prices. Needs real Stripe price IDs.

## Security Checks

### PII Redaction: PASS
- No email addresses found in log output (excluding URL references)
- No API keys (sk-*) leaked in any log entry
- No passwords visible in any log entry
- Stripe request IDs are logged but contain no PII

### Correlation IDs: NOT PRESENT
- No `X-Request-ID`, `correlation-id`, `trace-id`, or similar headers found in log output
- The middleware log format is: `timestamp [LEVEL] module: METHOD /path -> STATUS (timing)`
- **Recommendation**: Add correlation ID logging to the middleware for better request tracing

## Endpoint Coverage Observed

| Endpoint | Status | Notes |
|----------|--------|-------|
| `GET /health` | 200 | Consistent, healthy |
| `GET /docs` | 200 | Loads but Swagger broken due to /openapi.json |
| `GET /openapi.json` | 500 | Pydantic model_rebuild bug |
| `GET /api/metrics` | 200 | Working correctly |
| `GET /metrics` | 404 | Not exposed at root path |
| `GET /api/cache/stats` | 404 | Not yet implemented |
| `GET /api/cache/entries` | 404 | Not yet implemented |
| `GET /api/cache/namespaces` | 404 | Not yet implemented |
| `GET /api/lan-nodes` | 404 | Not yet implemented |

## Performance Notes

- Health check response times: 0.4ms - 3.6ms (excellent)
- `/api/metrics` response times: 0.5ms - 5.7ms (good)
- `/openapi.json` error response times: 154.7ms - 709.4ms (slow due to exception handling)
- All 404 responses: < 10ms (appropriate)

## Recommendations

1. **HIGH**: Fix `RequestLogResponse.model_rebuild()` in dashboard_models.py
2. **MEDIUM**: Add correlation ID (X-Request-ID) to middleware logging format
3. **LOW**: Expose `/metrics` at root path for Prometheus compatibility, or document that `/api/metrics` is the correct path
4. **LOW**: Consider reducing frontend cache UI polling frequency when backend returns 404 (16 polls in 17 minutes)
