# AI Trace Auditor

Audit LLM traces against regulatory compliance requirements. Open-source CLI that sits between your observability stack (Langfuse, Arize, OTel) and regulatory frameworks (EU AI Act, NIST AI RMF).

Your observability tools collect traces. Your GRC platform manages policies. **Nothing translates traces into compliance evidence.** This tool does.

## Install

```bash
pip install ai-trace-auditor
```

Or from source:

```bash
git clone https://github.com/BipinRimal314/ai-trace-auditor.git
cd ai-trace-auditor
pip install -e .
```

## Quick Start

```bash
# Audit traces against all regulations
aitrace audit traces.json

# Audit against a specific regulation
aitrace audit traces.json -r "EU AI Act" -o report.md

# Audit your Claude Code conversation traces
aitrace audit ~/.claude/projects/*/session-id.jsonl

# Inspect what requirements exist
aitrace requirements --show EU-AIA-12.1

# Just ingest and summarize traces
aitrace ingest traces.json --summary
```

## GitHub Action

Add compliance checks to your CI pipeline:

```yaml
- name: Audit AI traces
  uses: BipinRimal314/ai-trace-auditor@v0.2.0
  with:
    path: traces/exported.json
    regulation: "EU AI Act"
    output: compliance-report.md
    fail-on-gaps: "true"
```

The action fails if compliance gaps are found. Set `fail-on-gaps: "false"` to report without blocking.

## What It Checks

**EU AI Act Article 12 (Record-Keeping):**
- Event timestamps, operation identification
- Risk situation logging (errors, failure modes)
- Model version tracking for post-market monitoring
- Resource consumption (tokens, latency)
- Content recording (opt-in)
- Tool/function call audit trails
- Trace linkage for multi-step operations

**NIST AI RMF:**
- Production monitoring (MEASURE 2.4)
- Transparency documentation (MEASURE 2.8)
- Model explainability (MEASURE 2.9)
- Risk tracking (MEASURE 3.1)
- Post-deployment monitoring (MANAGE 4.1)
- Incident communication (MANAGE 4.3)

## Supported Trace Formats

| Format | Source |
|--------|--------|
| OTel OTLP JSON | OpenTelemetry GenAI semantic conventions |
| Langfuse JSON | Langfuse trace exports |
| Claude Code | `~/.claude/projects/` conversation traces |
| Raw JSONL | Any provider's API logs |

Auto-detected. Use `--format` to override.

## Example Output

Real output from auditing 1,522 Claude Code spans:

```
Overall Compliance Score: 79.3%

| Status    | Count |
|-----------|-------|
| Satisfied |    10 |
| Partial   |     5 |
| Missing   |     3 |

Top gaps:
  1. Not logging: Temperature parameter controlling output randomness
  2. Not logging: Maximum token limit for output generation
  3. Incomplete: Output responses generated by the AI model (31.9% coverage)
  4. Incomplete: Input prompts/messages (4.7% coverage)
  5. Not logging: Operation latency in milliseconds
```

## CI Integration

Exit code 0 = all satisfied, 1 = gaps found:

```bash
aitrace audit traces.json -r "EU AI Act" || echo "Compliance gaps detected"
```

## Library API

Use programmatically in your own tools:

```python
from ai_trace_auditor.ingest import ingest_file
from ai_trace_auditor.analysis.engine import ComplianceAnalyzer
from ai_trace_auditor.regulations.registry import RequirementRegistry

traces = ingest_file(Path("traces.json"))

registry = RequirementRegistry()
registry.load()

report = ComplianceAnalyzer(registry).analyze(
    traces=traces,
    regulations=["EU AI Act"],
)

print(f"Score: {report.overall_score:.1%}")
for result in report.requirement_results:
    if result.gaps:
        print(f"  {result.requirement.id}: {result.gaps[0].recommendation}")
```

## Disclaimer

This tool provides automated compliance assessments based on its interpretation of regulatory requirements. It is **not legal advice**. Consult qualified legal counsel for compliance decisions.

## License

Apache 2.0
