Metadata-Version: 2.4
Name: aipop
Version: 0.8.0
Summary: One CLI for AI security testing. Recon, scan, fuzz, chain, gate.
Author-email: Tyrian Institute <kenneth@tyrianinstitute.org>
License-Expression: MIT
Project-URL: Homepage, https://github.com/tyrianinstitute/AI-Purple-Ops
Project-URL: Repository, https://github.com/tyrianinstitute/AI-Purple-Ops
Project-URL: Issues, https://github.com/tyrianinstitute/AI-Purple-Ops/issues
Keywords: ai,security,llm,pentesting,red-team,prompt-injection,rag,fuzzing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml<7,>=6.0.2
Requires-Dist: python-dotenv<2,>=1.0.1
Requires-Dist: rich>=13.7.1
Requires-Dist: tenacity<9,>=8.3.0
Requires-Dist: typer<1,>=0.12.5
Requires-Dist: jsonschema<5,>=4.23.0
Requires-Dist: junit-xml<2,>=1.9.0
Requires-Dist: httpx<1,>=0.27.0
Requires-Dist: requests<3,>=2.31.0
Requires-Dist: tqdm<5,>=4.66.0
Requires-Dist: websocket-client<2,>=1.8.0
Requires-Dist: platformdirs<5,>=4.0
Requires-Dist: jinja2<4,>=3.1
Provides-Extra: cloud
Requires-Dist: openai>=1.6.0; extra == "cloud"
Requires-Dist: anthropic>=0.8.0; extra == "cloud"
Requires-Dist: boto3>=1.34.0; extra == "cloud"
Provides-Extra: local
Requires-Dist: transformers>=4.36.0; extra == "local"
Requires-Dist: accelerate>=0.25.0; extra == "local"
Requires-Dist: bitsandbytes>=0.41.0; extra == "local"
Provides-Extra: llamacpp
Requires-Dist: llama-cpp-python>=0.2.0; extra == "llamacpp"
Provides-Extra: reports
Requires-Dist: weasyprint>=60.0; extra == "reports"
Requires-Dist: haralyzer>=2.0; extra == "reports"
Requires-Dist: pillow>=10.0; extra == "reports"
Provides-Extra: intelligence
Requires-Dist: duckdb<2,>=1.0.0; extra == "intelligence"
Requires-Dist: scipy<2,>=1.9.0; extra == "intelligence"
Requires-Dist: pygad<4,>=3.3.0; extra == "intelligence"
Requires-Dist: alembic<2,>=1.13; extra == "intelligence"
Provides-Extra: adversarial
Requires-Dist: torch<3,>=2.1.0; extra == "adversarial"
Requires-Dist: transformers>=4.36.0; extra == "adversarial"
Requires-Dist: accelerate>=0.25.0; extra == "adversarial"
Requires-Dist: nanogcg>=0.1.0; extra == "adversarial"
Provides-Extra: pyrit
Requires-Dist: pyrit>=0.9.0; extra == "pyrit"
Provides-Extra: pro
Requires-Dist: promptfoo>=0.90.0; extra == "pro"
Provides-Extra: all-adapters
Requires-Dist: ai-purple-ops[cloud,llamacpp,local]; extra == "all-adapters"
Provides-Extra: all
Requires-Dist: ai-purple-ops[cloud,intelligence,reports]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest<9,>=8.3; extra == "dev"
Requires-Dist: pytest-cov<6,>=5.0; extra == "dev"
Requires-Dist: hypothesis<7,>=6.100; extra == "dev"
Requires-Dist: ruff<1,>=0.6; extra == "dev"
Requires-Dist: black<25,>=24.8; extra == "dev"
Requires-Dist: mypy<2,>=1.11; extra == "dev"
Requires-Dist: bandit<2,>=1.7.9; extra == "dev"
Requires-Dist: pip-audit<3,>=2.7; extra == "dev"
Requires-Dist: pre-commit<4,>=3.8; extra == "dev"
Requires-Dist: types-PyYAML<7,>=6.0.12; extra == "dev"
Requires-Dist: types-requests<3,>=2.31; extra == "dev"
Provides-Extra: pair-official
Requires-Dist: fschat==0.2.23; extra == "pair-official"
Requires-Dist: anthropic>=0.8.0; extra == "pair-official"
Requires-Dist: google-generativeai>=0.3.0; extra == "pair-official"
Requires-Dist: wandb>=0.16.0; extra == "pair-official"
Requires-Dist: pandas>=2.0.0; extra == "pair-official"
Provides-Extra: autodan-official
Requires-Dist: fschat==0.2.20; extra == "autodan-official"
Requires-Dist: transformers==4.28.0; extra == "autodan-official"
Requires-Dist: sentencepiece>=0.1.99; extra == "autodan-official"
Requires-Dist: nltk>=3.8.1; extra == "autodan-official"
Dynamic: license-file

<p align="center">
  <img src="branding/logo-dark-bg.png" alt="aipop" width="400">
</p>

<p align="center">
  <strong>One CLI for AI security testing. Unifies the tools you already use.</strong>
</p>

<p align="center">
  <a href="https://asciinema.org/a/q4dOO0SU8Vf4ESlS"><img src="https://asciinema.org/a/q4dOO0SU8Vf4ESlS.svg" width="800" alt="aipop demo"></a>
</p>

<p align="center">
  <code>pip install aipop</code>
</p>

---

## 5 commands. That's the whole workflow.

### 1. Recon — what is this thing?

```bash
aipop scan http://target:8000/chat
```

```
  phase 1/4 — HTTP fingerprinting (endpoints, headers, OpenAPI)
    ↳ 11 endpoints discovered
    ↳ OpenAPI spec found — extracted routes and schemas
    ↳ framework: FastAPI (uvicorn)
    ↳ model: gpt-4o-mini (from /health)
    ↳ upload: /upload — UNGUARDED
  phase 2/4 — behavioral probes (RAG, tools, memory)
    ↳ RAG detected — grounded response with specific billing details
    ↳ tools: not detected
    ↳ memory: stateless
  phase 3/4 — framework and guardrail fingerprinting
    ↳ guardrail: model-level (high confidence)
  confidence: high (10 signals)
```

Finds the framework, model, endpoints, upload surfaces, RAG, tools, memory, and guardrail type. All from HTTP fingerprinting first, behavioral probes second. No guessing.

### 2. Scan — direct injection testing

```bash
aipop scan http://target:8000/chat --suite rag_injection
```

```
  CRIT  rag_doc_summary_hijack    (1096ms)  → followed instruction embedded in document context
  CRIT  rag_doc_exfil_via_summary (1935ms)  → exfiltration instruction disguised as email postscript
  HIGH  rag_cross_doc_authority   (2165ms)  → cross-document authority override accepted

  status:  VULNERABLE
  vulns:   4 critical, 2 high
  confidence: firm (judge verified, not just keyword match)
```

254 test cases across 19 suites. Five-layer judge cascade classifies findings — behavioral gates, refusal detection, canary markers, LLM-as-judge, then heuristic. No more keyword-matching false positives.

### 3. Fuzz — indirect injection via poisoned documents

```bash
aipop fuzz http://target:8000 \
  --strategy hidden_text \
  --trigger "What is the refund policy?" \
  --upload-endpoint /upload
```

Crafts poisoned PDFs with hidden text, uploads them to the RAG pipeline, triggers with a benign query, detects leaked data. Three concealment strategies: hidden text (white on white), metadata injection, PDF annotations.

This is the attack that matters. Not "ignore previous instructions" — a poisoned document that a normal user triggers by asking a normal question.

### 4. Chain — multi-step attack sequences

```bash
aipop chain suites/chains/indirect_upload.yaml --target http://target:8000
```

```yaml
steps:
  - id: upload_poison
    action: http_request
    request:
      method: POST
      endpoint: /upload
      body:
        content: "{{payload}}"
  - id: trigger
    action: http_request
    request:
      method: POST
      endpoint: /chat
      body:
        message: "What is the refund policy?"
    expect:
      response_not_contains: ["api_key", "password"]
```

Upload → wait → trigger → classify. Five chain templates ship. Write your own in YAML.

### 5. Gate — block the deploy

```bash
aipop gate --fail-on critical --generate-evidence
```

Fails CI if critical findings exist. Generates an evidence pack with OWASP, MITRE ATLAS, and CVSS mappings. Exports to Ghostwriter, Dradis, or PDF.

---

## The problem this solves

You're testing an AI agent. You need PyRIT for multi-turn attacks. Promptfoo for template-driven testing. Garak for probing. Each has its own config format, its own output format, its own CLI. You're copy-pasting payloads between three terminals and stitching evidence together by hand.

AIPOP doesn't replace any of them. It orchestrates all of them under one CLI with unified evidence and a single scan command.

## What it finds

Five architectural seams that exist in every AI agent:

| Seam | What breaks | Example finding |
|------|------------|-----------------|
| **Context is untyped** | Retrieved docs treated as instructions | Poisoned PDF leaked credentials via RAG |
| **Tools trust the caller** | Tool calls with attacker-chosen args | `web_fetch` exfiltrated data to webhook |
| **State persists as instructions** | Behavior shifts across turns | Refused in turn 1, leaked in turn 3 |
| **Framing bypasses filters** | Semantic reframing evades safety | Authority reframe caused system prompt disclosure |
| **Monitors ≠ executors** | Encoded content evades guardrails | Base64 payload bypassed classifier |

## Adapters

| Adapter | Target |
|---------|--------|
| *auto* | Any HTTP endpoint — just pass the URL |
| `openai` | GPT-4o, GPT-4o-mini, o1, o3 |
| `anthropic` | Claude Opus 4, Claude Sonnet 4 |
| `ollama` | Local models (Llama 3, Mistral, Phi) |
| `bedrock` | AWS Bedrock |
| `mcp` | MCP servers |

## Install

```bash
pip install aipop
```

Python 3.11+

## License

[MIT](LICENSE)
