Metadata-Version: 2.4
Name: ai-testing-swarm
Version: 0.1.15
Summary: AI-powered testing swarm
Author-email: Arif Shah <ashah7775@gmail.com>
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: requests>=2.28
Requires-Dist: PyYAML>=6.0
Provides-Extra: openapi
Requires-Dist: jsonschema>=4.0; extra == "openapi"

# AI Testing Swarm

AI Testing Swarm is a **super-advanced, mutation-driven API testing framework** (with optional OpenAPI + OpenAI augmentation) built on top of **pytest**.

It generates a large set of deterministic negative/edge/security test cases for an API request, executes them (optionally in parallel, with retries/throttling), and produces a report (JSON/Markdown/HTML) with summaries.

> Notes:
> - UI testing is not the focus of the current releases.
> - OpenAI features are **optional** and disabled by default.

---

## Installation

```bash
pip install ai-testing-swarm
```

Optional (OpenAPI JSON schema validation for responses):

```bash
pip install "ai-testing-swarm[openapi]"
```

CLI entrypoint:

```bash
ai-test --help
```

---

## Quick start (cURL input)

Create `request.json`:

```json
{
  "curl": "curl --location https://postman-echo.com/post --header \"Content-Type: application/json\" --data \"{\\\"hello\\\":\\\"world\\\",\\\"count\\\":1}\""
}
```

Run:

```bash
ai-test --input request.json
```

Choose a report format:

```bash
ai-test --input request.json --report-format html
```

A report is written under:

- `./ai_swarm_reports/<METHOD>_<endpoint>/<METHOD>_<endpoint>_<timestamp>.<json|md|html>`

Reports include:
- per-test results
- summary counts by status code / failure type
- optional AI summary (if enabled)

---

## Input formats

### 1) Raw cURL

```json
{ "curl": "curl ..." }
```

### 2) Normalized request

```json
{
  "method": "POST",
  "url": "https://example.com/api/login",
  "headers": {"content-type": "application/json"},
  "params": {"a": "b"},
  "body": {"username": "u", "password": "p"}
}
```

### 3) OpenAPI-driven (optional)

```json
{
  "openapi": "./openapi.json",
  "path": "/pets",
  "method": "get",
  "headers": {"accept": "application/json"},
  "path_params": {"petId": "123"},
  "query_params": {"limit": 10},
  "body": null
}
```

- OpenAPI **JSON** works by default.
- OpenAPI **YAML** requires `PyYAML` installed.
- Base URL is read from `spec.servers[0].url`.
- When using OpenAPI input, the swarm will also *optionally* validate response status codes against `operation.responses`.
- If `jsonschema` is installed (via `ai-testing-swarm[openapi]`) and the response is JSON, response bodies are validated against the OpenAPI `application/json` schema.
  - Override with `AI_SWARM_OPENAPI_BASE_URL` if your spec doesn’t include servers.

---

## What test cases are generated?

The swarm always includes:
- **happy_path** (baseline)

Then generates broad coverage across:
- **Method misuse**: same path with wrong HTTP methods (GET/PUT/PATCH/DELETE etc.)
- **Headers**: missing/invalid `Content-Type`, accept variations, and other header tampering
- **Auth** (if `Authorization` header exists): missing/invalid token tests
- **Body/query mutations** (per field):
  - missing / null / empty / whitespace
  - type probes (int/bool/float/array/object)
  - boundary inputs (very long strings, huge ints, negative values)
  - unicode + special character payloads
- **Security payload probes** (per field): SQLi/XSS/path traversal/log4j patterns
- **Whole-body mutations**: null body, empty object, extra unexpected field

> Output is deterministic unless OpenAI augmentation is enabled.

---

## Safety mode (recommended for CI/demos)

Mutation testing can be noisy and may accidentally stress a real environment.
To force safe demo runs only against public test hosts:

```bash
ai-test --input request.json --public-only
```

Or via env:

```bash
export AI_SWARM_PUBLIC_ONLY=1
```

Allowed hosts in public-only mode:
- `httpbin.org`
- `postman-echo.com`
- `reqres.in`

---

## Performance features

### Parallel execution
- Enabled by default via thread pool.
- Control with:
  - `AI_SWARM_WORKERS` (default: `5`)

### Retry + backoff (flaky endpoints)
- Retries on transient errors and status codes (408/429/5xx etc.)
- Control with:
  - `AI_SWARM_RETRY_COUNT` (default: `1`)
  - `AI_SWARM_RETRY_BACKOFF_MS` (default: `250`)

### Throttling (RPS)
- Global throttle to avoid hammering a target:
  - `AI_SWARM_RPS` (default: `0` = disabled)

### Max test cap
- Avoids accidental DoS / CI timeouts:
  - `AI_SWARM_MAX_TESTS` (default: `80`)

---

## Reporting

Reports include:
- `summary.counts_by_failure_type`
- `summary.counts_by_status_code`
- `summary.slow_tests` (based on SLA)

SLA threshold:
- `AI_SWARM_SLA_MS` (default: `2000`)

Security:
- Sensitive headers are redacted in the report (Authorization/Cookie/api tokens etc.)

---

## Optional OpenAI augmentation (advanced)

### A) Generate additional test cases (planner augmentation)
Enable:

```bash
export AI_SWARM_USE_OPENAI=1
export OPENAI_API_KEY=... 
export AI_SWARM_MAX_AI_TESTS=30
```

### B) Human-readable AI summary in report
Enable:

```bash
export AI_SWARM_USE_OPENAI=1
export AI_SWARM_AI_SUMMARY=1
export OPENAI_API_KEY=...
```

Model selection:
- `AI_SWARM_OPENAI_MODEL` (default: `gpt-4.1-mini`)

---

## CLI help

```bash
ai-test --help
```

---

## Release decisions

The swarm produces a release decision:
- `APPROVE_RELEASE`
- `APPROVE_RELEASE_WITH_RISKS`
- `REJECT_RELEASE`

The decision is derived from deterministic rules (not an LLM).

---

## License

MIT
