# StackSage Cloud Cost Audit (MVP)

Run an end-to-end AWS cost audit: scan resources, analyze waste, estimate savings, and generate a human-readable report.

## Quick start

Requirements:
- Python 3.10+
- AWS credentials or a read-only role (see `docs/read-only-policy.json`)
- (Optional) `stacksage.yml` for custom configuration

Install deps (MVP uses boto3, jinja2, click):

```zsh
# optional: create venv
python -m venv .venv && source .venv/bin/activate
pip install boto3 jinja2 click pyyaml
```

### Configuration (Optional)

StackSage supports optional configuration via `stacksage.yml` in your repository root:

```bash
cp stacksage.yml.example stacksage.yml
# Edit stacksage.yml to customize exclusions, thresholds, and filters
```

See [docs/configuration.md](docs/configuration.md) for full configuration guide.

### Full audit

```zsh
python -m stacksage.cli.cli audit --role-arn YOUR_ROLE_ARN --out reports --format html
```

Optional region selection:
```zsh
python -m stacksage.cli.cli audit --regions us-east-1,eu-west-1 --out reports --format html
```

### Scan only

```zsh
python -m stacksage.cli.cli scan --role-arn YOUR_ROLE_ARN --out reports
```

### Analyze only

```zsh
python -m stacksage.cli.cli analyze --in-file reports/scan_raw.json --out reports/findings.json
```

### Report only

```zsh
python -m stacksage.cli.cli report --findings reports/findings.json --out reports
```

## CLI options
- `--role-arn` Assume a read-only role (optional; uses local creds if omitted)
- `--out` Output directory (`reports` by default)
- `--format` `html|json` for audit output
- `--regions` Comma-separated AWS regions (default: all enabled)
- `--cw-days` CloudWatch metrics lookback days (default: 14)
- `--cw-max-queries` Max CloudWatch metric queries per run (budget). Budget exhaustion lowers confidence and flags findings as skipped-budget.
- `--log-level` Logging level: `DEBUG|INFO|WARNING|ERROR` (default: `INFO`). Logs are structured JSON.

## Files
- `run_scan.py` Scan-only entrypoint (writes `reports/scan_raw.json` with metadata)
- `stacksage/scanner/aws_scanner.py` Scans EC2/EBS/RDS per region; S3 globally
- `stacksage/analyzer/analysis.py` Analyzes raw scan data and produces findings
	- CloudWatch helper `cw_avg`: retry/backoff (up to 2 retries) and error classification (`throttle`, `not_authorized`, `no_data`, `other`).
	- Provenance includes CloudWatch counters: `cloudwatch_queries_attempted`, `cloudwatch_queries_used`, `cloudwatch_queries_remaining`.
	- Skipped-budget behavior: detectors set `metric_status=skipped_budget`, `confidence=0.4`, and clear explanation.
- `stacksage/pricing.py` Static pricing and helper functions
- `stacksage/cli/cli.py` CLI with `audit`, `scan`, `analyze`, `report`
- `stacksage/cli/report_gen.py` Generates HTML report from findings JSON
- `stacksage/templates/report.html.jinja` HTML template (shows metadata, top savings, counts)
- `docs/read-only-policy.json` IAM policy to grant read-only access
- `docs/schema.md` Data shapes for resources/findings
- `docs/function_signatures.md` Signatures for planning/testing
- `docs/TODOs.md` Roadmap and tasks

## IAM Read-Only Policy
Provide customers with `docs/read-only-policy.json`. It includes actions:
- `ec2:Describe*`
- `s3:ListAllMyBuckets`, `s3:GetBucketLocation`
- `rds:Describe*`
- `cloudwatch:GetMetricStatistics` (Phase-1 offline analyzer does not call CloudWatch; keep for future utilization insights)
	- When enabled, StackSage uses only aggregate metrics (Averages) and a bounded query budget.
- `pricing:GetProducts` (optional; remove if using only static pricing)
- `ce:GetCostAndUsage`, `ce:GetDimensionValues` (for historical cost summary)
- `tag:GetResources`
- `iam:GetRole`
	(Remove unused log event permissions if present.)

Recommendations:
- Keep `cloudwatch:GetMetricStatistics` for utilization metrics.
- If customers have SCPs, ensure `ec2:Describe*` and `rds:Describe*` are allowed in all target regions.
- If historical spend is desired, include Cost Explorer permissions (`ce:GetCostAndUsage`).

## Development notes

- Install dev tooling + enable local git hooks (recommended):

```zsh
./scripts/setup_dev.sh
```

This installs both `pre-commit` and `pre-push` hooks, so ruff/black/isort run on every commit and push.

- Protecting `main` (recommended): enable GitHub branch protection for `main` and require the **CI** workflow checks to pass before merge. This prevents direct pushes and ensures linting/tests gate changes.

- Multi-region scanning is supported; omit `--regions` to scan all enabled regions.
- Findings include both estimated monthly cost and savings where applicable.
- Analyzer operates offline in Phase-1 and uses only scanner outputs (no live AWS calls).
- Optional real pricing: set `STACKSAGE_PRICING_MODE=api` to use AWS Pricing API for EC2.
 - Optional CloudWatch live-mode: enable `--use-cloudwatch` or `--live`. Budget enforced via `--cw-max-queries`.
 - Provenance counters are also written to `reports/run_provenance.json` when HTML report is generated.
 - Error classification from CloudWatch is attached to findings (`metrics_error`).
- Optional historical costs: if `ce:GetCostAndUsage` is allowed, audit includes last 30-day spend summary.
- Reports include an AI executive summary (heuristic, offline) and CSV export of findings.

## Troubleshooting
- If `pricing:GetProducts` is denied, static pricing is used.
- If Cost Explorer is denied, the report will omit historical spend.
- For IAM setup, see `docs/customer-iam-role-setup.md`. Use `--log-level DEBUG` for detailed structured logs.
 - If you see `skipped-budget` badges in the report, consider increasing `--cw-max-queries` or narrowing regions.

### Refreshing auditor/source credentials
- See `docs/refresh-auditor-creds.md` for step-by-step instructions to mint fresh auditor/source credentials (IAM user or CloudShell), export them to your shell or `.env`, and re-run `scripts/run_audit.sh` with a preflight identity check.
