# agentreflect

Generate `AGENTS.md` rules from AI coding agent failure logs. Closes the feedback loop in the agent quality trilogy.

```
Measure (coderace) → Generate (agentmd) → Guard (agentlint) → Learn (agentreflect)
```

## What it does

Every developer using Claude Code or Codex has this problem: their agent makes a mistake, they fix it manually, update `AGENTS.md`, and hope it doesn't happen again. `agentreflect` automates the "update AGENTS.md" step.

Feed it failure logs → get targeted rule suggestions → apply them to your `AGENTS.md`.

In notes mode, `agentreflect` now prefers AGENTS-style operational rules over generic QA advice. If a note says the agent missed an exact routing target, skipped downstream verification after an irreversible action, or relied on fuzzy recall instead of exact facts already present in files, the suggestions will mirror that operating rule directly.

## Install

```bash
pip install ai-agentreflect
```

For LLM-enhanced mode:
```bash
pip install 'ai-agentreflect[llm]'
```

## Usage

### From pytest output

```bash
# Capture failures
pytest --tb=short 2>&1 | tee failures.txt

# Generate suggestions
agentreflect generate --from-pytest failures.txt
```

### From git log

```bash
agentreflect generate --from-git
```

Analyzes `fix:`, `bug:`, `revert:` commits and agent-related mistake commits.

### From plain text notes

```bash
agentreflect generate --from-notes "agent forgot to check for None before accessing .value"
agentreflect generate --from-notes "fuzzy retrieval missed explicit facts already present in files; add an exact-match layer for handles and tool routing before semantic lookup"
```

## CI Integration

agentreflect ships a GitHub Action and a CI mode flag to integrate rule generation into your pull request workflow.

### GitHub Action quick-start

```yaml
# .github/workflows/agentreflect.yml
name: agentreflect
on: [pull_request]

jobs:
  learn:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: mikiships/agentreflect@v0.3.0
        with:
          mode: ci
          from-git: 'true'
          threshold: '10'   # fail PR if >= 10 new rules detected
```

### CI mode (machine-readable output)

```bash
# Write JSON to stdout, summary to stderr
agentreflect generate --from-pytest failures.txt --ci

# Fail with exit 1 if >= 5 rules generated (useful for PR gates)
agentreflect generate --from-git --ci --fail-on 5
```

CI mode JSON output:

```json
{
  "ci": true,
  "rules_count": 3,
  "rules_generated": 3,
  "source": "git log",
  "suggestions": [
    { "rule": "...", "rationale": "...", "confidence": 0.85, "category": "..." }
  ]
}
```

### Full quartet pipeline

See [`.github/workflows/examples/agent-quality-pipeline.yml`](.github/workflows/examples/agent-quality-pipeline.yml) for a complete MEASURE → GENERATE → GUARD → LEARN pipeline running all four agent quality tools in sequence:

1. **agentlint** — lint AGENTS.md for structural issues
2. **agentmd** — check context file freshness
3. **coderace** — review code changes
4. **agentreflect** — extract new rules from failures

## Output formats

### Markdown (default)

```
## agentreflect suggestions (2026-03-11)

### From: pytest failures (failures.txt)
- [ ] Always check for None before attribute access: use `if obj is not None` or `hasattr(obj, 'attr')`
- [ ] When catching AttributeError, log the object type with `type(obj).__name__`

_Source: 3 failures analyzed, 2 suggestions generated_
```

### Diff format

```bash
agentreflect generate --from-pytest failures.txt --format diff
```

Outputs a unified diff ready to apply to `AGENTS.md`.

### Apply directly

```bash
agentreflect generate --from-notes "agent used wrong variable" --apply AGENTS.md
# Asks for confirmation

agentreflect generate --from-pytest failures.txt --apply AGENTS.md --yes
# Applies without confirmation
```

## LLM-enhanced mode

```bash
export ANTHROPIC_API_KEY=your_key_here
agentreflect generate --from-pytest failures.txt --llm
```

Uses `claude-3-5-haiku-latest` for contextual, specific suggestions tailored to your actual failures. Cost: ~$0.001 per analysis.

Basic pattern mode works without any API key.

## Integration with the trilogy

| Tool | Role |
|------|------|
| [coderace](https://pypi.org/project/ai-coderace/) | Measure agent output quality |
| [agentmd](https://pypi.org/project/ai-agentmd/) | Generate AGENTS.md from scratch |
| [agentlint](https://pypi.org/project/ai-agentlint/) | Guard/validate AGENTS.md rules |
| **agentreflect** | **Learn from failures → update rules** |

## License

MIT
