# Zuul CI Slack Bot - Proposal Summary

## The Problem

Engineers debug CI failures manually - navigating Zuul web UIs, scrolling logs, asking teammates. We have a generic Zuul MCP server (mcp-zuul, 36 tools, published on PyPI) but it requires Claude Code setup per user. Most engineers won't do that.

Meanwhile, the CIOps team has deep tribal knowledge about failure patterns ("ImagePullBackOff = registry flake, safe to retry") that isn't captured anywhere systematic.

## The Proposal

A **Slack bot** that any engineer can use to ask about their CI jobs, powered by Claude AI + mcp-zuul + a curated knowledge base that the CIOps team manages.

```
Engineer in Slack                    CIOps in #ciops-patterns
"@zuulbot why did my                 Bot: "New pattern detected:
 build fail?"                         galera bootstrap failed.
      |                                Seen 3 times. Approve?"
      v                                      |
 Claude AI + mcp-zuul                   React checkmark
 + knowledge base                           |
      |                               Pattern added to KB
      v                               Next engineer gets
 "Registry flake.                     the answer instantly
  Safe to retry."
```

## Two-Tier Access

| | Everyone | CIOps Team |
|---|---|---|
| Ask questions | Yes | Yes |
| Get project-aware answers | Yes | Yes |
| Add failure patterns | - | `/zuulbot add-pattern` |
| Approve bot suggestions | - | React in #ciops-patterns |
| Review/graduate patterns | - | `/zuulbot patterns` |

Access control is channel-based - management commands only work in designated CIOps channels. No RBAC setup needed.

## How It Works

1. **Engineer** asks `@zuulbot why did build abc123 fail?` in any channel
2. **Bot** calls mcp-zuul tools (`diagnose_build`, `get_build_log`, etc.)
3. **Claude** interprets results using the knowledge base (failure patterns, job conventions)
4. **Bot** posts a project-aware diagnosis to Slack
5. If the error is **unknown** - bot tracks it, and after 3 occurrences, suggests it to CIOps for review

## The Learning Loop

The bot gets smarter over time:

- **Day 1**: 30+ curated failure patterns from existing team knowledge
- **Week 2**: Bot has detected 5 new patterns from engineer questions, CIOps approved 3
- **Month 2**: Knowledge base has doubled through organic use
- **Ongoing**: Monthly graduation promotes approved patterns to the curated baseline

## Architecture

```
PUBLIC (PyPI)                    INTERNAL (GitLab)
+-------------+                 +-----------------------------+
| mcp-zuul    |                 | zuul-ci-bot                 |
| 36 Zuul API |<--- MCP -----  | +-- Slack Bolt (bot)        |
| tools       |   protocol     | +-- Claude API (reasoning)   |
| generic,    |                 | +-- Knowledge Store          |
| unchanged   |                 |     +-- YAML baseline (git)  |
+-------------+                 |     +-- SQLite (learned)     |
                                | +-- Pattern Tracker          |
                                |     +-- #ciops-patterns      |
                                +-----------------------------+
```

**Key principle**: mcp-zuul stays public and generic. All project-specific knowledge lives in the internal repo.

## What Engineers Can Ask

| Question | Example |
|----------|---------|
| Why did a build fail? | "why did build abc123 fail?" |
| Is a job flaky? | "is deploy-ocp flaky?" |
| Change status | "status of change 1925" |
| Job configuration | "what jobs run for ci-framework?" |
| Queue health | "why are jobs stuck?" |

## Technology Stack

| Component | Technology | Why |
|-----------|------------|-----|
| Slack integration | Slack Bolt (Python, Socket Mode) | Official SDK, no public URL needed |
| AI reasoning | Claude API (tool_use) | Best tool orchestration, understands context |
| Zuul access | mcp-zuul (MCP protocol) | Already built, 36 tools, published |
| Knowledge - curated | YAML files in git | Version controlled, team reviews |
| Knowledge - learned | SQLite | Real-time updates, zero infra |
| Deployment | Single container (k8s) | Bot + mcp-zuul subprocess, Socket Mode |

## Phased Rollout

| Phase | What | Effort |
|-------|------|--------|
| **1. MVP** | Engineers ask questions, static knowledge | Small |
| **1.5** | Bot detects unknown patterns, posts to CIOps | ~50 lines |
| **2** | CIOps approves via Slack reactions | Medium |
| **3** | Graduation to YAML baseline | Medium |
| **4** | Jira/GitLab integrations | Medium-Large |

Each phase is additive - no changes to earlier phases.

## What We Already Have

- **mcp-zuul** - published on PyPI, 36 tools, works with any Zuul instance
- **30+ failure patterns** - documented in knowledge base files, ready to seed the bot
- **Proven architecture** - the same MCP + Claude pattern already powers personal tooling (zuul-job skill, babysit-tp)

## What We Need

- Slack app registration (bot token + app token)
- Anthropic API key (Claude API access)
- A container to run the bot (single pod, no ingress)
- CIOps channel for pattern management
- ~2 weeks for Phase 1 MVP

## Why Not Alternatives?

| Alternative | Why Not |
|---|---|
| Custom MCP server per project | Maintenance nightmare, duplicates mcp-zuul |
| Everyone installs Claude Code | High friction, requires per-user setup |
| MCP Resources/Prompts | Client support isn't mature enough |
| Web UI | Engineers are already in Slack |
| Static docs/wiki | Not interactive, gets stale, no learning loop |
