# Sanicode — Research & Standards Baseline

> Code sanitization standards, frameworks, and tooling landscape for building an AI-assisted CLI scanner/recommender.
>
> Last updated: 2026-02-20

---

## Table of Contents

1. [Project Vision](#project-vision)
2. [OWASP Standards & Guides](#owasp-standards--guides)
3. [NIST Recommendations](#nist-recommendations)
4. [DISA STIGs](#disa-stigs)
5. [CWE / SANS Top 25](#cwe--sans-top-25)
6. [PCI DSS](#pci-dss)
7. [SEI CERT Coding Standards](#sei-cert-coding-standards)
8. [Existing Tooling Landscape](#existing-tooling-landscape)
9. [AI-Assisted Code Analysis — State of the Art](#ai-assisted-code-analysis--state-of-the-art)
10. [Standards Cross-Reference Matrix](#standards-cross-reference-matrix)
11. [Architecture Notes](#architecture-notes)
12. [Disconnected / Air-Gapped Architecture](#disconnected--air-gapped-architecture)
13. [Operator vs. Application: Distribution & Lifecycle Strategy](#operator-vs-application-distribution--lifecycle-strategy)
14. [Observability, Pipelines & Dashboard Integration](#observability-pipelines--dashboard-integration)

---

## Project Vision

**Sanicode** is a PyPI-distributed CLI tool that uses an AI agent (configurable LLM backend via `sanicode config`) to build a codebase-level understanding via knowledge graph, then outputs baseline security/sanitization recommendations mapped to industry standards.

**Phase 1 — Scanner/Recommender:** Static analysis + LLM-powered contextual understanding. Scan a codebase, build a knowledge graph of data flows, identify sanitization gaps, and produce recommendations mapped to OWASP ASVS, NIST 800-53, and ASD STIG controls.

**Phase 2 — Refactoring Assistant:** Use the knowledge graph to inform code migration tools, suggest concrete fixes, and generate compliant code patterns.

**Differentiator vs. existing tools:** Existing SAST tools (Bandit, Semgrep) find pattern-level issues. Sanicode aims to understand *data flow context* — where untrusted data enters, how it flows through the application, and where sanitization is missing or insufficient — using LLM reasoning over a knowledge graph rather than just AST pattern matching.

---

## OWASP Standards & Guides

### OWASP Top 10:2025

Injection moved from #3 (2021) to **#5 (A05:2025)** but remains one of the most exploited categories. 100% of applications tested showed some form of injection vulnerability. The 2025 edition also references LLM prompt injection as an emerging concern (covered separately in the OWASP LLM Top 10, LLM01:2025).

Key prevention guidance: separate data from commands (parameterized queries, ORMs), positive server-side input validation, output encoding/escaping by context, disable dangerous interpreter features.

- **OWASP Top 10:2025 Landing Page:** https://owasp.org/Top10/2025/
- **A05:2025 Injection:** https://owasp.org/Top10/2025/A05_2025-Injection/

### OWASP ASVS (Application Security Verification Standard)

**Version 5.0.0** was released May 30, 2025 at Global AppSec EU Barcelona. This is a major restructuring from v4.0.3.

The former "V5: Validation, Sanitization and Encoding" chapter has been restructured into **"V1: Encoding and Sanitization"** in ASVS 5.0. This is the single most relevant formal standard for what sanicode does. It covers:

- **Injection Prevention (Section 1.2):** OS command injection, parameterized queries, contextual output encoding
- **General Sanitization:** Enforcing safe character sets, length limits, context-appropriate cleaning
- **SMTP/IMAP Injection:** Mail system input sanitization
- **Template Injection:** Preventing untrusted input in template construction
- **SSRF Prevention:** Allowlist validation of protocols, domains, paths, ports
- **SVG Sanitization:** Validating/sanitizing scriptable SVG content
- **ReDoS Prevention (new in 5.0):** Regular expressions free from exponential backtracking
- **JNDI Injection Prevention (new in 5.0):** Sanitizing input before JNDI queries

ASVS defines three verification levels (L1, L2, L3) providing increasing rigor — useful for sanicode to offer tiered recommendations.

Version identifiers use format: `v<version>-<chapter>.<section>.<requirement>`

- **ASVS Project Page:** https://owasp.org/www-project-application-security-verification-standard/
- **ASVS 5.0 — V1 Encoding and Sanitization:** https://asvs.dev/v5.0.0/V1-Encoding-and-Sanitization/
- **ASVS GitHub:** https://github.com/OWASP/ASVS
- **ASVS 4.0.3 V5 (legacy reference):** https://github.com/OWASP/ASVS/blob/v4.0.3/4.0/en/0x13-V5-Validation-Sanitization-Encoding.md

### OWASP Cheat Sheet Series

Developer-facing, actionable guidance. These are the primary references sanicode should map recommendations to:

| Cheat Sheet | URL |
|---|---|
| Input Validation | https://cheatsheetseries.owasp.org/cheatsheets/Input_Validation_Cheat_Sheet.html |
| Injection Prevention (General) | https://cheatsheetseries.owasp.org/cheatsheets/Injection_Prevention_Cheat_Sheet.html |
| SQL Injection Prevention | https://cheatsheetseries.owasp.org/cheatsheets/SQL_Injection_Prevention_Cheat_Sheet.html |
| XSS Prevention | https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html |
| DOM-based XSS Prevention | https://cheatsheetseries.owasp.org/cheatsheets/DOM_based_XSS_Prevention_Cheat_Sheet.html |
| OS Command Injection Defense | https://cheatsheetseries.owasp.org/cheatsheets/OS_Command_Injection_Defense_Cheat_Sheet.html |
| LDAP Injection Prevention | https://cheatsheetseries.owasp.org/cheatsheets/LDAP_Injection_Prevention_Cheat_Sheet.html |
| Full Cheat Sheet Index | https://cheatsheetseries.owasp.org/index.html |

### OWASP Testing Guide (WSTG)

Has specific test cases for input validation, XSS, SQLi, and command injection — all requiring sanitization controls. Useful for sanicode's test/verification recommendations.

- **WSTG:** https://owasp.org/www-project-web-security-testing-guide/

---

## NIST Recommendations

### NIST SP 800-53 Rev 5 — Security and Privacy Controls

The **SI (System and Information Integrity)** and **SA (System and Services Acquisition)** control families are most relevant.

**SI-10: Information Input Validation**
Requires checking valid syntax and semantics of system inputs — character set, length, numerical range, acceptable values. The supplemental guidance explicitly warns: if applications use attacker-supplied inputs to construct structured messages without proper encoding, attackers can insert malicious commands or special characters interpreted as control information. Prescreening inputs before passing to interpreters prevents unintentional command interpretation.

**SI-15: Information Output Filtering**
Requires validating information output to ensure consistency with expected content. Focuses on detecting extraneous content (e.g., unexpected SQL query results), preventing display, and alerting monitoring. Related controls: SI-3 (Malicious Code Protection), SI-4 (System Monitoring), SI-11 (Error Handling).

**SA-11: Developer Testing and Evaluation**
Covers secure code review and testing practices.

SI-10 and SI-15 form complementary input/output controls — sanicode should check for both.

- **SP 800-53 Rev 5 (full):** https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final
- **SI-10 Reference:** https://csf.tools/reference/nist-sp-800-53/r4/si/si-10/
- **SI-15 Reference:** https://csf.tools/reference/nist-sp-800-53/r5/si/si-15/
- **SI-10 MITRE ATT&CK Mapping:** https://center-for-threat-informed-defense.github.io/mappings-explorer/external/nist/attack-8.2/domain-enterprise/nist-rev5/SI-10/

### NIST SP 800-218 — Secure Software Development Framework (SSDF) v1.1

Developed in response to Executive Order 14028. Organizes practices into four groups:

1. **Prepare the Organization (PO)** — people, processes, technology readiness
2. **Protect the Software (PS)** — tamper protection
3. **Produce Well-Secured Software (PW)** — minimize vulnerabilities (this is where sanitization lives)
4. **Respond to Vulnerabilities (RV)** — identification, remediation, root cause

**PW.6: Configure the Compilation, Interpreter, and Build Processes to Improve Executable Security** — aims to decrease vulnerabilities before testing. Task PW.6.1 directs use of compiler/interpreter/build tools with security-improving features.

**Revision 1.2 (Draft)** published December 17, 2025 — updating to SSDF Version 1.2.

- **SP 800-218 (final):** https://csrc.nist.gov/pubs/sp/800/218/final
- **SP 800-218 PDF:** https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-218.pdf
- **SP 800-218 Rev 1.2 (draft):** https://csrc.nist.gov/pubs/sp/800/218/r1/ipd
- **CISA SSDF Resource:** https://www.cisa.gov/resources-tools/resources/nist-sp-800-218-secure-software-development-framework-v11-recommendations-mitigating-risk-software

### Other NIST References

- **NVD/CWE Alignment:** NIST's vulnerability database maps to CWE. Key sanitization-failure CWEs: CWE-20 (Improper Input Validation), CWE-79 (XSS), CWE-89 (SQL Injection), CWE-116 (Improper Encoding/Escaping).
- **SP 800-64 / SP 800-160:** System lifecycle-oriented, touch on secure coding principles.

---

## DISA STIGs

### Application Security and Development STIG (ASD STIG)

The primary STIG for application code and architecture (not infrastructure config). Derived from NIST 800-53. Current version is **v4 r11**.

Key input validation / sanitization controls:

| APSC-DV ID | CAT | Requirement |
|---|---|---|
| APSC-DV-002510 | I | Application must protect from command injection |
| APSC-DV-002520 | II | Application must protect from XSS |
| APSC-DV-002530 | II | Application must validate all input |
| APSC-DV-002560 | I | Application must not be subject to input handling vulnerabilities |
| APSC-DV-002590 | I | Application must not be vulnerable to overflow attacks |

Related controls cover race conditions (APSC-DV-001995), error handling (APSC-DV-003235), coding standards (APSC-DV-003215), security testing (APSC-DV-003150), code review (APSC-DV-003170), and flaw remediation (APSC-DV-003210).

Documentation requirements (Section 2.1): coding standards, vulnerability scan reports, and automated code review results must be available for security assessment — sanicode output could directly feed this requirement.

- **DISA STIGs Portal:** https://www.cyber.mil/stigs/
- **ASD STIG Viewer:** https://www.stigviewer.com/stigs/application_security_and_development

### Related STIGs

- **Web Server STIGs (Apache, IIS, nginx):** Server-level protections — WAF integration, header sanitization, request filtering.
- **Database STIGs:** Parameterized queries, stored procedure security (sanitization-adjacent).

---

## CWE / SANS Top 25

The CWE Top 25 Most Dangerous Software Weaknesses is updated annually by MITRE, compiled from CVE root cause mappings.

### 2025 CWE Top 25 (released December 2025)

Compiled from 39,080 CVEs. Top sanitization-relevant entries:

| Rank | CWE | Name |
|---|---|---|
| 1 | CWE-79 | Cross-site Scripting (XSS) |
| 2 | CWE-89 | SQL Injection |
| 3 | CWE-352 | Cross-Site Request Forgery |
| 10 | CWE-94 | Code Injection |

Other sanitization-relevant CWEs in the broader list:

- **CWE-20:** Improper Input Validation
- **CWE-78:** OS Command Injection
- **CWE-116:** Improper Encoding or Escaping of Output
- **CWE-77:** Command Injection
- **CWE-917:** Expression Language Injection

Sanicode should map findings to CWE IDs — this is the lingua franca across all standards.

- **CWE Top 25 (2025):** https://cwe.mitre.org/top25/archive/2025/2025_cwe_top25.html
- **CWE Top 25 (2024):** https://cwe.mitre.org/top25/archive/2024/2024_cwe_top25.html
- **CISA Alert (2025):** https://www.cisa.gov/news-events/alerts/2025/12/11/2025-cwe-top-25-most-dangerous-software-weaknesses
- **SANS CWE Top 25:** https://www.sans.org/top25-software-errors

---

## PCI DSS

### PCI DSS 4.0 Requirement 6 — Develop and Maintain Secure Systems

New controls became **fully enforced March 31, 2025**. Relevant to any application handling payment data.

Key sub-requirements for sanitization:

- **6.2.2:** Developer training at least annually on secure coding, including injection prevention
- **6.2.3:** Custom software code review prior to production release
- **6.2.4:** Software development must prevent/mitigate common attacks including injection, data manipulation, cryptographic weakness, access control bypass
- **6.5.1:** Addresses injection flaws specifically — SQL injection, OS command injection, LDAP/XPath injection
- **6.4.3 (new in 4.0):** Payment page script integrity — authorize each script, ensure integrity, maintain inventory (addresses Magecart-style attacks)

PCI DSS recommends using OWASP Guidelines, SANS CWE Top 25, and CERT Secure Coding as frameworks.

- **PCI DSS Requirement 6 Guide:** https://pcidssguide.com/pci-dss-requirement-6/
- **PCI DSS 4.0 Details:** https://www.herodevs.com/blog-posts/pci-dss-4-0-requirement-6-how-to-develop-and-maintain-secure-systems-and-software

---

## SEI CERT Coding Standards

Carnegie Mellon's language-specific secure coding standards. NIST references them, and PCI DSS recommends them.

Official standards exist for: **C, C++, Java, Android, Perl**. **No official Python standard exists** — this is a gap sanicode could help fill by applying CERT principles to Python codebases.

The Java standard includes **"Guidelines 00: Input Validation and Data Sanitization (IDS)"** which maps to CWE-1134 and covers the core sanitization patterns.

Key structure: each rule has a unique identifier, non-compliant code examples, compliant solutions, risk assessment, and related guidelines.

- **SEI CERT Coding Standards Wiki:** https://wiki.sei.cmu.edu/confluence/display/seccode
- **CWE Category for CERT Java IDS:** https://cwe.mitre.org/data/definitions/1134.html

---

## Existing Tooling Landscape

### Python SAST Tools

| Tool | Type | Languages | Key Strengths | Limitations |
|---|---|---|---|---|
| **Bandit** | OSS SAST | Python only | Fast, simple, 68 built-in checks, AST-based | No dependency scanning, no cross-function data flow |
| **Semgrep** | OSS SAST | Multi-language | Custom YAML rules, community registry, AI triage | Free tier limited, commercial features gated |
| **SonarQube** | OSS/Commercial | Multi-language | Broad ecosystem, 40% dev adoption | Heavy, Java-based, complex setup |
| **CodeQL** | Free/Commercial | Multi-language | Semantic code analysis, GitHub integration | GitHub-dependent, query language learning curve |
| **Pylint** | OSS Linter | Python | Broad code quality checks | Not security-focused |
| **Safety/pip-audit** | OSS SCA | Python deps | Known vulnerability detection in dependencies | Only checks dependencies, not code |

### Where Sanicode Fits

Existing tools do **pattern matching** (Bandit finds `subprocess.call(shell=True)`) or **rule-based analysis** (Semgrep matches YAML patterns). None of them:

1. Build a persistent **knowledge graph** of data flows across the codebase
2. Use **LLM reasoning** to understand context (is this input actually user-controlled? does this sanitization function actually cover the threat?)
3. Map findings to **multiple compliance frameworks** simultaneously (OWASP ASVS + NIST 800-53 + ASD STIG)
4. Provide **natural language recommendations** with framework-specific justification

Sanicode is complementary to these tools, not a replacement. It adds the contextual understanding layer.

---

## AI-Assisted Code Analysis — State of the Art

### Google DeepMind CodeMender (October 2025)

AI agent for code security that detects failures and self-corrects based on LLM judge feedback. All patches reviewed by humans before upstream submission. Google has begun submitting patches to critical open-source libraries. Plans to release as a general tool.

- https://deepmind.google/blog/introducing-codemender-an-ai-agent-for-code-security/

### Knowledge Graph Approaches

**CTIKG (LLM-Powered Knowledge Graphs for Cybersecurity):** Uses multiple LLM agents with dual memory design to build security-oriented knowledge graphs from CTI articles. Achieves 86.88% precision, 30%+ improvement over prior art. Relevant architecture pattern for sanicode's knowledge graph construction.

- https://openreview.net/forum?id=DOMP5AgwQz

**GALLa (Graph Aligned LLMs for Source Code Understanding):** ACL 2025 paper on aligning LLMs with graph structures for code understanding. Directly relevant to sanicode's approach.

### Multi-Agent Security Analysis

**AutoSafeCoder:** Multi-agent framework securing LLM code generation through static analysis and fuzz testing. Relevant pattern for sanicode's analysis pipeline.

**Agentic Security:** Open-source LLM vulnerability scanner for agent workflows. Uses reinforcement learning for adaptive probing.

- https://github.com/msoedov/agentic_security

### Key Research Papers

- "Gistify! Codebase-Level Understanding via Runtime Execution" (2025)
- "CodeNav: Beyond tool-use to using real-world codebases with LLM agents"
- "HexaCoder: Secure Code Generation via Oracle-Guided Synthetic Training Data"
- Survey: "A Survey on Code Generation with LLM-based Agents" (arXiv, Sept 2025)

### Awesome Lists

- https://github.com/tmylla/Awesome-LLM4Cybersecurity
- https://github.com/codefuse-ai/Awesome-Code-LLM

---

## Standards Cross-Reference Matrix

How sanitization requirements map across frameworks — this is what sanicode's recommendation engine should reference:

| Sanitization Domain | OWASP ASVS 5.0 | NIST 800-53 | ASD STIG | CWE | PCI DSS 4.0 |
|---|---|---|---|---|---|
| Input Validation | V1 (general) | SI-10 | APSC-DV-002530 | CWE-20 | 6.2.4, 6.5.1 |
| SQL Injection | V1.2 (injection prev.) | SI-10 | APSC-DV-002510 | CWE-89 | 6.5.1 |
| XSS | V1.2 | SI-10, SI-15 | APSC-DV-002520 | CWE-79 | 6.5.1 |
| OS Command Injection | V1.2.5 | SI-10 | APSC-DV-002510 | CWE-78 | 6.5.1 |
| Output Encoding | V1 (encoding) | SI-15 | APSC-DV-002520 | CWE-116 | 6.2.4 |
| SSRF | V1 (SSRF) | SI-10 | APSC-DV-002560 | CWE-918 | 6.2.4 |
| Template Injection | V1 (template) | SI-10 | APSC-DV-002560 | CWE-94 | 6.2.4 |
| ReDoS | V1 (ReDoS) | — | — | CWE-1333 | — |
| LDAP Injection | V1.2 | SI-10 | APSC-DV-002510 | CWE-90 | 6.5.1 |
| Buffer Overflow | — | SI-10 | APSC-DV-002590 | CWE-120 | 6.2.4 |

---

## Architecture Notes

### Proposed CLI Interface

```
sanicode config          # Configure LLM backend (OpenAI, Anthropic, Ollama, etc.)
sanicode scan <path>     # Scan a codebase, build knowledge graph
sanicode report          # Generate compliance-mapped report
sanicode recommend       # Output prioritized recommendations
sanicode graph           # Inspect/export the knowledge graph
sanicode watch           # Continuous monitoring mode (future)
```

### Knowledge Graph Design Considerations

The knowledge graph should capture:

- **Entry points:** HTTP handlers, CLI argument parsers, file readers, env var access, message queue consumers
- **Data flow paths:** How untrusted data moves through functions, modules, and services
- **Sanitization points:** Where validation/encoding/escaping occurs (and what type)
- **Sink points:** Where data reaches interpreters (SQL, shell, template engines, HTML output, file system)
- **Trust boundaries:** Where data crosses from untrusted to trusted contexts
- **Framework patterns:** ORM usage, template engine auto-escaping, parameterized query usage

### Compliance Mapping Engine

Each finding should include:

- CWE ID (universal identifier)
- OWASP ASVS requirement(s) with level (L1/L2/L3)
- NIST 800-53 control(s)
- ASD STIG check ID(s) and category (CAT I/II/III)
- PCI DSS requirement (if applicable)
- Natural language explanation and remediation guidance
- Confidence score (how certain the LLM is about the finding)

### Technology Stack Considerations

- **Language:** Python 3.10+
- **Distribution:** PyPI package
- **CLI Framework:** Click or Typer
- **AST Parsing:** `ast` module (Python), tree-sitter (multi-language future)
- **Knowledge Graph:** NetworkX (in-memory) or Neo4j (persistent, for large codebases)
- **LLM Integration:** LiteLLM or similar for multi-provider support
- **Output Formats:** Markdown, JSON, SARIF (for IDE/CI integration)
- **Config:** TOML (`sanicode.toml` or `~/.config/sanicode/config.toml`)

### SARIF Output

SARIF (Static Analysis Results Interchange Format) is the industry standard for SAST tool output. Supporting SARIF means sanicode results can be consumed by GitHub Code Scanning, VS Code SARIF Viewer, Azure DevOps, and other tools in the ecosystem. This should be a priority for interoperability.

---

## Disconnected / Air-Gapped Architecture

> *"The sky is open but the environment is closed."*

### The Constraint

Much of sanicode's target audience operates in disconnected (air-gapped) environments — classified networks, SCIF environments, FedRAMP-bounded clusters, or simply enterprise networks with no egress. This means:

- No cloud LLM APIs (no OpenAI, no Anthropic, no Google endpoints)
- No runtime package downloads (no `pip install` at scan time)
- No telemetry, no phone-home, no license servers
- All model weights, dependencies, and compliance data must be pre-staged
- Container images must be mirrored to internal registries

This isn't a limitation to work around — it's a first-class deployment target. The architecture must be **offline-native, not offline-compatible**.

### Deployment Models

Sanicode should support three deployment models, all from the same codebase:

#### Model 1: Local CLI (Developer Workstation)

The simplest mode. `pip install sanicode` on a workstation, point it at a local vLLM/Ollama instance or run in degraded mode with no LLM at all.

```
Developer laptop/workstation
├── sanicode (pip-installed CLI)
├── vLLM or Ollama (local model serving)
│   └── Model weights on local disk
└── Target codebase
```

**When to use:** Individual developer scanning, pre-commit checks, small codebases, environments where OpenShift isn't available.

**LLM options:** Ollama (simplest local setup), vLLM (better throughput), or no LLM (deterministic-only mode).

#### Model 2: Containerized Service on OpenShift / OpenShift AI

Sanicode runs as a long-lived service (Pod/Deployment) on OpenShift, with the LLM served by vLLM via OpenShift AI's KServe integration. Exposes a REST/gRPC API that other tools (CI pipelines, refactoring agents, IDE plugins) can call.

```
OpenShift Cluster (disconnected)
├── sanicode-api (Deployment)
│   ├── REST API (FastAPI)      → /api/v1/scan, /api/v1/report, /api/v1/graph
│   ├── gRPC API (optional)     → For high-throughput agent-to-agent comms
│   └── Knowledge graph store   → PVC-backed persistent storage
├── vLLM InferenceService (KServe on OpenShift AI)
│   ├── ServingRuntime           → registry.redhat.io/rhaiis/vllm-cuda-rhel9
│   ├── Model weights            → PVC (pre-staged via sneakernet/oc cp)
│   └── Tokenizer files          → Co-located on PVC (critical for disconnected)
├── Redis or PostgreSQL (optional)
│   └── Scan result caching, knowledge graph persistence
└── Internal registry
    └── All images mirrored via oc-mirror
```

**When to use:** Team/org-wide scanning, CI/CD pipeline integration, serving as a backend for refactoring agents, large codebases that benefit from GPU-accelerated LLM analysis.

**Key OpenShift AI integration points:**

- **Model serving:** KServe `ServingRuntime` + `InferenceService` CRDs deploy vLLM with GPU support. Model weights loaded from PVC, not S3 (no egress needed). The `ModelCar` pattern can serve models directly from OCI container images — meaning model weights can be distributed through the same image registry pipeline as application images.
- **Image mirroring:** Use `oc-mirror` or `oc image mirror` to pre-stage `quay.io/modh/vllm:rhoai-2.20-cuda` and sanicode's own image into the disconnected registry. Reference: [Deploying Validated Models in Disconnected OpenShift AI](https://ai-on-openshift.io/odh-rhoai/deploy-validated-models-on-disconnected/)
- **Tokenizer gotcha:** In disconnected environments, some model runtimes attempt to download tokenizer vocabulary files at startup. These must be pre-staged on the PVC alongside model weights. This is a known issue documented in [Deploying GPT-OSS Models in Disconnected Environments](https://medium.com/@yakovbeder/deploying-gpt-oss-models-on-red-hat-openshift-ai-in-disconnected-environments-c92a2218c237).
- **Autoscaling:** KServe + KEDA provides metric-based autoscaling for the vLLM pods. For sanicode's bursty workload (scan requests come in batches), KEDA outperforms Knative's default concurrency-based scaling. Reference: [Autoscaling vLLM with OpenShift AI](https://developers.redhat.com/articles/2025/11/26/autoscaling-vllm-openshift-ai-model-serving)
- **Multi-node inference:** For larger models (70B+), OpenShift AI supports `vllm-multinode-runtime` across multiple GPU nodes.

#### Model 3: Tekton Pipeline Task / CI Integration

Sanicode runs as a Tekton Task (or GitHub Actions step, or Jenkins stage) within the build pipeline. Scans code at PR/merge time, outputs SARIF for PR annotations, and blocks merge if CAT I findings are present.

```
Tekton Pipeline
├── git-clone Task
├── sanicode-scan Task (containerized)
│   ├── Reads source from workspace PVC
│   ├── Calls vLLM InferenceService (cluster-internal)
│   └── Outputs SARIF to workspace PVC
├── sanicode-gate Task
│   └── Fails pipeline if CAT I / CWE Top 25 findings exist
└── build-and-deploy Tasks (proceed if gate passes)
```

**When to use:** Automated enforcement in CI/CD, shift-left scanning, gating deployments on compliance.

### Pressure Testing the CLI-Only Assumption

A pure CLI has limitations that become apparent in real deployments:

| Concern | CLI-Only Problem | Service Model Solution |
|---|---|---|
| **Scan duration** | Large codebases take minutes-to-hours. CLI blocks the terminal. | Service accepts async scan jobs, returns results via polling/webhook. |
| **Knowledge graph persistence** | Graph rebuilt on every scan (expensive LLM calls repeated). | Service maintains persistent graph, incremental updates on code changes. |
| **Multi-user access** | Each developer runs their own scan with their own graph. No shared understanding. | Shared service provides org-wide knowledge graph. Teams see the same findings. |
| **Agent integration** | A refactoring agent would need to shell out to `sanicode` and parse stdout. | Agent calls REST/gRPC API, gets structured responses, iterates programmatically. |
| **Resource efficiency** | Every CLI invocation loads models, tokenizers, graph state. | Service keeps models warm. vLLM on GPU stays loaded. Graph stays in memory. |
| **Audit trail** | No record of past scans unless the user saves reports. | Service stores scan history, diffs, trend data. Supports ATO evidence collection. |

**Conclusion:** The CLI should be the *client*, not the whole system. Design:

```
sanicode scan <path>         →  Can run locally OR submit to remote sanicode-api
sanicode report              →  Can read local results OR pull from remote
sanicode config              →  Configure local mode, remote mode, or hybrid
sanicode serve               →  Start the API server (for containerized deployment)
```

The `sanicode` CLI is always the user-facing interface, but it can operate in three modes:
1. **Local mode:** Everything runs in-process (small codebases, no service needed)
2. **Remote mode:** CLI is a thin client to a sanicode-api service on OpenShift
3. **Hybrid mode:** AST parsing and graph construction run locally, LLM calls go to remote vLLM

### Model Selection for Disconnected Environments

Not all models are created equal for air-gapped code analysis. Key considerations:

#### Red Hat Granite Models (Recommended Primary)

IBM Granite code models are the natural fit for a Red Hat ecosystem tool:

- **Granite Code:** Trained on 116 programming languages, decoder-only, purpose-built for code generation and understanding. Available as 3B, 8B, 20B, and 34B variants. Apache 2.0 licensed.
- **Granite 4.0 Nano:** Compact models designed for edge/local/disconnected use. Runnable on vLLM, llama.cpp, and MLX. Apache 2.0, ISO 42001 aligned, cryptographically signed. Reference: [IBM Granite Code Models](https://github.com/ibm-granite/granite-code-models)
- **vLLM compatibility:** Granite 3.1+ confirmed working on vLLM. Granite 4.0 uses hybrid Mamba-2/Transformer architecture with Day 0 vLLM support. Reference: [vLLM Granite 3.1 PR](https://github.com/vllm-project/vllm/pull/11307)
- **Why Granite for sanicode:** Red Hat ships these, customers in disconnected Red Hat environments likely already have access to Granite weights through their Red Hat subscription. Reduces supply chain friction.

#### Other Viable Models

| Model | Size | Strengths | Disconnected Notes |
|---|---|---|---|
| **Granite Code 8B/20B** | 8–20B | Code-focused, Red Hat ecosystem | Likely pre-staged in RHOAI environments |
| **Granite 4.0 Nano** | ~2–4B | Edge-optimized, fast | Runs on CPU if needed |
| **DeepSeek-Coder-V2** | 16B/236B | Strong code reasoning | Large; needs significant GPU |
| **CodeLlama** | 7B/13B/34B | Meta's code model, well-tested | Good vLLM support, Apache-adjacent license |
| **Mistral/Mixtral** | 7B/8x7B/8x22B | Strong general reasoning | Good for compliance mapping, natural language |
| **StarCoder2** | 3B/7B/15B | BigCode, multi-language | Good lightweight option |
| **Llama 3.1** | 8B/70B/405B | Best general reasoning at scale | 70B is the sweet spot for disconnected GPU clusters |

#### Tiered Model Strategy

Design sanicode to use different models for different tasks:

- **Tier 1 (fast/cheap):** Granite Nano or StarCoder2 3B for AST-adjacent pattern classification. Can run on CPU.
- **Tier 2 (balanced):** Granite Code 8B or CodeLlama 13B for data flow analysis and sanitization assessment. Needs 1 GPU.
- **Tier 3 (deep reasoning):** Llama 3.1 70B or Mixtral 8x22B for complex compliance mapping and natural language report generation. Needs multi-GPU or quantized.

Sanicode config should allow specifying different endpoints for different task tiers:

```toml
[llm]
# Tier 1: Fast classification tasks
fast_endpoint = "http://vllm-granite-nano:8000/v1"
fast_model = "granite-4.0-nano"

# Tier 2: Code analysis tasks
analysis_endpoint = "http://vllm-granite-code:8000/v1"
analysis_model = "granite-code-8b"

# Tier 3: Deep reasoning tasks
reasoning_endpoint = "http://vllm-llama:8000/v1"
reasoning_model = "llama-3.1-70b-instruct"
```

### What Must Be Bundled (Zero-Egress Requirement)

Everything the tool needs at runtime must ship with the package or container image:

| Component | Distribution Method | Size Estimate |
|---|---|---|
| Sanicode Python package | PyPI wheel (pre-downloaded) or baked into container | ~5–20 MB |
| All Python dependencies | Vendored in wheel or container layer | ~100–200 MB |
| Compliance cross-ref database | JSON/TOML files in package `data/` dir | ~2–5 MB |
| Rule definitions (Bandit-style) | YAML files in package `rules/` dir | ~1–5 MB |
| Prompt templates | Jinja2 templates in package `prompts/` dir | ~500 KB |
| Remediation knowledge base | Distilled OWASP/NIST guidance (attributed) | ~5–10 MB |
| tree-sitter grammars | Pre-compiled `.so` files per language | ~50 MB (all languages) |
| Model weights (NOT in package) | Separate PVC / model registry / ModelCar OCI image | 4–140 GB depending on model |

**Model weights are never bundled with sanicode itself.** They're a separate supply chain artifact managed by the cluster operator / model admin.

### Degraded Mode (No LLM Available)

Sanicode MUST produce useful output even with no LLM configured. This is critical for:

- Initial adoption (before vLLM is set up)
- Environments where GPU resources are scarce
- CI pipeline runs where LLM latency is unacceptable
- Trust building (show deterministic results first, then enhance with LLM)

**Deterministic-only capabilities (no LLM needed):**
- AST-based pattern matching (Bandit-equivalent checks)
- Known-bad function detection (`eval()`, `subprocess.call(shell=True)`, raw SQL string formatting)
- Import analysis (which frameworks/libraries are in use)
- Data flow graph construction (source → sink tracing via AST)
- Compliance mapping (CWE → ASVS → NIST → STIG lookups)
- SARIF output generation

**LLM-enhanced capabilities (requires vLLM endpoint):**
- Context-aware assessment ("is this `eval()` actually reachable with user input?")
- Sanitization adequacy judgment ("does this `bleach.clean()` call actually prevent the identified CWE?")
- Natural language remediation guidance
- Cross-file data flow analysis beyond AST reach
- Framework-aware pattern recognition (Django ORM safe vs. raw SQL)
- Confidence scoring and false-positive reduction

### Container Build Strategy

```dockerfile
# Multi-stage build
FROM registry.redhat.io/ubi9/python-311 AS builder
COPY . /app
RUN pip install --no-deps --target /app/vendor -r requirements.txt
RUN pip wheel --no-deps -w /app/dist .

FROM registry.redhat.io/ubi9/python-311-minimal AS runtime
COPY --from=builder /app/dist/*.whl /tmp/
COPY --from=builder /app/vendor /app/vendor
RUN pip install --no-index --find-links /tmp/ sanicode

# UBI9-minimal for smallest attack surface
# No internet access needed at runtime
# All deps vendored at build time

EXPOSE 8080
ENTRYPOINT ["sanicode"]
CMD ["serve", "--host", "0.0.0.0", "--port", "8080"]
```

Key decisions:
- **UBI9 base:** Required for Red Hat ecosystem / OpenShift certification. Minimal variant for smaller image.
- **Vendored dependencies:** All Python deps baked in at build time. No pip calls at runtime.
- **Dual entrypoint:** Same image works as `sanicode serve` (API mode) or `sanicode scan` (one-shot CLI in Tekton).

### API Design for Agent Consumption

A refactoring agent (or migration tool like Konveyor/MTA) needs to interact with sanicode programmatically. The API should be designed for machine consumption first:

```
POST /api/v1/scan
  Body: { "repo_path": "/workspace/src", "config": {...} }
  Returns: { "scan_id": "uuid", "status": "accepted" }

GET /api/v1/scan/{scan_id}
  Returns: { "status": "complete", "summary": {...}, "findings_count": 42 }

GET /api/v1/scan/{scan_id}/findings
  Returns: { "findings": [...], "format": "sarif" }
  Query params: ?severity=high&cwe=79,89&framework=owasp-asvs

GET /api/v1/scan/{scan_id}/graph
  Returns: Knowledge graph in JSON-LD or NetworkX JSON format

POST /api/v1/analyze
  Body: { "code_snippet": "...", "context": "django view handler" }
  Returns: { "findings": [...], "recommendations": [...] }
  (Lightweight single-snippet analysis for IDE/agent use)

GET /api/v1/compliance/map?cwe=89
  Returns: { "owasp_asvs": [...], "nist_800_53": [...], "asd_stig": [...] }
  (Pure lookup, no LLM needed)

GET /api/v1/health
  Returns: { "status": "ok", "llm_available": true, "model": "granite-code-8b" }
```

### Integration Points for Refactoring Agents

How sanicode fits into the broader toolchain:

```
┌──────────────────────────────────────────────────────┐
│  Refactoring / Migration Agent                       │
│  (Konveyor, MTA, custom agent, Claude Code, etc.)    │
│                                                      │
│  1. "I'm migrating this Java EE app to Quarkus"     │
│  2. Calls sanicode: "What sanitization issues exist  │
│     in the current codebase?"                        │
│  3. Gets findings + knowledge graph                  │
│  4. Uses graph to understand data flow context        │
│  5. Generates migrated code that ALSO fixes           │
│     sanitization gaps                                │
│  6. Calls sanicode again: "Re-scan the new code"     │
│  7. Validates that findings are resolved              │
└──────────────┬───────────────────────┬───────────────┘
               │ REST/gRPC            │ SARIF
               ▼                      ▼
┌──────────────────────┐  ┌────────────────────────────┐
│  sanicode-api        │  │  CI/CD Pipeline (Tekton)   │
│  (OpenShift Pod)     │  │  Gate: block if CAT I      │
└──────────┬───────────┘  └────────────────────────────┘
           │ OpenAI-compat API
           ▼
┌──────────────────────┐
│  vLLM (OpenShift AI) │
│  Granite Code 8B     │
└──────────────────────┘
```

### References — Disconnected OpenShift AI Deployment

- [Benchmarking with GuideLLM in Air-Gapped OpenShift Clusters](https://developers.redhat.com/articles/2025/09/15/benchmarking-guidellm-air-gapped-openshift-clusters) — Red Hat Developer, Sep 2025
- [Deploying Validated Models in Disconnected OpenShift AI](https://ai-on-openshift.io/odh-rhoai/deploy-validated-models-on-disconnected/) — AI on OpenShift
- [Deploying GPT-OSS Models in Disconnected Environments](https://medium.com/@yakovbeder/deploying-gpt-oss-models-on-red-hat-openshift-ai-in-disconnected-environments-c92a2218c237) — Yakov Beder, Nov 2025
- [How to Deploy Language Models with Red Hat OpenShift AI](https://developers.redhat.com/articles/2025/09/10/how-deploy-language-models-red-hat-openshift-ai) — Red Hat Developer, Sep 2025
- [Autoscaling vLLM with OpenShift AI](https://developers.redhat.com/articles/2025/10/02/autoscaling-vllm-openshift-ai) — Red Hat Developer, Oct 2025
- [vLLM on CPUs with OpenShift (GPU-free inference)](https://developers.redhat.com/articles/2025/06/17/how-run-vllm-cpus-openshift-gpu-free-inference) — Red Hat Developer, Jun 2025
- [IBM Granite Code Models](https://github.com/ibm-granite/granite-code-models) — GitHub
- [IBM Granite Docs — Code Models](https://www.ibm.com/granite/docs/models/code) — IBM

---

## Operator vs. Application: Distribution & Lifecycle Strategy

### The Decision Framework

An OpenShift Operator is the right *end state* but the wrong *starting point*. The path is: **Application → Helm Chart → Operator**.

| Criteria | Application (kustomize) | Helm Chart | Operator |
|---|---|---|---|
| **Development effort** | Low | Low-Medium | High (Go + controller-runtime) |
| **Distribution** | `oc apply -f` | `helm install` | OLM / OperatorHub |
| **Lifecycle management** | Manual | Template-driven | Reconciler loop (self-healing) |
| **Custom Resources** | None | None | Full CRD ecosystem |
| **Config drift remediation** | None | None | Automatic (controller watches) |
| **Dependency management** | Manual | Manual | OLM resolves (e.g., requires OpenShift AI) |
| **Agent integration** | HTTP API calls | HTTP API calls | `oc apply` CR + `oc wait` (K8s-native) |
| **Audit trail** | Application logs | Application logs | K8s events + CR status history |
| **OperatorHub presence** | No | No | Yes |

Reference: [Helm and Operators on OpenShift, Part 1](https://www.redhat.com/en/blog/helm-and-operators-on-openshift-part-1) / [Part 2](https://www.redhat.com/en/blog/helm-and-operators-on-openshift-part-2)

### Why the Operator Pattern Fits Sanicode Long-Term

The Operator reconciler loop is the killer feature — not deployment, but *continuous reconciliation*. A sanicode Operator can:

1. **Watch for code changes** — trigger scans when a BuildConfig or ImageStream updates
2. **Manage scan jobs** — spawn Kubernetes Jobs, track completion, retry failures
3. **Detect regression** — compare current scan to previous, alert if fixed findings reappear
4. **Auto-discover LLM endpoints** — find InferenceService CRs from OpenShift AI in the cluster
5. **Express dependencies via OLM** — "sanicode requires OpenShift AI >= 2.20" resolved automatically
6. **Self-heal** — if someone deletes the sanicode deployment or corrupts config, Operator restores it
7. **GitOps-friendly** — scan configs live in git alongside application manifests, ArgoCD syncs them

Reference: [Operator Pattern: Kubernetes/OpenShift](https://medium.com/operators/operator-pattern-kubernetes-openshift-380ddc6a147c)

### Proposed Custom Resource Definitions

Design CRDs from day one, even before building the Operator. The CRDs define the user experience:

```yaml
# SanicodeConfig — Cluster-wide defaults (one per cluster)
apiVersion: sanicode.io/v1alpha1
kind: SanicodeConfig
metadata:
  name: sanicode-config
spec:
  llm:
    # Tiered model endpoints
    fast:
      inferenceService: granite-nano       # auto-discovered from OpenShift AI
    analysis:
      inferenceService: granite-code-8b
    reasoning:
      inferenceService: llama-3-1-70b
  defaults:
    profile: federal-moderate
    outputFormats: [sarif, markdown]
  storage:
    graphBackend: postgresql               # or networkx-file for simple setups
    resultsPVC: sanicode-results
```

```yaml
# SanicodeScan — "Scan this code" (per-namespace, per-app)
apiVersion: sanicode.io/v1alpha1
kind: SanicodeScan
metadata:
  name: my-app-scan
  namespace: my-app
spec:
  source:
    # Option A: Git repo
    gitRepo: https://internal-gitlab.example.com/team/my-app.git
    branch: main
    # Option B: PVC with code already checked out (Tekton workspace)
    # pvcName: source-code
    # path: /workspace/source
  profile: federal-moderate
  schedule: "0 2 * * *"                    # cron for recurring scans (optional)
  llm:
    tier: analysis                         # which tier to use from SanicodeConfig
  gate:
    failOn: [cat-i]                        # block if CAT I findings exist
    maxFindings: 50                         # or raw count threshold
status:
  phase: Completed                          # Pending → Running → Completed / Failed
  lastScanTime: "2026-02-20T02:00:00Z"
  findingsCount: 12
  catISummary: 0
  catIISummary: 8
  catIIISummary: 4
  reportRef: my-app-scan-report-20260220
  conditions:
    - type: GatePassed
      status: "True"
```

```yaml
# SanicodeProfile — Reusable compliance profile
apiVersion: sanicode.io/v1alpha1
kind: SanicodeProfile
metadata:
  name: federal-moderate
spec:
  description: "NIST 800-53 Moderate + ASD STIG CAT I/II + OWASP ASVS L2"
  frameworks:
    - name: nist-800-53
      baseline: moderate
      controls: [SI-10, SI-15, SA-11]
    - name: asd-stig
      categories: [cat-i, cat-ii]
    - name: owasp-asvs
      level: 2
      chapters: [V1]
  cweIncludes: [20, 77, 78, 79, 89, 94, 116, 918]
  severityThreshold: medium
```

```yaml
# SanicodeException — "Accept this finding with justification" (ATO-critical)
apiVersion: sanicode.io/v1alpha1
kind: SanicodeException
metadata:
  name: accept-eval-in-config-parser
  namespace: my-app
  annotations:
    sanicode.io/approved-by: wjackson@redhat.com
    sanicode.io/approved-date: "2026-02-20"
    sanicode.io/review-by: "2026-08-20"         # 6-month review
spec:
  scanRef: my-app-scan
  findingID: "CWE-95-config-parser-line-42"
  cwe: 95
  justification: |
    The eval() call in config_parser.py:42 only processes TOML values
    from a file that is root-owned (mode 0600) and mounted read-only
    from a ConfigMap. No user-controlled input reaches this path.
    Mitigated by file permission controls (AC-3) and monitoring (SI-4).
  mitigatingControls: [AC-3, SI-4]
  expiresAt: "2026-08-20T00:00:00Z"             # forces re-review
status:
  active: true
  acknowledgedBy: security-team
```

The `SanicodeException` CR is critical for ATO (Authority to Operate) workflows. Auditors need to see not just what was found, but what was *accepted*, by whom, with what justification, and when it expires. Making this a Kubernetes object means it's:
- **Versioned** — `oc get sanicodeexception -o yaml` shows history
- **RBAC-controlled** — only `security-team` role can create exceptions
- **Auditable** — Kubernetes audit logs capture who applied it and when
- **Expiring** — the Operator can re-flag findings when exceptions expire
- **GitOps-managed** — exceptions live in the same repo as the application, reviewed in PRs

### Staged Implementation Path

**Phase 1 — Python Application (MVP)**
- PyPI package with CLI + FastAPI server
- Deploy on OpenShift with kustomize overlays
- REST API for agent integration
- No CRDs, no Operator — just a Pod that scans code

**Phase 2 — Helm Chart**
- Proper packaging: ServingRuntime templates, PVC provisioning, Route/Ingress
- Values file for environment-specific config (disconnected registry URLs, model names)
- `helm install sanicode ./charts/sanicode --values disconnected-values.yaml`

**Phase 3 — Lightweight Python Operator (kopf)**
- Use [kopf](https://kopf.readthedocs.io/) (Kubernetes Operator Pythonic Framework) to build the reconciler in Python — same language as the rest of the project
- Implement CRDs: SanicodeConfig, SanicodeScan, SanicodeProfile, SanicodeException
- Spawn scan Jobs, watch InferenceService CRs, manage lifecycle
- This avoids the Go learning curve while proving the Operator pattern works

**Phase 4 — Production Operator (Go or Helm-based via Operator SDK)**
- If kopf proves insufficient (performance, RBAC granularity, OLM packaging), convert to Go using Operator SDK
- Or use Operator SDK's Helm-to-Operator path: wrap the Phase 2 Helm chart, add CRD controllers
- Publish to OperatorHub / Red Hat Certified Operator catalog
- Express OLM dependency on OpenShift AI operator

Reference: [Build Kubernetes Operators from Helm Charts in 5 Steps](https://www.openshift.com/blog/build-kubernetes-operators-from-helm-charts-in-5-steps), [IBM OpenShift AI Toolkit Operator](https://github.com/IBM/OpenShift-AI-Toolkit-Operator)

### kopf vs. Go Operator SDK Trade-offs

| Aspect | kopf (Python) | Operator SDK (Go) |
|---|---|---|
| **Language** | Python (same as sanicode core) | Go (new dependency) |
| **Learning curve** | Low for Python devs | Moderate (controller-runtime) |
| **OLM packaging** | Possible but less tooling | First-class OLM support |
| **Performance** | Fine for 10s of CRs | Better for 1000s of CRs |
| **Ecosystem** | Smaller community | Large, Red Hat-backed |
| **Certification** | Not Red Hat certified path | Red Hat certified Operator path |
| **Sanicode fit** | Great for Phase 3 (proving CRD model) | Right for Phase 4 (production) |

### Where Sanicode Sits Relative to OpenShift AI

Sanicode is NOT a component of OpenShift AI — it's a *consumer* of it. The relationship:

```
┌─────────────────────────────────────────────────┐
│  OpenShift Cluster                              │
│                                                  │
│  ┌──────────────────────────────────────────┐   │
│  │  OpenShift AI (Operator)                 │   │
│  │  ├── KServe                              │   │
│  │  ├── vLLM ServingRuntime                 │   │
│  │  ├── InferenceService: granite-code-8b   │   │
│  │  └── Model weights on PVC               │   │
│  └──────────────────────────────────────────┘   │
│         ▲ OpenAI-compatible API                  │
│         │                                        │
│  ┌──────┴───────────────────────────────────┐   │
│  │  Sanicode (Operator or Application)      │   │
│  │  ├── SanicodeConfig CR                   │   │
│  │  ├── SanicodeScan CRs (per-app)         │   │
│  │  ├── SanicodeProfile CRs                │   │
│  │  ├── SanicodeException CRs              │   │
│  │  ├── Knowledge graph (PVC)              │   │
│  │  └── REST/gRPC API                      │   │
│  └──────────────────────────────────────────┘   │
│         ▲ REST API / CRs                         │
│         │                                        │
│  ┌──────┴───────────────────────────────────┐   │
│  │  Consumers                               │   │
│  │  ├── Tekton Pipelines (CI/CD gating)    │   │
│  │  ├── Refactoring agents (Konveyor/MTA)  │   │
│  │  ├── IDE plugins                        │   │
│  │  └── Developer CLI (sanicode scan)      │   │
│  └──────────────────────────────────────────┘   │
└─────────────────────────────────────────────────┘
```

The OLM dependency chain would be: `Sanicode Operator → requires → OpenShift AI Operator` — meaning installing sanicode from OperatorHub would automatically ensure OpenShift AI is present (or prompt the admin to install it).

---

## Observability, Pipelines & Dashboard Integration

### Design Principle

Sanicode should integrate with OpenShift AI platform components rather than reinventing them. Three components map naturally to sanicode's needs:

| Platform Component | Sanicode Use Case | Why Not Build Our Own |
|---|---|---|
| **MLflow** | Track scan runs as "experiments" — parameters, metrics, artifacts | Already deployed in many RHOAI environments; rich UI; model/artifact versioning built in |
| **Data Science Pipelines (KFP 2.x)** | Orchestrate multi-step scan workflows | Handles retry, parallelism, scheduling, DAG visualization; Kubernetes-native |
| **Grafana + Prometheus** | Real-time dashboards for findings, trends, compliance posture | Standard OpenShift observability stack; alerting built in; teams already use it |

### MLflow Integration — Scan Run Tracking

MLflow's experiment tracking isn't just for ML training — its abstractions (experiments, runs, parameters, metrics, artifacts) map perfectly to code scanning:

| MLflow Concept | Sanicode Mapping |
|---|---|
| **Experiment** | A codebase / repository being scanned |
| **Run** | A single scan execution |
| **Parameters** | Scan config: profile, LLM tier, commit SHA, branch |
| **Metrics** | findings_total, cat_i_count, cat_ii_count, cwe_coverage, scan_duration_sec, llm_calls_count |
| **Artifacts** | SARIF report, knowledge graph snapshot, markdown report |
| **Tags** | compliance_framework, scan_mode (local/remote/hybrid), model_used |
| **Model Registry** | (Future) Register the prompt templates + rule sets as versioned "models" |

#### How It Works

```python
import mlflow

mlflow.set_tracking_uri("http://mlflow-server.sanicode.svc:5000")
mlflow.set_experiment("my-app-repository")

with mlflow.start_run(run_name=f"scan-{commit_sha[:8]}"):
    # Log scan configuration
    mlflow.log_params({
        "profile": "federal-moderate",
        "llm_tier": "analysis",
        "model": "granite-code-8b",
        "commit_sha": commit_sha,
        "branch": "main",
        "scan_mode": "hybrid",
    })

    # Run the scan
    results = sanicode_engine.scan(codebase_path, config)

    # Log metrics
    mlflow.log_metrics({
        "findings_total": results.total_findings,
        "cat_i_count": results.cat_i,
        "cat_ii_count": results.cat_ii,
        "cat_iii_count": results.cat_iii,
        "cwe_unique_count": len(results.unique_cwes),
        "scan_duration_sec": results.duration,
        "llm_calls": results.llm_call_count,
        "false_positive_rate": results.estimated_fp_rate,
        "compliance_score": results.compliance_score,  # 0-100
    })

    # Log artifacts
    mlflow.log_artifact(results.sarif_path, "reports")
    mlflow.log_artifact(results.graph_snapshot_path, "knowledge-graph")
    mlflow.log_artifact(results.markdown_report_path, "reports")
```

#### Value for Sanicode Users

- **Trend tracking across commits:** "Is our codebase getting more secure over time?" becomes a line chart in MLflow UI
- **Regression detection:** Compare metrics between runs — if `cat_i_count` goes up, something broke
- **Audit evidence:** MLflow preserves a timestamped, immutable record of every scan with its full config and results — directly supports ATO documentation requirements
- **A/B comparison of scan configs:** "Does switching from Granite 8B to Llama 70B actually find more issues?" — compare runs side by side
- **Already deployed:** Many OpenShift AI environments already have MLflow running via the [strangiato/mlflow-server Helm chart](https://ai-on-openshift.io/tools-and-applications/mlflow/mlflow/). Sanicode just needs to point at it.

#### Disconnected Considerations

MLflow server stores artifacts in S3-compatible storage (MinIO in disconnected environments) and metadata in PostgreSQL (via Crunchy Postgres operator). Both work fully offline. The MLflow server image needs to be mirrored like any other container image.

### Data Science Pipelines (Kubeflow Pipelines 2.x) — Scan Orchestration

OpenShift AI ships Data Science Pipelines based on [Kubeflow Pipelines 2.x](https://www.redhat.com/en/blog/whats-new-data-science-pipelines-red-hat-openshift-ai). This gives sanicode a Kubernetes-native pipeline engine without adding another dependency.

#### Why Use KFP Instead of Just Tekton?

Both are valid, but they serve different roles:

| Aspect | Tekton | Data Science Pipelines (KFP 2.x) |
|---|---|---|
| **Primary audience** | CI/CD — build, test, deploy | Data/ML workflows — train, evaluate, serve |
| **Pipeline definition** | YAML Task/Pipeline CRDs | Python SDK → compiled IR YAML |
| **Sanicode fit** | Gate deployments (pass/fail) | Orchestrate complex scan workflows (multi-repo, multi-profile, with analysis) |
| **DAG visualization** | Basic in Tekton Dashboard | Rich in RHOAI Dashboard — shows data flow between steps |
| **Artifact tracking** | Workspace PVCs (manual) | Built-in artifact store with lineage |
| **Caching** | None | Built-in step caching (skip unchanged steps) |
| **Parallelism** | Parallel Tasks in Pipeline | ParallelFor with configurable limits |

**Use both:** Tekton for CI/CD gating (simple pass/fail on PR). KFP for the scan orchestration pipeline itself (multi-step, with caching, parallelism, and artifact tracking).

#### Example: Sanicode Scan Pipeline in KFP 2.x

```python
from kfp import dsl, compiler

@dsl.component(base_image="registry.internal/sanicode:latest")
def clone_repo(repo_url: str, branch: str) -> dsl.OutputPath(str):
    """Clone the target repository."""
    import subprocess
    dest = "/workspace/source"
    subprocess.run(["git", "clone", "-b", branch, repo_url, dest], check=True)
    return dest

@dsl.component(base_image="registry.internal/sanicode:latest")
def ast_scan(source_path: str) -> dsl.Output[dsl.Artifact]:
    """Deterministic AST-based scan (no LLM needed)."""
    from sanicode.engine import scan_ast
    results = scan_ast(source_path)
    results.save("/tmp/ast_findings.json")
    return "/tmp/ast_findings.json"

@dsl.component(base_image="registry.internal/sanicode:latest")
def build_knowledge_graph(source_path: str) -> dsl.Output[dsl.Artifact]:
    """Build data flow knowledge graph from AST."""
    from sanicode.graph import build_graph
    graph = build_graph(source_path)
    graph.export("/tmp/knowledge_graph.json")
    return "/tmp/knowledge_graph.json"

@dsl.component(base_image="registry.internal/sanicode:latest")
def llm_analysis(
    graph_path: str,
    ast_findings_path: str,
    llm_endpoint: str,
    model: str,
) -> dsl.Output[dsl.Artifact]:
    """LLM-enhanced contextual analysis."""
    from sanicode.llm import analyze_with_context
    enhanced = analyze_with_context(graph_path, ast_findings_path, llm_endpoint, model)
    enhanced.save("/tmp/enhanced_findings.json")
    return "/tmp/enhanced_findings.json"

@dsl.component(base_image="registry.internal/sanicode:latest")
def generate_report(
    findings_path: str,
    profile: str,
) -> dsl.Output[dsl.Artifact]:
    """Generate compliance-mapped SARIF + markdown report."""
    from sanicode.report import generate
    report = generate(findings_path, profile)
    report.save_sarif("/tmp/report.sarif")
    report.save_markdown("/tmp/report.md")

@dsl.component(base_image="registry.internal/sanicode:latest")
def log_to_mlflow(
    findings_path: str,
    sarif_path: str,
    mlflow_uri: str,
    experiment_name: str,
):
    """Log scan results to MLflow for tracking."""
    import mlflow
    mlflow.set_tracking_uri(mlflow_uri)
    mlflow.set_experiment(experiment_name)
    with mlflow.start_run():
        # ... log params, metrics, artifacts
        pass

@dsl.pipeline(name="sanicode-scan-pipeline")
def sanicode_pipeline(
    repo_url: str,
    branch: str = "main",
    profile: str = "federal-moderate",
    llm_endpoint: str = "http://vllm-granite:8000/v1",
    model: str = "granite-code-8b",
    mlflow_uri: str = "http://mlflow-server:5000",
):
    # Step 1: Clone
    clone_task = clone_repo(repo_url=repo_url, branch=branch)

    # Steps 2 & 3: AST scan and graph build run in PARALLEL
    ast_task = ast_scan(source_path=clone_task.output)
    graph_task = build_knowledge_graph(source_path=clone_task.output)

    # Step 4: LLM analysis (needs both AST findings and graph)
    llm_task = llm_analysis(
        graph_path=graph_task.output,
        ast_findings_path=ast_task.output,
        llm_endpoint=llm_endpoint,
        model=model,
    )

    # Step 5: Generate report
    report_task = generate_report(
        findings_path=llm_task.output,
        profile=profile,
    )

    # Step 6: Log to MLflow
    log_to_mlflow(
        findings_path=llm_task.output,
        sarif_path=report_task.output,
        mlflow_uri=mlflow_uri,
        experiment_name=repo_url.split("/")[-1],
    )

# Compile to IR YAML for upload to RHOAI
compiler.Compiler().compile(sanicode_pipeline, "sanicode_pipeline.yaml")
```

#### Value for Sanicode Users

- **Step caching:** If the repo hasn't changed, skip the clone and AST scan — jump straight to LLM analysis. Saves GPU time.
- **ParallelFor:** Scan multiple repos in parallel with `dsl.ParallelFor(repos)` and configurable parallelism limits (new in KFP 2.4.1)
- **DAG visualization:** The RHOAI Dashboard renders the pipeline as a visual DAG — useful for explaining the scan process to auditors
- **Artifact lineage:** Track which knowledge graph version produced which report — full provenance chain
- **Scheduling:** DSPA supports cron-scheduled pipelines — nightly scans without cron jobs or external schedulers
- **GitOps-native:** With `spec.apiServer.pipelineStore: kubernetes`, pipelines are stored as Kubernetes CRs, manageable via ArgoCD

#### Disconnected Considerations

DSPA stores pipeline artifacts in S3-compatible storage (MinIO) and metadata in MariaDB/PostgreSQL. All run fully offline. Pipeline component images (`base_image` in `@dsl.component`) must be mirrored to the internal registry. The compiled IR YAML is a static artifact that can be transferred via sneakernet.

Reference: [What's new with data science pipelines in Red Hat OpenShift AI](https://www.redhat.com/en/blog/whats-new-data-science-pipelines-red-hat-openshift-ai), [DSPO GitHub](https://github.com/opendatahub-io/data-science-pipelines-operator), [Working with Data Science Pipelines (RHOAI 2.25)](https://docs.redhat.com/en/documentation/red_hat_openshift_ai_self-managed/2.25/html-single/working_with_data_science_pipelines/)

### Grafana + Prometheus — Operational Dashboards

Sanicode should expose Prometheus metrics and ship a pre-built Grafana dashboard. This gives teams real-time visibility into scan activity, findings trends, and compliance posture — using the tooling they already have.

#### Prometheus Metrics to Expose

Sanicode's FastAPI server (or Operator) exports these via `/metrics` endpoint:

```
# Scan activity
sanicode_scans_total{profile="federal-moderate", status="completed"} counter
sanicode_scans_duration_seconds{profile, llm_tier} histogram
sanicode_scans_in_progress gauge

# Findings
sanicode_findings_total{severity="cat-i", cwe="89", namespace="my-app"} gauge
sanicode_findings_by_category{category="injection", namespace} gauge
sanicode_findings_resolved_total{namespace} counter
sanicode_findings_regressed_total{namespace} counter
sanicode_exceptions_active{namespace} gauge
sanicode_exceptions_expired{namespace} counter

# Compliance posture
sanicode_compliance_score{profile="federal-moderate", namespace} gauge  # 0-100
sanicode_controls_passing{framework="nist-800-53", namespace} gauge
sanicode_controls_failing{framework="nist-800-53", namespace} gauge

# LLM usage
sanicode_llm_requests_total{model="granite-code-8b", tier="analysis"} counter
sanicode_llm_request_duration_seconds{model, tier} histogram
sanicode_llm_errors_total{model, error_type} counter
sanicode_llm_tokens_consumed_total{model, tier} counter

# Knowledge graph
sanicode_graph_nodes_total{type="entry_point|sink|sanitizer"} gauge
sanicode_graph_edges_total gauge
sanicode_graph_build_duration_seconds histogram
```

#### Grafana Dashboard Panels

Ship a pre-built `sanicode-dashboard.json` that users import into Grafana:

**Row 1 — Executive Summary:**
- Compliance score gauge (0–100) per namespace/profile
- Total findings by severity (CAT I / II / III stacked bar)
- Active exceptions count
- Scan success rate (last 24h)

**Row 2 — Findings Trends:**
- Findings over time (line chart, by severity) — "Are we getting better?"
- Top 10 CWEs found (horizontal bar)
- Findings by namespace (heatmap)
- Regression alerts (new findings in previously-clean areas)

**Row 3 — Scan Operations:**
- Scans per day (bar chart)
- Scan duration distribution (histogram)
- LLM call volume and latency (by model/tier)
- LLM error rate

**Row 4 — Knowledge Graph Health:**
- Graph size over time (nodes/edges)
- Entry points discovered vs. sinks covered
- Sanitization coverage ratio (paths with sanitizers / total paths)

#### Deployment Pattern

Use the community Grafana Operator from OperatorHub (the built-in OpenShift monitoring Grafana is read-only):

```yaml
# GrafanaDashboard CR — auto-imported by Grafana Operator
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
  name: sanicode-dashboard
  namespace: sanicode
spec:
  instanceSelector:
    matchLabels:
      dashboards: sanicode
  json: |
    { ... dashboard JSON ... }
---
# GrafanaDatasource CR — connect to OpenShift Prometheus
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
  name: prometheus
  namespace: sanicode
spec:
  instanceSelector:
    matchLabels:
      dashboards: sanicode
  datasource:
    name: Prometheus
    type: prometheus
    url: https://thanos-querier.openshift-monitoring.svc:9091
    access: proxy
    jsonData:
      httpHeaderName1: Authorization
      tlsSkipVerify: true
    secureJsonData:
      httpHeaderValue1: "Bearer ${PROMETHEUS_TOKEN}"
---
# ServiceMonitor — tell Prometheus to scrape sanicode metrics
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: sanicode-metrics
  namespace: sanicode
spec:
  selector:
    matchLabels:
      app: sanicode-api
  endpoints:
    - port: metrics
      interval: 30s
      path: /metrics
```

#### Alerting Rules

```yaml
# PrometheusRule — alert on critical findings
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: sanicode-alerts
  namespace: sanicode
spec:
  groups:
    - name: sanicode.findings
      rules:
        - alert: SanicodeCatIFindingDetected
          expr: sanicode_findings_total{severity="cat-i"} > 0
          for: 0m
          labels:
            severity: critical
          annotations:
            summary: "CAT I finding detected in {{ $labels.namespace }}"
            description: "{{ $value }} CAT I findings in {{ $labels.namespace }}. CWE: {{ $labels.cwe }}"

        - alert: SanicodeComplianceScoreLow
          expr: sanicode_compliance_score < 70
          for: 1h
          labels:
            severity: warning
          annotations:
            summary: "Compliance score below 70 in {{ $labels.namespace }}"

        - alert: SanicodeExceptionExpiring
          expr: sanicode_exceptions_expired > 0
          for: 0m
          labels:
            severity: warning
          annotations:
            summary: "Expired security exception in {{ $labels.namespace }}"

        - alert: SanicodeLLMErrorRate
          expr: rate(sanicode_llm_errors_total[5m]) / rate(sanicode_llm_requests_total[5m]) > 0.1
          for: 10m
          labels:
            severity: warning
          annotations:
            summary: "LLM error rate above 10% for model {{ $labels.model }}"
```

#### Disconnected Considerations

The Grafana Operator image and Grafana image must be mirrored. Prometheus is built into OpenShift — no additional deployment needed. ServiceMonitor CRs require the `openshift-user-workload-monitoring` ConfigMap to be enabled (set `enableUserWorkload: true` in the `cluster-monitoring-config` ConfigMap). Dashboard JSON and alerting rules are static artifacts shipped with sanicode's Helm chart.

Reference: [Custom Grafana Dashboards for OCP 4](https://www.redhat.com/en/blog/custom-grafana-dashboards-red-hat-openshift-container-platform-4), [Deploying Grafana on OpenShift 4](https://cloud.redhat.com/experts/o11y/ocp-grafana/), [Custom Queries for Observability](https://www.redhat.com/en/blog/custom-queries-for-observability-using-grafana-and-apis)

### How It All Fits Together

```
┌──────────────────────────────────────────────────────────────────┐
│  OpenShift Cluster (disconnected)                                │
│                                                                  │
│  ┌─────────────────────┐  ┌──────────────────────────────────┐  │
│  │  OpenShift AI        │  │  OpenShift Monitoring            │  │
│  │  ├── KServe/vLLM    │  │  ├── Prometheus (built-in)      │  │
│  │  ├── MLflow Server  │  │  ├── AlertManager               │  │
│  │  └── DS Pipelines   │  │  └── Thanos Querier             │  │
│  └────────┬────────────┘  └──────────────────┬───────────────┘  │
│           │                                   │                  │
│  ┌────────┴───────────────────────────────────┴───────────────┐  │
│  │  Sanicode                                                  │  │
│  │                                                            │  │
│  │  CLI / API Server                                          │  │
│  │  ├── Scan engine ──────────────────► vLLM (OpenAI API)    │  │
│  │  ├── MLflow client ────────────────► MLflow (experiments)  │  │
│  │  ├── /metrics endpoint ────────────► Prometheus (scrape)   │  │
│  │  └── Pipeline definitions ─────────► DSPA (orchestration)  │  │
│  │                                                            │  │
│  │  Helm Chart ships:                                         │  │
│  │  ├── GrafanaDashboard CR ──────────► Grafana (visualize)  │  │
│  │  ├── ServiceMonitor CR ────────────► Prometheus (scrape)   │  │
│  │  ├── PrometheusRule CR ────────────► AlertManager (alert)  │  │
│  │  └── Pipeline IR YAML ────────────► DSPA (register)       │  │
│  └────────────────────────────────────────────────────────────┘  │
│                                                                  │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  Grafana (Operator)                                        │  │
│  │  ├── Sanicode Dashboard (auto-imported)                   │  │
│  │  ├── Compliance Posture panel                             │  │
│  │  ├── Findings Trends panel                                │  │
│  │  └── Alerting → Slack / Email / PagerDuty                │  │
│  └────────────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────────┘
```

### Optional vs. Required Components

Not every deployment needs everything. Design for progressive adoption:

| Component | Required? | When to Add |
|---|---|---|
| **Sanicode CLI + API** | Yes | Always — this is the core |
| **vLLM / LLM endpoint** | No (degraded mode works) | When LLM-enhanced analysis is wanted |
| **MLflow** | No | When teams want scan history and trend tracking |
| **Data Science Pipelines** | No | When orchestrating multi-repo or scheduled scans |
| **Prometheus metrics** | Recommended | When running as a service (always expose /metrics) |
| **Grafana dashboard** | No | When teams want visual dashboards and alerting |
| **PrometheusRules / Alerting** | No | When automated alerting on findings is needed |

Each integration should be a `sanicode.toml` toggle:

```toml
[integrations.mlflow]
enabled = true
tracking_uri = "http://mlflow-server.sanicode.svc:5000"

[integrations.prometheus]
enabled = true
port = 9090

[integrations.pipelines]
enabled = false  # only when DSPA is available
```

---

## Next Steps

### Phase 1 — MVP (Python Package + CLI)
1. **Define MVP scope:** Python codebases, input validation + SQL injection + command injection detection
2. **Prototype knowledge graph construction:** AST parsing → data flow graphs → LLM enhancement
3. **Build compliance mapping engine:** CWE → OWASP ASVS → NIST 800-53 cross-reference database
4. **Implement `sanicode config`:** LLM backend configuration with tiered model support
5. **Implement `sanicode scan`:** Core scanning loop with knowledge graph construction (local mode)
6. **Implement `sanicode serve`:** FastAPI-based API server for containerized deployment
7. **Implement `sanicode report`:** Markdown + JSON + SARIF output
8. **Write tests and publish to PyPI**

### Phase 2 — Containerized Deployment + Observability
9. **Containerize:** UBI9-based multi-stage Dockerfile
10. **Helm chart:** Parameterized deployment for OpenShift (includes ServiceMonitor, GrafanaDashboard, PrometheusRule CRs)
11. **Prometheus metrics:** Expose `/metrics` endpoint with findings, compliance, LLM usage metrics
12. **Grafana dashboard:** Ship pre-built `sanicode-dashboard.json` with compliance, trends, and operations panels
13. **MLflow integration:** Log scan runs as experiments with parameters, metrics, and SARIF artifacts
14. **KFP pipeline definition:** Compile scan pipeline for Data Science Pipelines (parallel AST + graph build, LLM analysis, report generation)
15. **Tekton Task definition:** CI/CD gating (simpler pass/fail for PR pipelines)
16. **Document disconnected deployment:** oc-mirror commands, PVC setup, model staging guide, image list
17. **Test on OpenShift AI** with Granite Code via KServe/vLLM

### Phase 3 — Operator (kopf)
14. **Define CRDs:** SanicodeConfig, SanicodeScan, SanicodeProfile, SanicodeReport, SanicodeException
15. **Build kopf-based reconciler:** Watch CRs, spawn scan Jobs, manage lifecycle
16. **Implement auto-discovery:** Find InferenceService CRs from OpenShift AI
17. **Implement exception management:** Expiring exceptions with RBAC

### Phase 4 — Production Operator
18. **Evaluate kopf vs. Go conversion** based on Phase 3 learnings
19. **OLM packaging:** Bundle, catalog, dependency on OpenShift AI
20. **OperatorHub submission** (community or Red Hat certified)
21. **Refactoring agent integration:** MCP server / tool definitions for migration agents
