# ModelAudit Security Requirements Gap Analysis

**Date:** November 26, 2025
**Repository:** modelaudit
**Version Analyzed:** 0.2.19

---

## Executive Summary

After thorough analysis of the ModelAudit codebase, I've evaluated each of the 35 detection requirements. The scanner has strong coverage of RCE/deserialization threats and pickle-based attacks, with growing support for graph-based model threats. Key gaps exist in signature verification, provenance attestation, and dependency vulnerability scanning.

**Overall Coverage:**
- **12 requirements fully supported** (mostly around pickle/deserialization, code execution, SBOM, and reporting)
- **14 requirements partially supported** (need enhancement or additional patterns)
- **9 requirements not supported** (significant gaps requiring new features)

---

## Detailed Requirement Analysis

### Requirement 1: Unsafe Deserialization / RCE Payloads
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Pickle (.pkl), Joblib (.joblib), Torch (.pt, .pth)

**Evidence:**
- `pickle_scanner.py`: Comprehensive pickle opcode analysis (REDUCE, INST, OBJ, NEWOBJ, STACK_GLOBAL)
- `joblib_scanner.py`: Handles `.joblib` files with pickle detection
- `pytorch_zip_scanner.py`: Scans `.pt/.pth` for embedded pickle payloads
- `detectors/suspicious_symbols.py`: Defines `SUSPICIOUS_GLOBALS` with dangerous modules (os, subprocess, eval, exec, etc.)
- CVE-specific patterns for CVE-2020-13092 (sklearn joblib) and CVE-2024-34997 (joblib NumpyArrayWrapper)

**Gap:** None - this is well-covered.

---

### Requirement 2: Malicious Code in Model Graph
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Must
**Relevant Formats:** ONNX, TensorFlow SavedModel (.pb), TorchScript (.pt), TF-Lite

**Evidence:**
- `tf_savedmodel_scanner.py`: Detects dangerous TensorFlow ops (PyFunc, EagerPyFunc, ReadFile, WriteFile)
- `onnx_scanner.py`: Basic ONNX structure validation
- `tflite_scanner.py`: Custom operator detection
- `detectors/suspicious_symbols.py`: `SUSPICIOUS_OPS` includes PyFunc, ShellExecute, DecodeRaw

**Gaps:**
- TorchScript graph inspection not implemented (complex graph analysis)
- No dynamic graph construct detection in ONNX
- Limited eval-like node detection across formats

**To Add Support:**
1. Implement TorchScript graph analyzer using `torch.jit.freeze()` inspection
2. Add ONNX opcode scanning for suspicious operations like PyOp, Custom nodes
3. Extend TensorFlow scanner for more dangerous ops (e.g., `tf.py_function`, `tf.numpy_function`)

**Effort:** Medium-High (2-4 weeks for comprehensive graph analysis)

---

### Requirement 3: Custom Operator / Plugin Loading
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Must
**Relevant Formats:** ONNX, TorchScript, TF

**Evidence:**
- `tflite_scanner.py:94-104`: Detects `CUSTOM` operator codes
- `onnx_scanner.py`: Can detect non-standard op types
- `tf_savedmodel_scanner.py`: Identifies custom ops in TensorFlow graphs

**Gaps:**
- No detection of external library loading (`.so`, `.dll` references)
- TorchScript custom op detection incomplete
- Missing analysis of operator registry additions

**To Add Support:**
1. Add pattern matching for `torch.ops.load_library()` calls
2. Scan for `onnxruntime.register_custom_ops_library()` patterns
3. Detect TensorFlow custom op registration (`tf.load_op_library`)

**Effort:** Medium (1-2 weeks)

---

### Requirement 4: Embedded Binary or Script Artifacts
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Any zipped model bundle, TF SavedModel, ONNX zip

**Evidence:**
- `zip_scanner.py`: Scans archives for executables (`.exe`, `.dll`, `.so`, `.sh`, `.py`)
- `tar_scanner.py`: Similar archive scanning
- `detectors/suspicious_symbols.py`: `EXECUTABLE_SIGNATURES` detects PE, ELF, Mach-O headers

**Gaps:**
- TensorFlow SavedModel bundle scanning could be more thorough
- ONNX zip containers not explicitly scanned for binaries
- Detection of base64-encoded binaries could be stronger

**To Add Support:**
1. Extend SavedModel scanner to enumerate and check all files in `assets/` directory
2. Add ONNX external data file scanning
3. Implement binary pattern detection in embedded data blobs

**Effort:** Low-Medium (1 week)

---

### Requirement 5: Unverified Model Signature / Checksum Mismatch
**Status:** ❌ NOT SUPPORTED
**Priority:** Must
**Relevant Formats:** All

**Evidence:**
- `utils/secure_hasher.py`: Calculates SHA256 hashes
- `integrations/sbom_generator.py`: Includes file hashes in SBOM output
- No signature verification logic exists

**Gaps:**
- No GPG signature verification
- No in-toto attestation parsing
- No SLSA provenance verification
- No checksum validation against expected values

**To Add Support:**
1. Implement GPG signature verification using `python-gnupg`
2. Add in-toto link/layout verification
3. Implement SLSA provenance parser and validator
4. Add `--expected-hash` CLI option for checksum validation
5. Support HuggingFace model card checksum fields

**Effort:** High (3-4 weeks for comprehensive provenance support)

---

### Requirement 6: Unsigned or Tampered Model Weights
**Status:** ❌ NOT SUPPORTED
**Priority:** Must
**Relevant Formats:** .bin, .safetensors, .gguf, .h5, .ckpt

**Evidence:**
- No weight signature detection exists in the codebase
- `safetensors_scanner.py`: Only validates format, not signatures

**Gaps:**
- No detection of missing signatures in `.safetensors` files
- No validation of weight file integrity beyond format checks
- No support for signed weight formats

**To Add Support:**
1. Implement SafeTensors metadata signature verification
2. Add GGUF checksum validation
3. Create framework for model-specific signature policies
4. Support for signing schemes (e.g., sigstore/cosign)

**Effort:** High (2-3 weeks)

---

### Requirement 7: Vulnerable Dependency Versions
**Status:** ❌ NOT SUPPORTED
**Priority:** Must
**Relevant Formats:** requirements.txt, environment.yml, containers

**Evidence:**
- No CVE database integration exists
- No `requirements.txt` scanning for known vulnerabilities
- The codebase references package versions but doesn't check for CVEs

**Gaps:**
- No integration with vulnerability databases (NVD, OSV, GitHub Advisory)
- No manifest file scanning for vulnerable packages
- No container image dependency scanning

**To Add Support:**
1. Integrate with `pip-audit` or `safety` for Python CVE detection
2. Add OSV (Open Source Vulnerabilities) API integration
3. Implement `requirements.txt`, `environment.yml`, `Pipfile` parsing
4. Add PyPI metadata fetching for version comparisons

**Effort:** Medium-High (2-3 weeks)

---

### Requirement 8: Insecure Dependency Licensing
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Should
**Relevant Formats:** Manifests

**Evidence:**
- `integrations/license_checker.py`: Comprehensive license detection
  - SPDX license patterns (MIT, Apache-2.0, GPL, AGPL, CC variants)
  - `check_commercial_use_warnings()` for commercial restriction detection
  - AGPL and non-commercial license warnings

**Gaps:**
- No policy configuration for allowed/disallowed licenses
- No transitive dependency license analysis
- Limited to file-level license detection

**To Add Support:**
1. Add license policy configuration (JSON/YAML based)
2. Implement `--license-policy` CLI option
3. Add severity escalation for policy violations

**Effort:** Low-Medium (1 week)

---

### Requirement 9: Sensitive Information Leakage
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Model metadata, tokenizer files, config.json

**Evidence:**
- `detectors/secrets.py`: Comprehensive secrets detection
  - 50+ secret patterns (AWS, OpenAI, GitHub tokens, etc.)
  - Shannon entropy analysis
  - ML context-aware false positive filtering

**Gaps:**
- PII detection not specifically implemented
- Tokenizer vocabulary secret scanning could be enhanced
- No endpoint/URL credential detection

**To Add Support:**
1. Add PII patterns (SSN, email, phone in training data)
2. Implement tokenizer vocabulary scanning for leaked secrets
3. Add URL credential detection (basic auth in URLs)

**Effort:** Low-Medium (1 week)

---

### Requirement 10: Embedded Training Data Samples
**Status:** ❌ NOT SUPPORTED
**Priority:** Should
**Relevant Formats:** Pickle, H5, SavedModel, Torch

**Evidence:**
- No specific training data detection exists
- `license_checker.py:664`: Has `is_dataset` flag but doesn't detect embedded samples

**Gaps:**
- No detection of dataset fragments in model files
- No privacy-sensitive sample detection
- No metadata indicating training data inclusion

**To Add Support:**
1. Implement heuristics for detecting embedded samples (string patterns, image headers)
2. Add pickle payload analysis for dataset objects
3. Detect common dataset class references (pandas DataFrame, torch.utils.data.Dataset)
4. Flag unusually large metadata fields that might contain samples

**Effort:** Medium (2 weeks)

---

### Requirement 11: Unreferenced Large Binary Blobs
**Status:** ❌ NOT SUPPORTED
**Priority:** Should
**Relevant Formats:** All

**Evidence:**
- No graph-to-tensor reference tracking exists
- `weight_distribution_scanner.py`: Analyzes distribution but not references

**Gaps:**
- No detection of tensors unreferenced by model graph
- No supply chain insertion detection via orphan files
- Limited structural integrity validation

**To Add Support:**
1. For ONNX: Parse graph and compare tensor names to initializer names
2. For TensorFlow: Cross-reference variable names with graph nodes
3. For PyTorch: Analyze state_dict keys against model structure
4. Flag orphan files in model archives

**Effort:** Medium-High (2-3 weeks)

---

### Requirement 12: Hidden/Obfuscated Code Strings
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Pickle, TorchScript, ONNX

**Evidence:**
- `detectors/suspicious_symbols.py:202-225`: `SUSPICIOUS_STRING_PATTERNS` includes base64, hex patterns
- `pickle_scanner.py`: Detects encoded payloads
- `jinja2_template_scanner.py`: SSTI pattern detection with obfuscation bypass patterns

**Gap:** None - this is well-covered.

---

### Requirement 13: Suspicious Metadata Fields
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Should
**Relevant Formats:** All

**Evidence:**
- `manifest_scanner.py`: Scans config files for blacklisted terms
- `metadata_scanner.py`: Checks for suspicious URLs and exposed secrets
- `detectors/suspicious_symbols.py:444-496`: `SUSPICIOUS_CONFIG_PATTERNS` for network access, credentials, execution

**Gaps:**
- No detection of `run_cmd`, `shell`, `download_url` specifically
- Limited coverage of arbitrary metadata fields
- Could expand to more model formats

**To Add Support:**
1. Add explicit patterns for dangerous metadata keys (`run_cmd`, `init_hook`, `download_url`)
2. Implement metadata schema validation against known good patterns
3. Extend to GGUF, ONNX, and other format metadata

**Effort:** Low (3-5 days)

---

### Requirement 14: Poisoned or Malicious Weights (Data Poisoning)
**Status:** ✅ SUPPORTED (Consider Severity Upgrade)
**Priority:** Could
**Relevant Formats:** All

**Evidence:**
- `weight_distribution_scanner.py`: Comprehensive weight analysis
  - Outlier neuron detection
  - Dissimilar weight vector detection
  - Statistical anomaly identification

**Gap:** Currently INFO severity - consider upgrading to WARNING for clearer policy enforcement.

---

### Requirement 15: Model Card Completeness
**Status:** ❌ NOT SUPPORTED
**Priority:** Should
**Relevant Formats:** Model card JSON/YAML

**Evidence:**
- `metadata_scanner.py`: Can handle README.md, model_card.md
- `manifest_scanner.py`: Extracts model metadata
- No completeness validation exists

**Gaps:**
- No schema for required model card fields
- No compliance metadata validation
- No warnings for missing purpose/dataset/author information

**To Add Support:**
1. Define model card schema (based on HuggingFace model card spec)
2. Implement required field validation
3. Add `--require-model-card` CLI option
4. Output completeness score

**Effort:** Medium (1-2 weeks)

---

### Requirement 16: SBOM Generation & Policy Check
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** All

**Evidence:**
- `integrations/sbom_generator.py`: Full CycloneDX v1.6 support
  - SPDX license expressions
  - SHA256 hashes
  - ML-BOM properties (`ml:framework`, `ml:model_type`)
  - Risk scoring integration

**Gap:** None - comprehensive SBOM support exists.

---

### Requirement 17: Framework Mis-versioning / Compatibility Check
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Should
**Relevant Formats:** All

**Evidence:**
- `scanners/__init__.py`: NumPy version compatibility detection
- `manifest_scanner.py:319`: Extracts `transformers_version` from configs

**Gaps:**
- No comparison of declared vs actual framework versions
- No mismatch detection/warnings
- Limited framework coverage

**To Add Support:**
1. Parse framework version from model metadata
2. Compare against installed framework version
3. Warn on significant version mismatches
4. Add version range compatibility checks

**Effort:** Medium (1-2 weeks)

---

### Requirement 18: Deprecated or Dangerous Tensor Ops
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Should
**Relevant Formats:** ONNX, TF, Torch

**Evidence:**
- `detectors/suspicious_symbols.py`: `TENSORFLOW_DANGEROUS_OPS` with descriptions
- `tf_savedmodel_scanner.py`: Detects dangerous TensorFlow ops

**Gaps:**
- No deprecated op database
- Limited ONNX op deprecation checks
- No PyTorch op deprecation tracking

**To Add Support:**
1. Add deprecated op list per framework version
2. Implement op deprecation warnings
3. Add security-specific op blocklists

**Effort:** Medium (1-2 weeks)

---

### Requirement 19: External Resource References
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Config, checkpoints

**Evidence:**
- `detectors/network_comm.py`: URL pattern detection, domain scanning
- `metadata_scanner.py`: Suspicious URL detection in text files
- `detectors/suspicious_symbols.py:314-318`: Metadata URL patterns

**Gaps:**
- Limited detection in model graph nodes
- No S3/GCS bucket reference scanning in configs
- Could expand to file path detection

**To Add Support:**
1. Add cloud storage URL patterns (s3://, gs://, azure://)
2. Scan config files for external resource references
3. Detect absolute file paths in metadata

**Effort:** Low (3-5 days)

---

### Requirement 20: Unencrypted Model Storage
**Status:** ❌ NOT SUPPORTED
**Priority:** Should
**Relevant Formats:** All

**Evidence:**
- No encryption detection exists in the codebase

**Gaps:**
- No detection of encryption headers/markers
- No policy enforcement for encryption-at-rest
- No warnings for plaintext storage

**To Add Support:**
1. Detect encryption headers (GPG, AES)
2. Add `--require-encryption` policy option
3. Integrate with cloud storage encryption status (S3 SSE, GCS CMEK)

**Effort:** Medium (1-2 weeks)

---

### Requirement 21: Unsafe Pickle Reducers / Global References
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Pickle, Joblib

**Evidence:**
- `pickle_scanner.py`: Comprehensive `__reduce__` detection
- `detectors/suspicious_symbols.py`: `SUSPICIOUS_GLOBALS` with full module coverage
- Pickle opcode analysis for global loading

**Gap:** None - this is the core strength of ModelAudit.

---

### Requirement 22: Suspicious Import Hooks
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Pickle, Torch, TF

**Evidence:**
- `detectors/suspicious_symbols.py:91-98`: Detects os, subprocess, socket references
- `pickle_scanner.py`: Flags dangerous module imports during deserialization
- `detectors/network_comm.py`: Network library import detection

**Gap:** None - well-covered.

---

### Requirement 23: Anomalous Large Embedding Tables / Vectors
**Status:** ❌ NOT SUPPORTED
**Priority:** Could
**Relevant Formats:** NLP embedding models

**Evidence:**
- `weight_distribution_scanner.py`: General weight distribution analysis
- No specific embedding table size analysis

**Gaps:**
- No steganographic data detection
- No oversized vector detection
- No embedding anomaly detection

**To Add Support:**
1. Add embedding size heuristics (abnormally large vocab)
2. Implement entropy analysis on embedding matrices
3. Detect unusual patterns in embedding distributions

**Effort:** Medium-High (2-3 weeks)

---

### Requirement 24: Unsafe Eval or Exec Calls
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Pickle, TorchScript

**Evidence:**
- `detectors/suspicious_symbols.py:184-198`: `DANGEROUS_BUILTINS` includes eval, exec, compile
- `pickle_scanner.py`: Detects builtins.eval, builtins.exec
- `jinja2_template_scanner.py`: Eval/exec pattern detection

**Gap:** None.

---

### Requirement 25: Unsafe Dynamic Imports
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Pickle, Torch, TF

**Evidence:**
- `detectors/suspicious_symbols.py:99-139`: Detects `__import__`, `importlib`, `runpy`
- Pattern matching for dynamic import patterns

**Gap:** None.

---

### Requirement 26: Version Drift or Patch Lag
**Status:** ❌ NOT SUPPORTED
**Priority:** Should
**Relevant Formats:** All

**Evidence:**
- No model registry comparison exists
- No upstream version tracking

**Gaps:**
- No comparison with model registry versions
- No security baseline tracking
- No patch lag detection

**To Add Support:**
1. Integrate with HuggingFace Hub API for version checking
2. Add model version comparison logic
3. Implement security advisory tracking for model versions

**Effort:** High (3-4 weeks)

---

### Requirement 27: Absence of Provenance Chain / Attestation
**Status:** ❌ NOT SUPPORTED
**Priority:** Must
**Relevant Formats:** All

**Evidence:**
- No attestation parsing exists
- No lineage validation

**Gaps:**
- No in-toto attestation support
- No build-time attestation validation
- No model lineage tracking

**To Add Support:**
1. Implement in-toto attestation parser
2. Add SLSA provenance verification
3. Create attestation presence checks
4. Add `--require-attestation` CLI option

**Effort:** High (3-4 weeks)

---

### Requirement 28: Hash Collisions or Weak Hashes
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Should
**Relevant Formats:** All

**Evidence:**
- `utils/secure_hasher.py`: Uses SHA256 by default
- `integrations/sbom_generator.py:239-249`: Outputs SHA256, SHA1, MD5

**Gaps:**
- No warning when only MD5/SHA1 provided for verification
- No hash algorithm policy enforcement

**To Add Support:**
1. Add warnings when weak hashes are used for integrity
2. Implement hash algorithm policy (require SHA256+)
3. Flag models with only weak hash verification

**Effort:** Low (3-5 days)

---

### Requirement 29: Malicious Metadata URLs
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** Config, tokenizer, model card

**Evidence:**
- `metadata_scanner.py`: Suspicious URL detection
- `detectors/network_comm.py`: C&C pattern detection, blacklisted domains

**Gap:** None - comprehensive URL scanning.

---

### Requirement 30: Model Schema Violations
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Should
**Relevant Formats:** ONNX, TF

**Evidence:**
- `onnx_scanner.py`: Basic ONNX structure validation
- `gguf_scanner.py`: GGUF format validation
- `tflite_scanner.py`: TFLite magic byte validation

**Gaps:**
- No full ONNX schema validation
- No TensorFlow GraphDef schema validation
- Limited to format detection vs schema compliance

**To Add Support:**
1. Implement ONNX spec schema validation
2. Add TensorFlow GraphDef validation
3. Create per-format schema validators

**Effort:** Medium-High (2-3 weeks)

---

### Requirement 31: Policyable Risk Classification
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** All

**Evidence:**
- `scanners/base.py`: `IssueSeverity` enum (CRITICAL, WARNING, INFO, DEBUG)
- `risk_scoring.py`: 0.0-1.0 normalized risk scores
- CLI supports configurable output formats

**Gap:** None - severity levels and scoring exist.

---

### Requirement 32: Machine-Readable Report Export
**Status:** ✅ FULLY SUPPORTED
**Priority:** Must
**Relevant Formats:** All

**Evidence:**
- `integrations/sarif_formatter.py`: Full SARIF 2.1.0 support
- JSON output format with CVE/CWE references
- `--format json` CLI option

**Gap:** None.

---

### Requirement 33: Integration Hooks
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Should
**Relevant Formats:** All

**Evidence:**
- CLI tool with `--format json` for CI/CD parsing
- Exit codes (0=clean, 1=issues, 2=errors)
- TTY detection for CI environments

**Gaps:**
- No REST API
- No GitLab/Jenkins native plugins
- No registry scanning integration

**To Add Support:**
1. Add REST API wrapper (FastAPI)
2. Create GitLab CI template
3. Create Jenkins pipeline library
4. Add Docker registry scanning integration

**Effort:** Medium-High (2-3 weeks per integration)

---

### Requirement 34: Extensible Rules Engine
**Status:** ⚠️ PARTIALLY SUPPORTED
**Priority:** Should
**Relevant Formats:** All

**Evidence:**
- `BaseScanner` class for scanner extensibility
- Configuration support via `--config` option
- Custom blacklist patterns supported

**Gaps:**
- No external rule file format
- No rule DSL for custom signatures
- Limited runtime rule addition

**To Add Support:**
1. Define YAML/JSON rule format
2. Implement rule loader
3. Add custom signature support
4. Create rule authoring documentation

**Effort:** Medium (2-3 weeks)

---

### Requirement 35: GPU Binary Signature Detection
**Status:** ❌ NOT SUPPORTED
**Priority:** Could
**Relevant Formats:** Torch, TF

**Evidence:**
- No CUDA/PTX analysis exists
- No GPU binary scanning

**Gaps:**
- No CUDA kernel analysis
- No PTX inspection
- No outdated GPU kernel detection

**To Add Support:**
1. Add CUDA binary header detection
2. Implement PTX pattern scanning
3. Add CUDA version extraction and validation
4. Create GPU kernel signature database

**Effort:** High (3-4 weeks)

---

## Summary Matrix

| Req # | Capability | Status | Priority | Effort to Add | Product Fit / Plan |
|-------|-----------|--------|----------|---------------|-------------------|
| 1 | Unsafe Deserialization | ✅ Supported | Must | N/A | **Core** - Complete |
| 2 | Malicious Code in Graph | ⚠️ Partial | Must | Medium-High | **Core** - Add ASAP |
| 3 | Custom Operator Loading | ⚠️ Partial | Must | Medium | **Core** - Add ASAP |
| 4 | Embedded Binary Artifacts | ⚠️ Partial | Must | Low-Medium | **Core** - Add ASAP |
| 5 | Model Signature Verification | ❌ Not Supported | Must | High | **Core** - Add ASAP |
| 6 | Unsigned Weight Detection | ❌ Not Supported | Must | High | **Core** - Add ASAP |
| 7 | Vulnerable Dependencies | ❌ Not Supported | Must | Medium-High | **Core** - Add ASAP |
| 8 | License Policy | ⚠️ Partial | Should | Low-Medium | On Roadmap |
| 9 | Sensitive Info Leakage | ⚠️ Partial | Must | Low-Medium | **Core** - Add ASAP |
| 10 | Embedded Training Data | ❌ Not Supported | Should | Medium | On Roadmap |
| 11 | Unreferenced Blobs | ❌ Not Supported | Should | Medium-High | On Roadmap |
| 12 | Obfuscated Code | ✅ Supported | Must | N/A | **Core** - Complete |
| 13 | Suspicious Metadata | ⚠️ Partial | Should | Low | On Roadmap |
| 14 | Weight Poisoning | ✅ Supported | Could | Severity upgrade | **Core** - Complete |
| 15 | Model Card Completeness | ❌ Not Supported | Should | Medium | On Roadmap |
| 16 | SBOM Generation | ✅ Supported | Must | N/A | **Core** - Complete |
| 17 | Framework Versioning | ⚠️ Partial | Should | Medium | On Roadmap |
| 18 | Deprecated Tensor Ops | ⚠️ Partial | Should | Medium | On Roadmap |
| 19 | External Resource Refs | ⚠️ Partial | Must | Low | **Core** - Add ASAP |
| 20 | Unencrypted Storage | ❌ Not Supported | Should | Medium | Future Consideration |
| 21 | Unsafe Pickle Reducers | ✅ Supported | Must | N/A | **Core** - Complete |
| 22 | Suspicious Imports | ✅ Supported | Must | N/A | **Core** - Complete |
| 23 | Anomalous Embeddings | ❌ Not Supported | Could | Medium-High | Unplanned |
| 24 | Unsafe Eval/Exec | ✅ Supported | Must | N/A | **Core** - Complete |
| 25 | Unsafe Dynamic Imports | ✅ Supported | Must | N/A | **Core** - Complete |
| 26 | Version Drift | ❌ Not Supported | Should | High | Future Consideration |
| 27 | Provenance/Attestation | ❌ Not Supported | Must | High | **Core** - Add ASAP |
| 28 | Weak Hashes | ⚠️ Partial | Should | Low | On Roadmap |
| 29 | Malicious URLs | ✅ Supported | Must | N/A | **Core** - Complete |
| 30 | Schema Violations | ⚠️ Partial | Should | Medium-High | On Roadmap |
| 31 | Risk Classification | ✅ Supported | Must | N/A | **Core** - Complete |
| 32 | Machine-Readable Export | ✅ Supported | Must | N/A | **Core** - Complete |
| 33 | Integration Hooks | ⚠️ Partial | Should | Medium-High | On Roadmap |
| 34 | Extensible Rules | ⚠️ Partial | Should | Medium | On Roadmap |
| 35 | GPU Binary Detection | ❌ Not Supported | Could | High | Unplanned |

---

## Priority Implementation Recommendations

### Immediate (Must-Have Gaps)

These are critical gaps that should be addressed first:

1. **Requirement 5/27: Signature and Provenance Verification**
   - Implement GPG signature verification
   - Add SLSA provenance parser
   - Add in-toto attestation support
   - This is foundational for supply chain security

2. **Requirement 6: Signed Weight File Detection**
   - Add detection for missing signatures in weight files
   - Support sigstore/cosign signing verification

3. **Requirement 7: Vulnerable Dependency Scanning**
   - Integrate with `pip-audit` or OSV API
   - Scan `requirements.txt`, `environment.yml`
   - This addresses a major blind spot

### Short-term (Partial Coverage to Complete)

1. **Requirement 2/3: Enhanced Graph Scanning**
   - Add TorchScript graph inspection
   - Enhance ONNX custom operator detection
   - Expand TensorFlow op coverage

2. **Requirement 4: Complete Binary Artifact Detection**
   - Extend SavedModel asset scanning
   - Add ONNX external data file scanning

3. **Requirement 9: Enhanced Secrets Detection**
   - Add PII patterns
   - Enhance tokenizer vocabulary scanning

4. **Requirement 19: Cloud Storage URLs**
   - Add S3, GCS, Azure blob patterns
   - Detect external resource references in configs

### Medium-term (Should-Have)

1. **Requirement 15: Model Card Completeness**
   - Define required field schema
   - Add completeness scoring

2. **Requirement 30: Full Schema Validation**
   - Implement ONNX spec validation
   - Add TensorFlow GraphDef validation

3. **Requirement 33: REST API and CI/CD**
   - Create FastAPI wrapper
   - Develop GitLab/Jenkins integrations

4. **Requirement 34: Custom Rules Engine**
   - Define YAML rule format
   - Implement rule loader

---

## Key Files Reference

| Component | File Path | Purpose |
|-----------|-----------|---------|
| Pickle Scanner | `modelaudit/scanners/pickle_scanner.py` | Core RCE detection |
| Suspicious Symbols | `modelaudit/detectors/suspicious_symbols.py` | Pattern definitions |
| Secrets Detector | `modelaudit/detectors/secrets.py` | API key/credential detection |
| Network Detector | `modelaudit/detectors/network_comm.py` | URL/C&C detection |
| SBOM Generator | `modelaudit/integrations/sbom_generator.py` | CycloneDX output |
| SARIF Formatter | `modelaudit/integrations/sarif_formatter.py` | SARIF 2.1.0 output |
| License Checker | `modelaudit/integrations/license_checker.py` | SPDX license detection |
| Weight Distribution | `modelaudit/scanners/weight_distribution_scanner.py` | Anomaly detection |
| TensorFlow Scanner | `modelaudit/scanners/tf_savedmodel_scanner.py` | TF graph analysis |
| ONNX Scanner | `modelaudit/scanners/onnx_scanner.py` | ONNX structure validation |

---

## Conclusion

ModelAudit has strong foundations for ML model security scanning, particularly around pickle deserialization attacks and code execution vectors. The primary gaps are in:

1. **Supply chain security** (signatures, provenance, attestation)
2. **Dependency vulnerability scanning** (CVE database integration)
3. **Advanced graph analysis** (TorchScript, enhanced ONNX)

Addressing the Must-Have gaps (5, 6, 7, 27) should be the immediate priority to provide comprehensive model security coverage.
