# CIS Aggregator Security Design

**Date**: 2026-03-10
**Issue**: [#492 — GAP-18: collective intelligence aggregator represents a single high-value attack target](https://github.com/LookNoHandsMom/LLMHosts.com/issues/492)
**Status**: Design — pending implementation
**Related**: [docs/plans/2026-03-10-vision-2026-masterplan.md §Pillar 4](./2026-03-10-vision-2026-masterplan.md)

---

## Overview

The Collective Intelligence System (CIS) aggregator ingests opt-in hardware telemetry and benchmark data from ~10,000+ operator nodes and surfaces aggregated performance profiles that drive model recommendations across the platform. Because every operator and end-user who relies on those recommendations is downstream of this system, the aggregator is a high-value attack target. A single successful poisoning campaign can degrade inference quality for the entire network without touching a single user's device.

This document records the threat model, the layered defenses adopted, and the acceptance criteria that must hold before CIS data is used to drive public recommendations.

---

## Threat Model

**Adversaries**

- **Compromised operator nodes** — the most likely threat. A legitimate account holder whose device or credentials have been taken over, or a malicious operator who joined the network specifically to manipulate aggregate benchmark data.
- **Sybil attacker** — an adversary who registers many fake accounts and devices to amplify the statistical weight of fabricated benchmark submissions, shifting aggregated medians in a targeted direction.
- **Nation-state or sophisticated MitM** — an adversary positioned between operator nodes and the aggregator (BGP hijack, rogue CA, DNS poisoning) who replaces or replays legitimate submissions.
- **Supply chain attacker** — malicious code injected into the `llmhosts` package that silently alters benchmark readings before they are signed and submitted.
- **Insider** — a platform employee or contractor with write access to the aggregator's backing store who directly modifies aggregate records or raw submission logs.

**What adversaries can gain**

- **Poisoned model recommendations** — steering users toward specific models or quantizations that perform poorly, serve adversary-controlled backends, or exhaust user VRAM.
- **Operator identity exfiltration** — correlating submission IP addresses, hardware fingerprints, and timing patterns to de-anonymize the physical locations or organizations behind operator nodes.
- **Competitive intelligence** — harvesting high-resolution benchmark data to understand the performance envelope of rival hardware fleets.
- **Network destabilization** — flooding the aggregator with malformed or extreme data to trigger correctness failures in the recommendation engine.

**Primary attack vectors**

1. Malicious benchmark submissions (inflated or deflated tokens/sec figures).
2. Replay attacks — retransmitting a previously accepted submission with a future timestamp.
3. Sybil flooding — registering hundreds of devices under sock-puppet accounts to dilute honest data.
4. Insider direct mutation of aggregated records.
5. MitM interception and substitution of submission payloads in transit.

---

## Defense-in-Depth Architecture

### 1. Submission Validation Layer

Every inbound submission is validated before it touches any persistent store. The schema is enforced with Pydantic strict-mode models. Numeric fields (tokens/sec, VRAM used, latency) are checked against hardware-tier bounds — a consumer RTX 4090 cannot legitimately report 500,000 tokens/sec for a 70B Q4 model. Values outside tier-specific envelope bounds are rejected with a `422` and logged.

Each device token carries an HMAC signature computed at registration time. The aggregator verifies the HMAC on every submission; unsigned or incorrectly signed payloads are dropped before schema validation. A per-device-token rate limit (one submission per benchmark run type per 6-hour window) prevents high-frequency flooding. Submissions must include a UTC timestamp; the aggregator rejects any submission whose timestamp falls outside a ±5-minute window of server time, defeating replay attacks.

### 2. Sybil Resistance

Device tokens are issued only to accounts in good standing with a completed billing instrument and at least one verified payment. At registration time, the client library generates a hardware fingerprint hash (GPU UUID, CPU model hash, board serial if available, salted with the account ID) and submits it alongside the device token. The aggregator stores the fingerprint hash — not the raw identifiers — and deduplicates submissions: if two device tokens present the same fingerprint hash, the second registration is flagged for manual review and its submissions are quarantined from the live aggregate until resolved.

### 3. Aggregation-Time Protections

Raw per-node records are never surfaced through any API. Before any benchmark tier is published, the aggregator verifies that at least **k=500** distinct device tokens contributed to that hardware-tier bucket. Tiers with fewer than 500 contributors are withheld from the recommendation engine until the threshold is met.

Published aggregate statistics are protected with differential privacy (Laplace mechanism, ε≤0.5). This bounds the influence any single node's data can have on a published figure, and prevents reconstruction attacks that attempt to reverse-engineer individual submissions from repeated queries.

Aggregation uses the **median** rather than the mean as the central statistic. The median is inherently outlier-resistant: an adversary controlling k nodes in a population of N can only shift the aggregate median if k > N/2. Prior to computing the median, a z-score filter removes submissions more than 3σ from the current hardware-tier median, providing an additional layer of protection against targeted inflation or deflation attempts. A rate-of-change monitor alerts on-call engineers when a hardware-tier median shifts by more than 15% within any 24-hour window — a signal that a coordinated campaign may be underway.

### 4. Infrastructure Isolation

The CIS aggregator runs in a dedicated network segment with no egress to the billing or authentication databases. Operator nodes have write-only access to the submission endpoint; no node can query another node's data. The recommendation engine has read-only access to the aggregated output tables and cannot reach the raw submission log. Lateral movement from the aggregator cannot reach credential stores or billing records.

### 5. Audit Trail

All submissions are recorded with the device token hash (BLAKE3 of the token, not the plaintext), submission IP, hardware fingerprint hash, and a monotonically increasing sequence number. Anomaly detection runs over the audit log every 15 minutes, flagging accounts with submission burst patterns inconsistent with normal benchmark cadence. The audit log is append-only and replicated to a separate storage account that the aggregator process cannot delete from.

---

## Key Controls Table

| Threat | Control | Implementation |
|---|---|---|
| Sybil attack | Device token bound to paid account + hardware fingerprint hash deduplication | Registration gating; fingerprint stored as BLAKE3 hash; duplicates quarantined |
| Benchmark poisoning | Median aggregation + z-score filter (>3σ rejected) | Pre-aggregation outlier pass; median replaces mean in all published stats |
| De-anonymization | k-anonymity k≥500 before any tier is published | Aggregator withholds tier data until 500 distinct tokens contributed |
| Differential privacy | Laplace noise ε≤0.5 on all published benchmark figures | Applied at query time against the aggregated table |
| Insider mutation | Network isolation; append-only audit log; separate read replica for rec engine | Aggregator process has no DELETE/UPDATE on audit log; rec engine is read-only |
| MitM submissions | HTTPS + HMAC-signed device token on every submission | Server rejects any payload with invalid or missing HMAC before processing |
| Replay attacks | Submission timestamp window ±5 minutes | Aggregator rejects submissions outside window; sequence number prevents re-use |
| Supply chain tampering | HMAC computed on measured benchmark output; open-source client library | Independent benchmark signing; community can audit measurement code |
| Rate-based flooding | Per-device-token rate limit (1 per benchmark type per 6h) | Enforced at API gateway before aggregator process receives request |
| Sudden median shift | Rate-of-change monitor; alert on >15% shift in 24h | Background job; pages on-call; optionally freezes tier pending investigation |

---

## Acceptance Criteria

- [ ] No raw per-node benchmark data is exposed via any API endpoint, internal or external.
- [ ] Aggregated data for a hardware tier requires k≥500 contributing device tokens before the tier is published to the recommendation engine.
- [ ] Sybil submissions sharing the same hardware fingerprint hash are deduplicated and quarantined for review.
- [ ] The z-score outlier filter rejects benchmark submissions more than 3σ from the current hardware-tier median before those submissions influence the aggregate.
- [ ] The CIS aggregator process has no direct database access to billing or authentication tables; confirmed by network policy and integration test.
- [ ] All submissions must include an HMAC-signed device token; the aggregator rejects any submission with an invalid or missing HMAC before schema validation.
- [ ] Replay attack protection: submissions with timestamps outside ±5 minutes of server time are rejected with a logged `422`.
- [ ] Differential privacy (ε≤0.5, Laplace mechanism) is applied to all published aggregate benchmark figures.
- [ ] The audit log is append-only and replicated to a storage account the aggregator process cannot modify or delete.
- [ ] A rate-of-change alert fires when any hardware-tier median shifts >15% within a 24-hour window.

---

## Out of Scope

This document covers the aggregation pipeline and its direct interfaces. It does not cover client-side measurement integrity (addressed separately under supply chain attestation work), the recommendation engine's own security properties, or the general platform authentication hardening covered in `docs/plans/2026-02-22-trustshield-security-design.md`.
