# System Architecture Document
## aci-cache — End-to-End Technical Architecture

**Version:** 1.0 | **Date:** May 9, 2026 | **Status:** Draft

---

## 1. Architecture Overview

### 1.1 What aci-cache IS

A **library** (not a service, not a proxy, not a sidecar) that lives **inside** the developer's application process. It wraps their Redis client, intercepts `get`/`set`/`delete` calls, and adds an adaptive invalidation layer.

### 1.2 What aci-cache is NOT

- Not a separate running process or container
- Not a Redis proxy that sits between app and Redis
- Not a database caching layer (doesn't auto-cache queries)
- Not a Redis replacement

### 1.3 High-Level Architecture

```
┌──────────────────────────── App Instance 1 ──────────────────────────┐
│                                                                       │
│   Developer's Code                                                    │
│   ┌──────────────────────────────────────────────────────────────┐    │
│   │  user = cache.get("user:123")                                │    │
│   │  cache.set("user:123", updated_user)                         │    │
│   └─────────────────────┬────────────────────────────────────────┘    │
│                         │                                             │
│   ┌─────────────────────▼────────────────────────────────────────┐   │
│   │                    AdaptiveCache                              │   │
│   │  ┌──────────┐  ┌──────────────┐  ┌────────────────────────┐  │   │
│   │  │Controller│  │Strategy      │  │Pub/Sub Coordinator     │  │   │
│   │  │(bg thread│  │Engine        │  │(bg thread)             │  │   │
│   │  │ /timer)  │  │              │  │                        │  │   │
│   │  │          │  │ ┌──────────┐ │  │  Subscribe:            │  │   │
│   │  │ Monitors │  │ │TTL      ││ │  │  ├─ aci:invalidation  │  │   │
│   │  │ write    │  │ │Eager    ││ │  │  └─ aci:strategy      │  │   │
│   │  │ rate     │  │ │Batched  ││ │  │                        │  │   │
│   │  │          │  │ └──────────┘ │  │  On message:           │  │   │
│   │  │ Switches │  │              │  │  → cache.delete(key)   │  │   │
│   │  │ strategy │  │ Executes the │  │  → switch strategy     │  │   │
│   │  └────┬─────┘  │ active one   │  └───────────┬────────────┘  │   │
│   │       │        └──────┬───────┘              │               │   │
│   └───────┼───────────────┼──────────────────────┼───────────────┘   │
│           │               │                      │                    │
└───────────┼───────────────┼──────────────────────┼────────────────────┘
            │               │                      │
            ▼               ▼                      ▼
     ┌─────────────────────────────────────────────────┐
     │                  Redis Server                    │
     │  ┌──────────────┐  ┌──────────────────────────┐ │
     │  │  Key-Value   │  │  Pub/Sub Channels        │ │
     │  │  Store       │  │  ├─ aci:invalidation     │ │
     │  │              │  │  └─ aci:strategy         │ │
     │  └──────────────┘  └──────────────────────────┘ │
     └─────────────────────────────────────────────────┘
```

---

## 2. Component Architecture

### 2.1 Component Diagram

```mermaid
graph TB
    subgraph "AdaptiveCache (Public API)"
        A[get / set / delete / flush]
    end
    
    subgraph "Core Engine"
        B[Strategy Engine]
        C[Write Rate Tracker]
    end
    
    subgraph "Strategies"
        D[TTL Strategy]
        E[Eager Strategy]
        F[Batched Strategy]
    end
    
    subgraph "Background Workers"
        G[Controller Thread]
        H[Subscriber Thread]
        I[Batch Flusher Thread]
    end
    
    subgraph "External"
        J[(Redis Key-Value)]
        K[Redis Pub/Sub]
    end
    
    A --> B
    A --> C
    B --> D
    B --> E
    B --> F
    C --> G
    G -->|switches| B
    G -->|publishes| K
    H -->|listens| K
    H -->|invalidates| J
    H -->|switches| B
    E -->|publishes| K
    F --> I
    I -->|publishes| K
    A -->|read/write| J
```

### 2.2 Component Responsibilities

| Component | Responsibility | Runs On |
|-----------|---------------|---------|
| **AdaptiveCache** | Public API surface. Routes calls through strategy engine. | Main thread |
| **Strategy Engine** | Holds reference to active strategy, executes `on_write` hooks | Main thread |
| **Write Rate Tracker** | Appends timestamps on write, exposes count-in-window | Main thread (append), Controller (read) |
| **Controller** | Periodically calculates write rate, selects strategy, publishes update | Background thread (Python) / `setInterval` (Node.js) |
| **Subscriber** | Listens on pub/sub channels, triggers local cache deletes + strategy switches | Background thread (Python) / event listener (Node.js) |
| **Batch Flusher** | Periodically drains the key buffer and publishes invalidation | Background thread (Python) / `setInterval` (Node.js) |

---

## 3. Data Flows

### 3.1 Read Flow

```
Developer calls cache.get("user:123")
           │
           ▼
┌─ AdaptiveCache.get() ─────────────────┐
│                                        │
│  1. Construct full key:                │
│     namespace? → "aci:myapp:user:123"  │
│     no namespace? → "user:123"         │
│                                        │
│  2. redis_client.get(full_key)         │
│     ├─ HIT  → stats.hits++, return val │
│     └─ MISS → stats.misses++, return ∅ │
│                                        │
│  Note: aci-cache does NOT auto-fill    │
│  cache on miss. That's the dev's job.  │
│  We only manage invalidation.          │
└────────────────────────────────────────┘
```

> [!IMPORTANT]
> aci-cache is an **invalidation** library, not a cache-aside library. It doesn't decide what to cache or when to populate. The developer still writes their own cache-miss → DB-fetch → cache-fill logic. aci-cache just makes sure stale entries get removed intelligently.

### 3.2 Write Flow

```
Developer calls cache.set("user:123", data)
           │
           ▼
┌─ AdaptiveCache.set() ──────────────────────────┐
│                                                  │
│  1. redis_client.setex(key, TTL, value)          │
│                                                  │
│  2. write_tracker.record(timestamp)              │
│     (append to sliding window list)              │
│                                                  │
│  3. current_strategy.on_write(key)               │
│     ├─ TTL:     no-op (rely on expiry)           │
│     ├─ EAGER:   publish("aci:invalidation",      │
│     │           {keys: ["user:123"]})             │
│     └─ BATCHED: buffer.append("user:123")        │
│                                                  │
│  4. stats.writes++                               │
└──────────────────────────────────────────────────┘
```

### 3.3 Cross-Instance Invalidation Flow

```
Instance A                    Redis Pub/Sub              Instance B
    │                              │                          │
    │  cache.set("user:123", d)   │                          │
    │  strategy = EAGER            │                          │
    │                              │                          │
    │  publish("aci:invalidation", │                          │
    │   {keys:["user:123"],        │                          │
    │    source:"inst_A"})         │                          │
    │ ───────────────────────────▶ │                          │
    │                              │  deliver to subscribers  │
    │                              │ ───────────────────────▶ │
    │                              │                          │
    │                              │     Subscriber thread:   │
    │                              │     1. Parse message     │
    │                              │     2. source == me?     │
    │                              │        NO → proceed      │
    │                              │     3. redis.del("user:123")
    │                              │        (on Instance B's Redis)
    │                              │                          │
    │                              │     Next get("user:123") │
    │                              │     → MISS → dev fetches │
    │                              │       fresh from DB      │
```

### 3.4 Strategy Switch Flow

```
Controller (runs on one instance)
    │
    │  Every 3 seconds:
    │  1. Count writes in last 5s window
    │  2. write_rate = count / 5.0
    │  3. Select strategy:
    │     rate > 50 → eager
    │     rate < 10 → ttl
    │     else      → batched
    │
    │  If strategy changed:
    │  4. Apply locally: current_strategy = new
    │  5. Publish to all instances:
    │     publish("aci:strategy", {
    │       strategy: "eager",
    │       write_rate: 65.4
    │     })
    │
    │  All instances (including non-controllers):
    │  6. Subscriber receives strategy update
    │  7. Switches local strategy
    │  8. If switching FROM batched: flush buffer
    │  9. If switching TO batched: start flusher
```

---

## 4. Threading / Concurrency Model

### 4.1 Python Threading Model

```
┌─────────────────────────────────────────────────────────────┐
│              Developer's Application Process                 │
│                                                              │
│  ┌──────────────────────────────────────────────────┐       │
│  │  MAIN THREAD(s) — Developer's web framework      │       │
│  │  (Django/FastAPI/Flask with multiple workers)     │       │
│  │                                                    │       │
│  │  Calls: cache.get(), cache.set(), cache.delete()  │       │
│  └──────────────────────────────────────────────────┘       │
│                                                              │
│  ┌──────────────────────────────────────────────────┐       │
│  │  DAEMON THREAD 1: Pub/Sub Subscriber              │       │
│  │  Blocking loop: pubsub.listen()                   │       │
│  │  Calls: redis.delete(key) on invalidation         │       │
│  │  Calls: switch_strategy() on strategy update      │       │
│  └──────────────────────────────────────────────────┘       │
│                                                              │
│  ┌──────────────────────────────────────────────────┐       │
│  │  DAEMON THREAD 2: Controller (if is_controller)   │       │
│  │  Periodic loop: sleep(controller_interval)        │       │
│  │  Reads: write_timestamps (with lock)              │       │
│  │  Publishes: strategy updates                      │       │
│  └──────────────────────────────────────────────────┘       │
│                                                              │
│  ┌──────────────────────────────────────────────────┐       │
│  │  DAEMON THREAD 3: Batch Flusher                   │       │
│  │  Active ONLY when strategy == "batched"           │       │
│  │  Periodic loop: sleep(batch_interval)             │       │
│  │  Drains buffer, publishes invalidation            │       │
│  └──────────────────────────────────────────────────┘       │
└──────────────────────────────────────────────────────────────┘
```

**Shared State & Locking:**

| Variable | Accessed By | Protection |
|----------|------------|------------|
| `current_strategy` | All threads | `threading.Lock` — read/write under lock |
| `write_timestamps` | Main (append), Controller (read+trim) | `threading.Lock` |
| `batch_buffer` | Main (append), Flusher (drain) | `threading.Lock` — release before publish |
| `stats` | Main (increment), any (read) | `threading.Lock` or `threading.atomic` counters |

### 4.2 Node.js Concurrency Model

```
┌─────────────────────────────────────────────────────────────┐
│              Developer's Node.js Process                     │
│                                                              │
│  ┌──────────────────────────────────────────────────┐       │
│  │  EVENT LOOP — Single threaded                     │       │
│  │                                                    │       │
│  │  cache.get() / cache.set()                        │       │
│  │    → All async, returns Promises                  │       │
│  │    → No threading issues (single event loop)      │       │
│  │                                                    │       │
│  │  Controller: setInterval(checkRate, 3000)         │       │
│  │  Flusher:    setInterval(flushBatch, 2500)        │       │
│  │  Subscriber: redis.subscribe() — event-driven     │       │
│  │                                                    │       │
│  │  No locks needed — everything is single-threaded  │       │
│  └──────────────────────────────────────────────────┘       │
└──────────────────────────────────────────────────────────────┘
```

> [!NOTE]
> Node.js is dramatically simpler for this use case. No locks, no threading concerns, no race conditions. The event loop handles everything.

---

## 5. Multi-Instance Coordination

### 5.1 The Leader Problem

Only ONE instance should run the Controller (to avoid conflicting strategy decisions). Two approaches:

**Approach A: Config-Based (v0.1 — Simple)**
```python
# Instance 1 (primary)
cache = AdaptiveCache(redis, is_controller=True)

# Instance 2-N (followers)
cache = AdaptiveCache(redis, is_controller=False)
```

**Approach B: Redis-Based Leader Election (v1.0 — Automatic)**
```python
# All instances try to acquire a Redis lock
# SET aci:controller:lock <instance_id> NX EX 10
# Winner runs controller, losers check every 10s
# If lock expires (leader died), another takes over
```

### 5.2 Instance Discovery

Instances don't need to know about each other. They coordinate entirely through Redis Pub/Sub channels:

```
Instance A ───publish──▶ Redis Pub/Sub ───deliver──▶ Instance B
Instance A ◀──subscribe── Redis Pub/Sub ◀──publish── Instance C
```

No service registry, no DNS, no API calls between instances.

### 5.3 Same Redis vs Separate Redis

```
Option 1: Same Redis (simpler, recommended for most)
┌──────┐     ┌──────┐     ┌──────┐
│Inst A│     │Inst B│     │Inst C│
└──┬───┘     └──┬───┘     └──┬───┘
   │            │            │
   └────────────┼────────────┘
                │
         ┌──────▼──────┐
         │   Redis     │  ← Cache + Pub/Sub on same instance
         └─────────────┘

Option 2: Separate Redis (production at scale)
┌──────┐     ┌──────┐     ┌──────┐
│Inst A│     │Inst B│     │Inst C│
└──┬───┘     └──┬───┘     └──┬───┘
   │   \        │   \        │   \
   │    \       │    \       │    \
   ▼     ▼      ▼     ▼     ▼     ▼
┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐
│R_a │ │Pub │ │R_b │ │Pub │ │R_c │ │Pub │
│loc.│ │Sub │ │loc.│ │Sub │ │loc.│ │Sub │
└────┘ └─┬──┘ └────┘ └─┬──┘ └────┘ └─┬──┘
         │             │             │
         └─────────────┼─────────────┘
                       │
              Shared Pub/Sub Redis
```

---

## 6. Package Structure

### 6.1 Python Package (`aci-cache`)

```
aci-cache/
├── pyproject.toml              # Package metadata (PEP 621)
├── README.md
├── LICENSE                     # MIT
├── src/
│   └── aci_cache/
│       ├── __init__.py         # Exports: AdaptiveCache, AsyncAdaptiveCache
│       ├── cache.py            # AdaptiveCache class (public API)
│       ├── async_cache.py      # AsyncAdaptiveCache (asyncio variant)
│       ├── controller.py       # Write rate monitor + strategy selector
│       ├── subscriber.py       # Pub/Sub listener + invalidation handler
│       ├── tracker.py          # Write rate sliding window tracker
│       ├── config.py           # Configuration dataclass + validation
│       ├── stats.py            # CacheStats dataclass + collection
│       ├── types.py            # Type aliases and protocols
│       └── strategies/
│           ├── __init__.py
│           ├── base.py         # Strategy protocol/ABC
│           ├── ttl.py          # TTL strategy (no-op on_write)
│           ├── eager.py        # Eager strategy (publish per write)
│           └── batched.py      # Batched strategy (buffer + flush)
├── tests/
│   ├── conftest.py             # Shared fixtures (mock Redis, etc.)
│   ├── test_cache.py           # AdaptiveCache unit tests
│   ├── test_controller.py      # Controller logic tests
│   ├── test_strategies.py      # Strategy behavior tests
│   ├── test_subscriber.py      # Pub/Sub handling tests
│   ├── test_config.py          # Config validation tests
│   ├── test_integration.py     # Multi-instance integration tests
│   └── test_benchmark.py       # Performance benchmarks
└── docs/
    ├── quickstart.md
    ├── configuration.md
    ├── strategies.md
    └── deployment.md
```

**pyproject.toml (key sections):**
```toml
[project]
name = "aci-cache"
version = "0.1.0"
description = "Adaptive cache invalidation for Redis. Drop-in wrapper, zero config."
requires-python = ">=3.9"
license = {text = "MIT"}
dependencies = ["redis>=4.0"]

[project.optional-dependencies]
async = ["redis[hiredis]>=4.0"]
dev = ["pytest", "pytest-asyncio", "fakeredis", "ruff", "mypy"]
```

### 6.2 Node.js Package (`aci-cache`)

```
aci-cache/
├── package.json
├── tsconfig.json
├── README.md
├── LICENSE                      # MIT
├── src/
│   ├── index.ts                 # Exports: AdaptiveCache
│   ├── cache.ts                 # AdaptiveCache class (public API)
│   ├── controller.ts            # Write rate monitor + strategy selector
│   ├── subscriber.ts            # Pub/Sub listener + invalidation handler
│   ├── tracker.ts               # Write rate sliding window tracker
│   ├── config.ts                # Configuration interface + validation
│   ├── stats.ts                 # CacheStats interface + collection
│   ├── types.ts                 # Type definitions
│   └── strategies/
│       ├── index.ts
│       ├── base.ts              # Strategy interface
│       ├── ttl.ts
│       ├── eager.ts
│       └── batched.ts
├── dist/                        # Compiled JS + .d.ts (generated)
├── tests/
│   ├── cache.test.ts
│   ├── controller.test.ts
│   ├── strategies.test.ts
│   ├── subscriber.test.ts
│   ├── integration.test.ts
│   └── benchmark.test.ts
└── docs/
    ├── quickstart.md
    ├── configuration.md
    └── deployment.md
```

**package.json (key sections):**
```json
{
  "name": "aci-cache",
  "version": "0.1.0",
  "description": "Adaptive cache invalidation for Redis. Drop-in wrapper, zero config.",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "files": ["dist/"],
  "peerDependencies": {
    "ioredis": ">=5.0.0"
  },
  "devDependencies": {
    "typescript": "^5.0",
    "vitest": "^1.0",
    "ioredis-mock": "^8.0"
  }
}
```

---

## 7. Deployment Patterns

### 7.1 Single App, Multiple Workers (Most Common)

```
                    Load Balancer
                  ┌───┼───┼───┐
                  │   │   │   │
              Worker1 W2  W3  W4    ← All same app, all have aci-cache
                  │   │   │   │
                  └───┼───┼───┘
                      │
               ┌──────▼──────┐
               │    Redis    │
               └─────────────┘

Worker1: AdaptiveCache(redis, is_controller=True)
Worker2-4: AdaptiveCache(redis, is_controller=False)
```

### 7.2 Microservices, Shared Redis

```
   User Service        Order Service       Product Service
   ┌───────────┐       ┌────────────┐      ┌───────────────┐
   │aci-cache  │       │aci-cache   │      │aci-cache      │
   │ns="users" │       │ns="orders" │      │ns="products"  │
   └─────┬─────┘       └─────┬──────┘      └──────┬────────┘
         │                   │                     │
         └───────────────────┼─────────────────────┘
                             │
                      ┌──────▼──────┐
                      │   Redis     │
                      └─────────────┘

Each service uses a different namespace.
Invalidation messages only affect the same namespace.
Each service independently adapts its strategy.
```

### 7.3 Kubernetes

```yaml
# No special K8s config needed. aci-cache is just a library.
# The only consideration is leader election for the controller.

apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 5
  template:
    spec:
      containers:
      - name: my-app
        env:
        - name: ACI_IS_CONTROLLER
          value: "auto"    # Uses Redis-based leader election
        - name: ACI_HIGH_THRESHOLD
          value: "50"
```

---

## 8. Failure Modes & Recovery

```mermaid
graph TD
    A[Redis Cache Down] -->|cache.get returns null| B[App falls through to DB]
    A -->|cache.set throws| C[App handles error]
    
    D[Redis Pub/Sub Down] -->|Subscriber detects| E[Auto-reconnect with backoff]
    D -->|During downtime| F[No invalidation messages]
    F -->|Safety net| G[TTL expiry still works]
    
    H[Controller Thread Crash] -->|Strategy freezes| I[Last known strategy continues]
    H -->|Logged as error| J[App continues serving]
    
    K[Batch Flusher Crash] -->|Buffer lost| L[TTL safety net handles staleness]
    
    M[Instance Goes Down] -->|Other instances unaffected| N[Missed messages until TTL]
```

---

## 9. Performance Budget

| Operation | Raw Redis | With aci-cache | Overhead |
|-----------|-----------|----------------|----------|
| `get(key)` | ~0.3ms | ~0.4ms | +0.1ms (namespace prefixing + stats) |
| `set(key)` — TTL mode | ~0.4ms | ~0.5ms | +0.1ms (timestamp tracking + stats) |
| `set(key)` — Batched mode | ~0.4ms | ~0.5ms | +0.1ms (buffer append is O(1)) |
| `set(key)` — Eager mode | ~0.4ms | ~1.2ms | +0.8ms (pub/sub publish on critical path) |
| Memory per instance | — | ~2-4MB | Controller state + sliding window + buffer |
| Background CPU | — | < 1% core | Controller + subscriber + flusher |

> [!NOTE]
> Eager mode has the highest overhead because it performs a Redis `PUBLISH` on every `set()` call. This is intentional — eager mode is only activated during high write rates where consistency matters more than latency.

---

## 10. Security Considerations

| Concern | Mitigation |
|---------|-----------|
| Pub/Sub messages contain key names | Keys are identifiers only, no values are transmitted |
| Another app could publish fake invalidation messages | Use namespaced channels; optionally add HMAC signing (v1.0) |
| Redis AUTH | Library passes through auth to underlying redis client |
| Redis TLS | Library passes through TLS to underlying redis client |
| Denial of service via mass invalidation | Rate-limit pub/sub message processing (configurable) |

---

## 11. Migration Path from Current ACI Project

| Current ACI Component | Maps To | Changes Required |
|----------------------|---------|-----------------|
| `app/controller.py` → `_loop()` | `src/aci_cache/controller.py` | Remove Flask coupling, accept callbacks |
| `app/strategies/*` | `src/aci_cache/strategies/*` | Remove `os.environ`, accept config object |
| `app/subscriber.py` | `src/aci_cache/subscriber.py` | Remove `cache_delete` import, accept callback |
| `app/cache.py` | `src/aci_cache/cache.py` | Becomes the public `AdaptiveCache` class |
| `app/db.py` | **Removed** | Users provide their own DB layer |
| `app/app.py` | **Removed** | Users have their own app |
| `load_gen/*` | Moves to `tests/benchmark/` | Becomes the benchmark test suite |
| `dashboard/*` | Becomes optional CLI tool | `aci dashboard` command (v1.0) |

---

**END OF ARCHITECTURE DOCUMENT**
