# Meshulash Guard SDK — Design Reference

> Living document capturing architectural decisions, server mapping, and future roadmap for the meshulash-guard Python package.

---

## What This Is

A Python client SDK (`meshulash-guard`) that gives developers a clean, Pythonic interface to the Meshulash security engine. The heavy lifting (NER models, text classification, regex detection, validation) runs server-side. The SDK is a **translation layer** — it takes developer-friendly scanner configurations, translates them into guardline specs the server understands, sends them over HTTP, and returns clean result objects.

### Deployment Model

Customers either:
- Run the security server **on-prem**
- Use the **cloud-hosted** server (GCP Cloud Run, per-tenant isolation)

```python
# Cloud customer (most common)
guard = Guard(api_key="sk-xxx", tenant_id="meshulash-3lqgn")

# On-prem customer (overrides server URL)
guard = Guard(api_key="sk-xxx", tenant_id="meshulash-3lqgn", server_url="https://security.customer.com")
```

### Production Routing (GCP)

```
Client SDK
    ↓ POST /api/security/scan
    ↓ Headers: X-Api-Key, X-Tenant-Id
    ↓
GCP Global HTTPS Load Balancer
    ↓ Routes by: URL path (/api/security/*) + X-Tenant-Id header match
    ↓ Each tenant has a route rule mapping to their backend service
    ↓
Tenant's Cloud Run Security Server (isolated: own VPC, SQL, service account)
    ↓ Validates API key via backend call (backend is always on GCP)
    ↓ Runs 5-stage pipeline with inline guardline_specs from SDK
    ↓
Response back to SDK
```

**On-prem flow:** Same chain but client hits the on-prem server directly (no LB). The on-prem security server still calls the cloud backend for API key validation.

**Key facts:**
- `tenant_id` is always required — it routes requests (cloud) and identifies the customer (on-prem)
- `api_key` is validated by the cloud backend, not the security server itself
- The backend always lives on GCP, even for on-prem customers
- Admin overrides the security server URL from the UI for on-prem deployments

---

## Architecture Overview

```
Developer Code
    ↓
meshulash-guard SDK          (this repo — translation layer + types)
    ↓  translate scanner configs → guardline_specs
    ↓  HTTP POST with JSON body
Security Server              (AIM repo — ML models, NER, classifiers, validators)
    ↓  5-stage pipeline: Recognize → Validate → Score → Gate → Act
    ↓  JSON response
SDK Result Objects           (clean types back to developer)
```

### The Translation Pattern

The SDK's core job: each scanner knows how to translate itself into a guardline spec.

```python
# What the developer writes:
pii = PIIScanner(
    labels=[PIILabel.EMAIL, PIILabel.PHONE],
    action=Action.REPLACE,
    condition=Condition.ANY,
)

# What the SDK sends to the server (guardline_specs):
{
    "sdk_pii_1": {
        "name": "PIIScanner",
        "condition": "any",
        "action": "replace",
        "level": 2,
        "types": {"regex": True, "ner": True, "tc": False},
        "required": {
            "regex": ["EMAIL_ADDRESS", "PHONE_NUMBER"],
            "ner": ["EMAIL_ADDRESS", "PHONE_NUMBER"],
            "tc": []
        },
        "allowlist": [],
        "bundles": []
    }
}
```

---

## Server Capabilities (What Already Exists)

### 5-Stage Pipeline

| Stage | What It Does | Configurable via SDK? |
|-------|-------------|----------------------|
| **1. Recognize** | Regex + NER + Text Classification detect matches | Yes — guardline_specs controls what to look for |
| **2. Validate** | Format validators confirm matches (Luhn, phone, email, etc.) | No (v1) — auto-runs per label based on label registry |
| **3. Score** | Confidence scoring, filter low-quality matches | No (v1) — uses label registry metadata |
| **4. Gate** | Evaluate guardline conditions against scored matches | Yes — condition, required items, k value |
| **5. Act** | BLOCK / REPLACE / LOG decision | Yes — action per scanner |

### 53 Canonical Labels (Label Registry)

The server's `LABEL_REGISTRY` defines all detectable entity types. Each label has:
- **Detection sources**: `regex`, `ner`, or both
- **Validation mode**: `TRUSTED_REGEX` (regex alone sufficient), `VALIDATOR_REQUIRED` (must pass checksum/format), `NER_ONLY` (pure ML detection)
- **Validators**: specific validation functions (Luhn, phone format, etc.)
- **Bundles**: which risk bundles include this label (PII, PHI, PCI, SECRETS, TECH)

#### Identity (7 labels)
| Label | Detection | Validators |
|-------|-----------|------------|
| `PERSON_NAME` | NER | — |
| `NATIONAL_ID` | regex + NER | US SSN checksum, Israeli ID checksum |
| `DRIVER_LICENSE` | regex + NER | US/IL format validation |
| `USERNAME` | NER | — |
| `TAX_NUMBER` | NER | — |
| `ACCOUNT_ID` | NER | — |
| `BUSINESS_ID` | NER | — |

#### Contact (11 labels)
| Label | Detection | Validators |
|-------|-----------|------------|
| `EMAIL_ADDRESS` | regex + NER | RFC 5321/5322 |
| `PHONE_NUMBER` | regex + NER | libphonenumber (IL/US priority) |
| `STREET_ADDRESS` | NER | — |
| `CITY` | NER | — |
| `COUNTY` | NER | — |
| `STATE` | NER | — |
| `COUNTRY` | NER | — |
| `POSTAL_CODE` | NER | — |
| `IP_ADDRESS` | regex | IPv4/IPv6 format |
| `MAC_ADDRESS` | regex | MAC format |
| `URL` | regex + NER | URL format |

#### Financial (5 labels)
| Label | Detection | Validators |
|-------|-----------|------------|
| `CREDIT_CARD` | regex + NER | Luhn checksum |
| `IBAN` | regex | — |
| `BANK_ACCOUNT` | regex + NER | US/IL format |
| `ROUTING_NUMBER` | regex + NER | ABA checksum |
| `CRYPTO_ADDRESS` | NER | — |

#### Credentials (20 labels)
| Label | Detection | Validators |
|-------|-----------|------------|
| `PASSWORD` | regex + NER | Entropy check |
| `SECRET_KEY` | regex | Entropy check |
| `ACCESS_TOKEN` | regex | Entropy check |
| `PRIVATE_KEY` | regex | Entropy check |
| `CONNECTION_STRING` | regex | URL format |
| `AWS_ACCESS_KEY` | regex | — |
| `AWS_SECRET_KEY` | regex | Entropy check |
| `GCP_API_KEY` | regex | — |
| `GCP_SERVICE_ACCOUNT` | regex | — |
| `OPENAI_API_KEY` | regex | Entropy check |
| `AZURE_API_KEY` | regex | Entropy check |
| `STRIPE_SECRET_KEY` | regex | Entropy check |
| `STRIPE_PUBLISHABLE_KEY` | regex | Entropy check |
| `SLACK_TOKEN` | regex | Entropy check |
| `GITHUB_TOKEN` | regex | Entropy check |
| `TWILIO_API_KEY` | regex | — |
| `SENDGRID_API_KEY` | regex | — |
| `HEROKU_API_KEY` | regex | — |
| `API_KEY` | NER | — |
| `TOKEN` | NER | — |

#### Technical (2 labels)
| Label | Detection | Validators |
|-------|-----------|------------|
| `USER_AGENT` | regex | — |
| `AWS_ARN` | regex | — |

#### Temporal (1 label)
| Label | Detection | Validators |
|-------|-----------|------------|
| `DATE_TIME` | regex | — |

#### Organizational (1 label)
| Label | Detection | Validators |
|-------|-----------|------------|
| `ORGANIZATION` | NER | — |

### 5 Text Classification Heads

| Head (internal) | Purpose | Status |
|-----------------|---------|--------|
| `topic` | General topic classification | Exists |
| `emotion` | Emotional content detection | Exists |
| `hate_speech` | Toxicity / hate speech | Exists |
| `cysecbench` | Cybersecurity threat classification | Exists |
| `jailbreak` | Prompt injection / jailbreak detection | Coming soon |

> TC heads are **hidden from the developer**. The SDK exposes label enums per scanner, and internally maps each label to its head.

### Risk Bundles (Pre-grouped Label Sets)

| Bundle | Contains |
|--------|----------|
| `PII` | Identity + Contact + Financial + some Credentials |
| `PHI` | Protected Health Information (separate model) |
| `PCI` | Financial labels (CREDIT_CARD, IBAN, BANK_ACCOUNT, ROUTING_NUMBER) |
| `SECRETS` | All credential labels |
| `TECH` | USER_AGENT, AWS_ARN |

### Gating Conditions

| Condition | Logic |
|-----------|-------|
| `any` | At least one detection from any enabled recognizer type |
| `all` | All required items must be detected |
| `k_of` | K or more total hits across all types |
| `contextual_analysis` | TC must trigger AND (NER or regex must trigger) |

### Actions

| Action | Server Behavior |
|--------|----------------|
| `replace` | Redact detected text with placeholders `[LABEL-HASH]`, status = `SECURED` |
| `block` | Return original text unmodified, status = `BLOCKED` |
| `log` | Return original text unmodified, status = `CLEAN` (logged server-side) |

### Validators (16+ Auto-Run)

Validators run automatically per label based on the label registry's `validation_mode`. They are **not configurable in v1** — this is by design, as they reduce false positives.

| Validator | Used By |
|-----------|---------|
| `validate_email` | EMAIL_ADDRESS |
| `validate_phone` | PHONE_NUMBER |
| `validate_credit_card` | CREDIT_CARD |
| `validate_us_ssn` | NATIONAL_ID |
| `validate_israeli_id` | NATIONAL_ID |
| `validate_ipv4` / `validate_ipv6` | IP_ADDRESS |
| `validate_mac` | MAC_ADDRESS |
| `validate_url` | URL, CONNECTION_STRING |
| `validate_us_routing_number` | ROUTING_NUMBER |
| `validate_bank_account_us_il` | BANK_ACCOUNT |
| `validate_driver_license_us_il` | DRIVER_LICENSE |
| `validate_entropy_secret` | PASSWORD, SECRET_KEY, ACCESS_TOKEN, PRIVATE_KEY, and most credential labels |
| `validate_passport_us_il` | (available, not mapped to label) |
| `validate_imei` | (available, not mapped to label) |
| `validate_date` / `validate_time` | DATE_TIME |

---

## SDK Design

### Developer API

```python
from meshulash_guard import Guard
from meshulash_guard.scanners.pii import PIIScanner, PIILabel
from meshulash_guard.scanners.topic import TopicScanner, TopicLabel
from meshulash_guard.scanners.toxicity import ToxicityScanner, ToxicityLabel
from meshulash_guard.scanners.cyber import CyberScanner, CyberLabel
from meshulash_guard.scanners.jailbreak import JailbreakScanner, JailbreakLabel
from meshulash_guard.actions import Action, Condition

# Initialize
guard = Guard(
    server_url="https://security.company.com",
    api_key="sk-xxx"
)

# Configure scanners — labels scoped to each scanner
pii = PIIScanner(
    labels=[PIILabel.EMAIL, PIILabel.PHONE, PIILabel.CREDIT_CARD],
    action=Action.REPLACE,
    condition=Condition.ANY,
)

topics = TopicScanner(
    labels=[TopicLabel.PHISHING, TopicLabel.MALWARE],
    action=Action.BLOCK,
    threshold=0.7,
)

# Scan input
result = guard.scan_input(
    text="My email is john@test.com",
    scanners=[pii, topics],
)

# Result object
result.status           # "clean" | "secured" | "blocked"
result.processed_text   # "My email is [EMAIL_ADDRESS-A1B2]"
result.placeholders     # {"[EMAIL_ADDRESS-A1B2]": "john@test.com"}
result.level            # 0-5 severity
result.scanners         # per-scanner breakdown
result.risk_categories  # ["PII"]

# Deanonymize LLM response (client-side, no server call)
restored = guard.deanonymize(llm_response_text)
```

### Placeholder Cache / Vault

- The placeholder dictionary from `scan_input()` is stored on the `Guard` instance (session-scoped)
- `guard.deanonymize(text)` does string replacement client-side — no server round-trip
- Supports multiple scan calls accumulating into the same session
- `guard.clear_cache()` resets the session

### Scanner Base Contract

```python
class BaseScanner:
    """Base class for all scanners. Each scanner translates itself into a guardline spec."""

    def to_guardline_spec(self) -> dict:
        """Translate scanner config into a guardline spec the server understands."""
        raise NotImplementedError

class BaseNormalizer:
    """Base class for all normalizers. Contract defined now, server support in v2."""

    def to_normalizer_spec(self) -> dict:
        """Translate normalizer config into a server-side normalizer spec."""
        raise NotImplementedError
```

Adding a new scanner = subclass `BaseScanner` + define label enum + implement `to_guardline_spec()`.

### Input vs Output Scanning

Defined from v1, even though output scanning is v2:

```python
# Input scanning (v1)
result = guard.scan_input(text, scanners=[...])

# Output scanning (v2) — needs both prompt and response
result = guard.scan_output(prompt, output, scanners=[...])
```

---

## Server Changes Required

### v1 (Required for SDK launch)

| Change | Description |
|--------|-------------|
| **New `/api/security/scan` endpoint** | Accepts JSON body with `text` + `guardline_specs` inline (instead of fetching from DB) |
| **JSON content-type support** | Current server only accepts `multipart/form-data` |
| **Per-scanner response breakdown** | Response includes which scanners triggered, with per-scanner detection details |
| **API key auth on new endpoint** | Same `X-Api-Key` header pattern |

### v2 (Near Future)

| Change | Description |
|--------|-------------|
| **Normalizer registry** | Server-side registry of normalizers, configurable per-request |
| **Output scanning endpoint** | Accepts both prompt + output text, runs output-specific scanners |
| **New TC heads** | Jailbreak head (coming soon), potentially more |
| **New recognizer types** | Secrets detection (entropy + known patterns), language detection, code detection |

---

## Roadmap

### v1 — Ship First

| Component | Description |
|-----------|-------------|
| SDK core | `Guard`, `BaseScanner`, `BaseNormalizer`, `scan_input()`, result types, HTTP client |
| PIIScanner | All 53 labels via label enums, risk bundles, conditions, actions |
| TopicScanner | Topic classification labels (head hidden) |
| ToxicityScanner | Hate speech / toxicity labels |
| CyberScanner | Cybersecurity threat labels |
| JailbreakScanner | Jailbreak / prompt injection labels (when head is ready) |
| Placeholder cache | Client-side deanonymize support |
| Server: new endpoint | JSON body, inline guardline_specs, per-scanner response |
| Docs site | MkDocs + Material on custom domain |

### v2 — Near Future

| Component | Description |
|-----------|-------------|
| **Normalizer registry** | Unicode normalization, invisible text stripping, homoglyph normalization, encoding detection/decode |
| **New input scanners** | Secrets detection, language restriction, code detection, sentiment scoring, gibberish detection, ban-substrings (inspired by LLM Guard) |
| **Output scanning** | Relevance check (cosine similarity), language consistency, malicious URL detection, NoRefusal detection, output PII leakage |

### Someday-Maybe

| Component | Description |
|-----------|-------------|
| New validators | IBAN checksum, additional format validators |
| File scanning via SDK | PDF/document scanning through the SDK |
| Streaming support | Scan streaming LLM responses chunk-by-chunk |
| Guardrails Hub-style marketplace | Community-contributed scanners |
| Custom regex via SDK | Let developers define custom regex patterns through the SDK |

---

## Industry References

| Resource | URL |
|----------|-----|
| LLM Guard (Protect AI) | https://github.com/protectai/llm-guard |
| LLM Guard docs | https://protectai.github.io/llm-guard/ |
| Guardrails AI | https://github.com/guardrails-ai/guardrails |
| Guardrails on_fail actions | https://www.guardrailsai.com/docs/concepts/validator_on_fail_actions |
| Guardrails Hub | https://hub.guardrailsai.com |
| MkDocs Material | https://squidfunk.github.io/mkdocs-material/ |

---

*Last updated: 2026-02-26 — initial design discussion*
