---
icon: lucide/shield-check
---

<!-- SPDX-FileCopyrightText: 2026 PythonWoods <dev@pythonwoods.dev> -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Advanced Features

Deep reference for the Three-Pass Pipeline, Zenzic Shield, accessibility checks, and
programmatic usage from Python.

---

## Reference integrity (v0.2.0)

`zenzic check references` runs the **Three-Pass Reference Pipeline** — the core engine behind
every reference-quality and security check Zenzic performs.

### Why three passes?

Markdown [reference-style links][syntax] separate *where a link points* (the definition) from
*where it appears* (the usage). A single-pass scanner cannot resolve a reference that appears
before its definition. Zenzic solves this with a deliberate three-pass structure:

| Pass | Name | What happens |
| :---: | :--- | :--- |
| 1 | **Harvest** | Stream the file line-by-line; record all `[id]: url` definitions into a `ReferenceMap`; run the Shield on every URL and line |
| 2 | **Cross-Check** | Re-stream the file; for every `[text][id]` usage, look up `id` in the now-complete `ReferenceMap`; flag missing IDs as **Dangling References** |
| 3 | **Integrity Report** | Compute the integrity score; append **Dead Definitions**, duplicate-ID warnings, and alt-text warnings to the findings list |

Pass 2 only begins when Pass 1 completes without security findings. If the Shield fires during
harvesting, Zenzic exits immediately with code 2 — no reference resolution occurs on files that
contain leaked credentials.

### What the pipeline catches

| Issue | Type | Blocks exit? |
| :--- | :---: | :---: |
| **Dangling Reference** — `[text][id]` where `id` has no definition | error | Yes |
| **Dead Definition** — `[id]: url` defined but never used by any link | warning | No (yes with `--strict`) |
| **Duplicate Definition** — same `id` defined twice; first wins (CommonMark §4.7) | warning | No |
| **Missing alt-text** — `![](url)` or `<img>` with blank/absent alt | warning | No |
| **Secret detected** — credential pattern found in a reference URL or line | security | **Exit 2** |

### Reference Integrity Score

Each file receives a per-file score:

```text
Reference Integrity = (resolved definitions / total definitions) × 100
```

A file where every defined reference is used at least once scores 100. Unused (dead) definitions
pull the score down. When a file has no definitions at all, the score is 100 by convention.

The integrity score is a **per-file diagnostic** — it does not feed into the `zenzic score`
overall quality score. Use it to identify files that accumulate unused reference link
boilerplate.

---

## Zenzic Shield

The Shield runs **inside Pass 1** — every URL extracted from a reference definition is scanned
the moment the harvester encounters it, before any other processing continues. The Shield also
applies a defence-in-depth pass to non-definition lines to catch secrets in plain prose.

### Detected credential patterns

| Pattern name | Regex | What it catches |
| :--- | :--- | :--- |
| `openai-api-key` | `sk-[a-zA-Z0-9]{48}` | OpenAI API keys |
| `github-token` | `gh[pousr]_[a-zA-Z0-9]{36}` | GitHub personal/OAuth tokens |
| `aws-access-key` | `AKIA[0-9A-Z]{16}` | AWS IAM access key IDs |
| `stripe-live-key` | `sk_live_[0-9a-zA-Z]{24}` | Stripe live secret keys |
| `slack-token` | `xox[baprs]-[0-9a-zA-Z]{10,48}` | Slack bot/user/app tokens |
| `google-api-key` | `AIza[0-9A-Za-z\-_]{35}` | Google Cloud / Maps API keys |
| `private-key` | `-----BEGIN [A-Z ]+ PRIVATE KEY-----` | PEM private keys (RSA, EC, etc.) |

### Shield behaviour

- **Every line is scanned** — including lines inside fenced code blocks (labelled or unlabelled).
  A credential committed in a `bash` example is still a committed credential.
- Detection is **non-suppressible** — `--exit-zero`, `exit_zero = true` in `zenzic.toml`, and
  `--strict` have no effect on Shield findings.
- Exit code 2 is reserved **exclusively** for Shield events. It is never used for ordinary check
  failures.
- Files with security findings are **excluded from link validation** — Zenzic does not ping URLs
  that may contain leaked credentials.
- **Code block link isolation** — while the Shield scans inside fenced blocks, the link and
  reference validators do not. Example URLs inside code blocks (e.g. `https://api.example.com`)
  never produce false-positive link errors.

!!! danger "If you receive exit code 2"
    Treat it as a build-blocking security incident. Rotate the exposed credential immediately,
    then remove or replace the offending reference URL. Do not commit the secret into history.

---

## Hybrid scanning logic

Zenzic applies different scanning rules to prose and code blocks because the two contexts have
different risk profiles:

| Content location | Shield (secrets) | Snippet syntax | Link / ref validation |
| :--- | :---: | :---: | :---: |
| Prose and reference definitions | ✓ | — | ✓ |
| Fenced block — supported language (`python`, `yaml`, `json`, `toml`) | ✓ | ✓ | — |
| Fenced block — unsupported language (`bash`, `javascript`, …) | ✓ | — | — |
| Fenced block — unlabelled (` ``` `) | ✓ | — | — |

**Why links are excluded from fenced blocks:** documentation examples routinely contain
illustrative URLs (`https://api.example.com/v1/users`) that do not exist as real endpoints.
Checking them would produce hundreds of false positives with no security value.

**Why secrets are included everywhere:** a credential embedded in a `bash` example is still
a committed secret. It lives in git history, is indexed by code-search tools, and can be
extracted by automated scanners that do not respect Markdown formatting.

**Why syntax checking is limited to known parsers:** validating Bash or JavaScript would
require third-party parsers or subprocesses, violating the No-Subprocess Pillar. Zenzic
validates what it can validate purely in Python.

---

## Alt-text accessibility

`zenzic check references` also flags images that lack meaningful alt text:

- **Markdown inline images** — `![](url)` or `![   ](url)` (blank alt string)
- **HTML `<img>` tags** — `<img src="...">` with no `alt` attribute, or `alt=""` with no
  content

An explicitly empty `alt=""` is treated as intentionally decorative and is **not** flagged.
A completely absent `alt` attribute, or whitespace-only alt text, is flagged as a warning.

Alt-text findings are warnings — they appear in the report but do not affect the exit code
unless `--strict` is active.

---

## Programmatic usage

Import Zenzic's scanner functions directly into your own Python tooling.

### Single-file scan

Use `ReferenceScanner` to run the three-pass pipeline on one file:

```python
from pathlib import Path
from zenzic.core.scanner import ReferenceScanner

scanner = ReferenceScanner(Path("docs/guide.md"))

# Pass 1 — harvest definitions; collect Shield findings
security_findings = []
for lineno, event_type, data in scanner.harvest():
    if event_type == "SECRET":
        security_findings.append(data)
        # In production: raise SystemExit(2) or typer.Exit(2) here

# Pass 2 — resolve reference links (must be after harvest)
cross_check_findings = scanner.cross_check()

# Pass 3 — compute integrity score and consolidate all findings
report = scanner.get_integrity_report(cross_check_findings, security_findings)

print(f"Integrity score: {report.score:.1f}")
for f in report.findings:
    level = "WARN" if f.is_warning else "ERROR"
    print(f"  [{level}] {f.file_path}:{f.line_no} — {f.detail}")
```

### Multi-file scan

Use `scan_docs_references` to scan every `.md` file in a repository and optionally
validate external URLs:

```python
from pathlib import Path
from zenzic.core.scanner import scan_docs_references
from zenzic.models.config import ZenzicConfig

config, _ = ZenzicConfig.load(Path("."))

reports, link_errors = scan_docs_references(
    Path("."),
    config,
    validate_links=True,   # set False to skip HTTP validation
)

for report in reports:
    if report.security_findings:
        raise SystemExit(2)   # your code is responsible for exit-code enforcement
    for finding in report.findings:
        print(finding)

for error in link_errors:
    print(f"[LINK] {error}")
```

`scan_docs_references` deduplicates external URLs across the entire docs tree before
firing HTTP requests — 50 files linking to the same URL result in exactly one HEAD request.

### Hybrid Adaptive Engine — v0.5.0a1

`scan_docs_references` is the single unified entry point for all scan modes.
It selects sequential or parallel execution **automatically** based on the
number of files in the repository:

| Repo size | Engine behaviour | Reason |
| :--- | :--- | :--- |
| < 50 files | Sequential (always) | Process-spawn overhead (~200–400 ms) exceeds the parallelism benefit |
| ≥ 50 files, `workers=1` | Sequential | Explicit serial override |
| ≥ 50 files, `workers=None` or `workers=N` | Parallel (`ProcessPoolExecutor`) | CPU-bound regex work dominates; linear scaling |
| 5 000+ files | Parallel with `workers=cpu_count` | Proven 3–6× speedup on 8-core runners |

The 50-file threshold (`ADAPTIVE_PARALLEL_THRESHOLD`) is the conservative
break-even point where parallelism pays for its own startup cost.

```python
from pathlib import Path
from zenzic.core.scanner import scan_docs_references

# Default: sequential (workers=1, zero overhead)
reports, _ = scan_docs_references(Path("."))

# Explicit parallel: 4 workers, auto-activates only if ≥ 50 files
reports, _ = scan_docs_references(Path("."), workers=4)

# Fully automatic: ProcessPoolExecutor picks worker count from os.cpu_count()
reports, _ = scan_docs_references(Path("."), workers=None)

# With external link validation (works in both sequential and parallel mode)
reports, link_errors = scan_docs_references(Path("."), validate_links=True, workers=None)
```

**Determinism guarantee:** results are always sorted by `file_path` regardless
of execution mode.  The same input always produces the same ordered output.

**Pickling contract for plugin rules (`BaseRule` subclasses):**

Rules are validated for pickle-serializability at engine construction time
(**eager validation**).  A non-serialisable rule raises `PluginContractError`
immediately — before any file is scanned.

- **Rules must be defined at module level.**  A class defined inside a function
  or lambda cannot be pickled and will be rejected at load time.
- **All instance attributes must be pickleable.**  Pre-compiled `re.compile()`
  patterns, strings, and numbers are always safe.  File handles, database
  connections, and lambda closures are not.
- **No mutable global state.**  Workers receive independent copies of the rule
  engine (via pickle).  A global counter mutated inside `check()` will be
  local to each worker process and discarded on completion — results will differ
  from sequential mode silently.  Return all state as `RuleFinding` objects.

See [Writing Plugin Rules](../developers/plugins.md) for the complete contract,
examples, and packaging instructions.

---

## Fenced-code and frontmatter exclusion

The harvester and cross-checker both skip content that should never trigger findings:

- **YAML frontmatter** — the leading `---` block (first line only) is skipped in its entirety,
  including any reference-like syntax it might contain.
- **Fenced code blocks** — lines inside ` ``` ` or `~~~` fences are ignored. URLs in code
  examples never produce false positives.

This exclusion is applied consistently in both Pass 1 and Pass 2.

---

## Nav-Aware Linking (v0.4.0rc4)

Zenzic does not only check whether a linked file exists on disk — it checks whether that
file is **reachable** through the site navigation.  This catches an entire class of
navigation defects that file-existence checks miss entirely.

### Dark pages

A **dark page** is a file that exists on disk and is physically served by the engine at
its URL — but is missing from the site navigation.  The link works.  The page loads.
The user who follows it arrives successfully.  And then they are lost: no breadcrumb,
no menu entry, no way back through the navigation tree.

Dark pages are invisible to users browsing your site.  They are the documentation
equivalent of a room with no door — the room exists, but no one can find it without
already knowing where it is.

Zenzic flags links to dark pages as `UNREACHABLE_LINK`.  This is not a broken link.
It is a **navigation defect**: the link is syntactically correct, the file resolves,
but the destination is unreachable through normal browsing.

### How it works

When a build-engine config (`mkdocs.yml`) is present, Zenzic constructs a **Virtual Site
Map (VSM)** before running link validation.  The VSM maps every `.md` source file to:

- its **canonical URL** (e.g. `docs/guide/installation.md` → `/guide/installation/`)
- its **routing status** — one of `REACHABLE`, `ORPHAN_BUT_EXISTING`, `IGNORED`, or
  `CONFLICT`

A file is `REACHABLE` if it appears in the `nav:` section of `mkdocs.yml`.  A file is
`ORPHAN_BUT_EXISTING` if it lives on disk but has no nav entry — the engine copies it to
`site/` and serves it, but no user can find it through navigation.

### UNREACHABLE_LINK

When a link resolves to a dark page (`ORPHAN_BUT_EXISTING` or `IGNORED`) in the VSM,
Zenzic emits:

```text
  [UNREACHABLE_LINK] index.md:22 — 'guide/secret.md' resolves to '/guide/secret/'
  which exists on disk but is not listed in the site navigation (UNREACHABLE_LINK)
  — add it to nav in mkdocs.yml or remove the link
    │ - [Secret page](guide/secret.md)
```

The Visual Snippet (`│`) shows the exact source line so you can locate and fix the link
without searching through the file.

### Routing collision (CONFLICT)

Two source files that map to the same canonical URL produce a `CONFLICT` in the VSM.
The most common case is the **Double Index**: `index.md` and `README.md` coexisting in
the same directory.  Both produce the same URL (`/dir/`) — the build engine's behaviour
is undefined.  Zenzic detects this before the build runs.

### Engine behaviour

| Adapter | UNREACHABLE_LINK? | Trigger |
| :--- | :---: | :--- |
| **MkDocs** (with `mkdocs.yml` + `nav:`) | Yes | File not listed in `nav:` (`ORPHAN_BUT_EXISTING`) |
| **MkDocs** (no `nav:` declared) | No | All files auto-included by MkDocs |
| **Zensical** | Yes | File or directory starting with `_` (`IGNORED`) |
| **Vanilla** (no engine config) | No | No routing concept |

!!! tip "Fix an UNREACHABLE_LINK"
    Either add the target page to `nav:` in `mkdocs.yml`, or replace the link with one
    pointing to a reachable page.

### Private pages (Zensical)

Files and directories whose name starts with an underscore (`_`) are treated as **private**
by Zenzic when the Zensical engine is active.  Links to these resources are flagged as
`UNREACHABLE_LINK` because Zensical never serves `_`-prefixed paths to the public.

```text
docs/
├── index.md
├── features.md
└── _private/           ← Zensical ignores this directory entirely
    └── notes.md        ← links to this file → UNREACHABLE_LINK
```

```text
[UNREACHABLE_LINK] index.md:8 — '_private/notes.md' resolves to '/_private/notes/'
which exists on disk but is not listed in the site navigation (UNREACHABLE_LINK) —
add it to nav in mkdocs.yml or remove the link
  │ - [Private Notes](_private/notes.md)
```

This rule applies to any path segment starting with `_`:

| Path | Status |
| :--- | :--- |
| `_private/notes.md` | `IGNORED` → `UNREACHABLE_LINK` |
| `_drafts/wip.md` | `IGNORED` → `UNREACHABLE_LINK` |
| `public/page.md` | `REACHABLE` — served normally |

!!! note "MkDocs does not have this rule"
    MkDocs does not treat underscore-prefixed directories as private.  Only Zensical
    enforces the `_`-prefix convention.  When switching engines, audit any `_`-prefixed
    directories in your docs tree.

---

## Multi-language documentation

When your project uses [MkDocs i18n](https://github.com/ultrabug/mkdocs-static-i18n) or
Zensical's locale system, Zenzic adapts automatically:

- **Locale directories suppressed from orphan detection** — files under `docs/it/`, `docs/fr/`,
  etc. are not reported as orphans. The adapter detects locale directories from the engine's
  i18n configuration.
- **Cross-locale link resolution** — the MkDocs and Zensical adapters resolve links that cross
  locale boundaries (e.g. a link from `docs/it/page.md` to `docs/en/page.md`) without false
  positives.
- **Vanilla mode skips orphan check entirely** — when no build-engine config is present, every
  file would appear as an orphan. Zenzic skips the check rather than report noise.

!!! tip "Force Vanilla mode to suppress orphan check"
    ```bash
    zenzic check all --engine vanilla
    ```

[syntax]: https://spec.commonmark.org/0.31.2/#link-reference-definitions
