---
title: "Crucible Shim Standard"
description: "Helper module contract for exposing embedded Crucible assets in Fulmen libraries"
author: "Schema Cartographer"
date: "2025-10-09"
last_updated: "2025-10-21"
status: "stable"
tags: ["standards", "library", "crucible", "assets", "2025.10.2"]
---

# Crucible Shim Standard

## Purpose

Provide a stable interface for consuming projects to access Crucible documentation, schemas, templates, and
metadata embedded within helper libraries. Eliminates the need for consumers to know the on-disk layout or build
process used inside Crucible while keeping responsibilities clearly separated between the shim and other helper
modules such as Docscribe.

## Capabilities

1. Expose version metadata (`CrucibleVersion`, `CrucibleCommit`, `CrucibleSyncDate`).
2. Enumerate available asset categories (`docs`, `schemas`, `config`, `templates`), even when a category currently has no assets (e.g., templates remain empty until populated).
3. Provide typed accessors for common asset types while returning native-language representations for structured data and raw bytes/strings for document content.
4. Offer streaming APIs for large assets to avoid loading entire file into memory when the language runtime supports it.
5. Deliver change detection utilities (checksum, last modified) to help consumers detect updates.

## API Outline

| Function / Property             | Description                                                                                           |
| ------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `CrucibleVersion()`             | Returns structured metadata (`version`, `commit`, `syncedAt`) sourced from Crucible sync manifest.    |
| `ListAssets(category, prefix?)` | Returns metadata array `AssetSummary` (see below) in stable, lexicographic order.                     |
| `ReadAsset(id)`                 | Returns asset contents as bytes/string.                                                               |
| `OpenAsset(id)`                 | Stream interface (reader/async iterator) for languages with native streaming abstractions (optional). |
| `FindSchema(id)`                | Convenience wrapper returning parsed schema (native map/object) plus metadata.                        |
| `FindConfig(id)`                | Returns parsed configuration (native map/object) plus metadata.                                       |
| `GetDoc(id)`                    | Returns raw markdown/bytes for documentation assets; Docscribe handles any parsing/frontmatter work.  |

Helper libraries MUST ensure asset IDs remain stable between releases (`docs/standards/...` path-based).

## Version Metadata

`CrucibleVersion()` MUST return a structured object with version and sync information sourced from metadata generated by `goneat ssot sync`.

**Expected Fields** (subject to goneat implementation):

| Field        | Type    | Description                                                      |
| ------------ | ------- | ---------------------------------------------------------------- |
| `version`    | string  | CalVer string from Crucible's `VERSION` file (e.g., "2025.10.2") |
| `commit`     | string  | Git commit hash at sync time, or `"unknown"` if unavailable      |
| `syncedAt`   | string  | ISO-8601 timestamp of sync operation                             |
| `dirty`      | boolean | Whether source had uncommitted changes (development workflow)    |
| `syncMethod` | string  | Sync method: `"local-path"`, `"git-ref"`, or `"git-tag"`         |

**Metadata Source**: `.crucible/metadata.yaml` (auto-generated by `goneat ssot sync`)

**Schema Definition**: Owned and versioned by goneat. See goneat documentation for canonical schema and field semantics.

Helpers MAY include additional fields from the metadata file. When metadata is missing or fields are unavailable, return `null` for individual fields rather than omitting the object.

**Example payload**:

```json
{
  "version": "2025.10.2",
  "commit": "a1b2c3d4e5f6",
  "dirty": false,
  "syncedAt": "2025-10-22T18:42:11Z",
  "syncMethod": "git-tag"
}
```

**Note**: Metadata structure is defined by goneat and works universally for any SSOT repository. Helper libraries consume the generated metadata file and expose it through idiomatic language APIs.

## Asset Identifier Rules

Crucible assets are indexed by stable identifiers that are independent of language wrapper layout. Helpers MUST treat IDs as case-sensitive, forward-slash-delimited paths without language prefixes.

| Category      | ID Pattern                                                                               | Example                                       |
| ------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------- |
| `docs`        | Relative path beneath `docs/` including `.md` extension                                  | `standards/agentic-attribution.md`            |
| `schemas`     | `{category}/{namespace?}/{version}/{name}` with version directory and no file extension  | `observability/logging/v1.0.0/logger-config`  |
| `config`      | `{category}/{version}/{filename-without-extension}`                                      | `terminal/v1.0.0/terminal-overrides-defaults` |
| `templates`\* | `{category}/{version}/{name}` (pending population; document expectations once available) | `workflows/v2025.10.0/bootstrap`              |

\*Templates remain provisional. Until assets ship, implementations MAY expose the category but SHOULD document it as empty.

Additional guidance:

- IDs never include the language folder name (`docs/crucible-py/`, `docs/crucible-go/`, etc.).
- Helper libraries MUST preserve the original casing of asset IDs and MUST normalise any discovered filesystem paths to use forward slashes (`/`).
- `ListAssets` SHOULD return results in lexicographic order (case-sensitive) to guarantee deterministic suggestions, and the order MUST remain stable when prefix filters are applied.
- Helper libraries MAY support optional `prefix` filters for enumeration but MUST return full canonical IDs.
- When new asset categories are added, the manifest in `config/library/v1.0.0/module-manifest.yaml` must be updated in tandem.

## Asset Metadata Contract

`ListAssets` and the convenience helpers SHOULD return an `AssetSummary` object with the following fields:

| Field      | Type                | Notes                                                                                                |
| ---------- | ------------------- | ---------------------------------------------------------------------------------------------------- |
| `id`       | string              | Canonical asset identifier (see table above).                                                        |
| `path`     | string              | Internal path used inside the packaged assets (optional but recommended for debugging).              |
| `category` | string              | One of `docs`, `schemas`, `config`, `templates`.                                                     |
| `format`   | string &#124; null  | File format/extension actually served (e.g., `json`, `yaml`, `md`).                                  |
| `checksum` | string &#124; null  | SHA-256 hex digest of the asset contents. Helpers MAY compute lazily; document behaviour if omitted. |
| `size`     | integer &#124; null | Byte size of the underlying asset.                                                                   |
| `modified` | string &#124; null  | ISO-8601 timestamp representing the sync time if available.                                          |

All optional fields SHOULD be populated when the information is readily available—especially when values are already present in the sync manifest (`config/sync/sync-keys.yaml`) or other metadata files. When a field cannot be determined cheaply, helpers MAY return `null`/`None` and document the limitation.

## Schema Extension Resolution

Crucible occasionally publishes both JSON (`.schema.json`) and YAML (`.schema.yaml`) variants of the same schema. Helper libraries MUST implement the following lookup behaviour:

1. Attempt to load the JSON artefact (`.schema.json`).
2. If JSON is absent, fall back to YAML (`.schema.yaml`) and parse into native structures.
3. Record which extension was selected in the returned metadata (e.g., `AssetSummary.format = "json"`).
4. If neither variant exists, raise `AssetNotFound` with suggestions.

Helpers MUST expose parsed schema content as native maps/objects consistent with the host language.

## Configuration Assets

- `FindConfig(id)` returns a tuple/object containing the parsed configuration (native map/object) and accompanying metadata.
- IDs include the terminal filename without extension to disambiguate multiple configs within the same version directory.
- Config helpers SHOULD surface metadata (`checksum`, `size`, `modified`) alongside the parsed payload so consumers can detect drift.
- When a configuration asset is formatted as JSON, helpers MUST parse it into equivalent native data types; consumers should not re-parse raw strings.
- Libraries MAY expose additional ergonomic variants (e.g., `find_config(category, version, name)`), but all variants MUST resolve to the canonical ID specification above. Refer to the [Three-Layer Config standard](../enterprise-three-layer-config.md) for integration patterns.

## Documentation Access & Docscribe Integration

- `GetDoc(id)` returns a pair/struct containing the raw markdown content (string/bytes) and the associated `AssetSummary`.
- Docscribe is responsible for frontmatter extraction, header detection, and other document processing concerns.
- Helper libraries MAY provide convenience wrappers (`get_doc_with_metadata`) that internally call Docscribe, but such wrappers live outside the Crucible Shim contract.
- Streaming interfaces MUST yield raw markdown bytes; libraries can adapt to language idioms (Python context manager, Go `io.ReadCloser`, Node.js `Readable`).

### Accessing General Documentation

The crucible shim provides access to **all Crucible documentation**, not just schemas and configs. This enables microtools, commit hooks, CI validators, and other tooling to reference standards and documentation programmatically without requiring a local crucible checkout.

**Why this matters:**

- **Sustainability**: Tools don't need local crucible checkout
- **Consistency**: Same access pattern across all ecosystem tools
- **Versioning**: Documentation is version-tracked via helper library dependency
- **Offline**: No network dependency for standards/documentation access

#### Common Documentation Paths

Helper libraries expose Crucible documentation using relative paths beneath `docs/`:

| Documentation Category | Example Paths                                                                                                                                            |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Standards**          | `standards/agentic-attribution.md`<br>`standards/makefile-standard.md`<br>`standards/observability/logging.md`<br>`standards/coding/go.md`               |
| **Architecture**       | `architecture/fulmen-forge-microtool-standard.md`<br>`architecture/fulmen-template-cdrl-standard.md`<br>`architecture/fulmen-helper-library-standard.md` |
| **Guides**             | `guides/consuming-crucible-assets.md`<br>`guides/bootstrap-goneat.md`<br>`guides/fulmen-library-bootstrap-guide.md`                                      |
| **SOPs**               | `sop/repository-operations-sop.md`<br>`sop/repository-structure.md`                                                                                      |

#### Use Cases

**1. Commit Validation Hooks**
Access agentic-attribution standard for commit message format validation:

```go
import "github.com/fulmenhq/gofulmen/crucible"

// Load agentic attribution standard
standard, err := crucible.GetDoc("standards/agentic-attribution.md")
if err != nil {
    return fmt.Errorf("failed to load attribution standard: %w", err)
}

// Validate commit message format (implementation detail omitted)
if !hasRequiredTrailers(commitMsg, standard) {
    return fmt.Errorf("commit missing required attribution trailers")
}
```

**2. Documentation Generators**
Access coding standards to generate boilerplate or validate generated code:

```python
from pyfulmen import crucible

# Get Go coding standards for doc generator
go_standards = crucible.get_doc("standards/coding/go.md")

# Extract naming conventions section
conventions = extract_section(go_standards, "Naming Conventions")
```

**3. CI Compliance Checkers**
Validate repository structure against standards:

```typescript
import { crucible } from "@fulmenhq/tsfulmen";

// Load repository structure standard
const structureStd = await crucible.getDoc("sop/repository-structure.md");

// Validate repository compliance
const violations = validateStructure(repoPath, structureStd);
if (violations.length > 0) {
  process.exit(1);
}
```

**4. Template Refit Tools**
Access CDRL workflow documentation for automated template customization:

```go
import "github.com/fulmenhq/gofulmen/crucible"

// Load CDRL standard
cdrlStandard, err := crucible.GetDoc("architecture/fulmen-template-cdrl-standard.md")
if err != nil {
    return err
}

// Parse refit workflow steps
workflow := parseCDRLWorkflow(cdrlStandard)
```

#### Discovering Available Documentation

Use `ListAssets("docs", prefix)` to discover available documentation:

**Go:**

```go
import "github.com/fulmenhq/gofulmen/crucible"

// List all standards
standards, err := crucible.ListAssets("docs", "standards/")
for _, asset := range standards {
    fmt.Printf("Standard: %s\n", asset.ID)
}

// List architecture docs
archDocs, err := crucible.ListAssets("docs", "architecture/")

// List guides
guides, err := crucible.ListAssets("docs", "guides/")
```

**Python:**

```python
from pyfulmen import crucible

# List all standards
standards = crucible.list_assets("docs", prefix="standards/")
for asset in standards:
    print(f"Standard: {asset.id}")

# List architecture docs
arch_docs = crucible.list_assets("docs", prefix="architecture/")
```

**TypeScript:**

```typescript
import { crucible } from "@fulmenhq/tsfulmen";

// List all standards
const standards = await crucible.listAssets("docs", "standards/");
for (const asset of standards) {
  console.log(`Standard: ${asset.id}`);
}

// List architecture docs
const archDocs = await crucible.listAssets("docs", "architecture/");
```

#### Cross-Language API Consistency

All helper libraries provide consistent APIs for documentation access:

| Language       | Get Document                                     | List Documents                                |
| -------------- | ------------------------------------------------ | --------------------------------------------- |
| **Go**         | `crucible.GetDoc(path string) (string, error)`   | `crucible.ListAssets("docs", prefix string)`  |
| **Python**     | `crucible.get_doc(path: str) -> str`             | `crucible.list_assets("docs", prefix: str)`   |
| **TypeScript** | `crucible.getDoc(path: string): Promise<string>` | `crucible.listAssets("docs", prefix: string)` |

#### Complete Example: Commit Hook Validator

**Go Implementation:**

```go
package main

import (
    "fmt"
    "os"
    "strings"

    "github.com/fulmenhq/gofulmen/crucible"
    "github.com/fulmenhq/gofulmen/foundry"
)

func main() {
    // Load agentic attribution standard from embedded Crucible docs
    attrStandard, err := crucible.GetDoc("standards/agentic-attribution.md")
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to load attribution standard: %v\n", err)
        os.Exit(foundry.ExitFailure)
    }

    // Read commit message from git
    commitMsgBytes, err := os.ReadFile(".git/COMMIT_EDITMSG")
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to read commit message: %v\n", err)
        os.Exit(foundry.ExitFailure)
    }
    commitMsg := string(commitMsgBytes)

    // Validate required trailers exist
    requiredTrailers := []string{"Committer-of-Record:", "Co-Authored-By:"}
    for _, trailer := range requiredTrailers {
        if !strings.Contains(commitMsg, trailer) {
            fmt.Fprintf(os.Stderr, "Commit missing required trailer: %s\n", trailer)
            fmt.Fprintf(os.Stderr, "See attribution standard for format: standards/agentic-attribution.md\n")
            os.Exit(foundry.ExitFailure)
        }
    }

    fmt.Println("✅ Commit message follows attribution standard")
    os.Exit(foundry.ExitSuccess)
}
```

**Python Implementation:**

```python
#!/usr/bin/env python3
import sys
from pathlib import Path

from pyfulmen import crucible, foundry

def main():
    # Load agentic attribution standard
    try:
        attr_standard = crucible.get_doc("standards/agentic-attribution.md")
    except Exception as e:
        print(f"Failed to load attribution standard: {e}", file=sys.stderr)
        sys.exit(foundry.EXIT_FAILURE)

    # Read commit message
    commit_msg_path = Path(".git/COMMIT_EDITMSG")
    if not commit_msg_path.exists():
        print("Failed to read commit message", file=sys.stderr)
        sys.exit(foundry.EXIT_FAILURE)

    commit_msg = commit_msg_path.read_text()

    # Validate required trailers
    required_trailers = ["Committer-of-Record:", "Co-Authored-By:"]
    for trailer in required_trailers:
        if trailer not in commit_msg:
            print(f"Commit missing required trailer: {trailer}", file=sys.stderr)
            print("See attribution standard: standards/agentic-attribution.md", file=sys.stderr)
            sys.exit(foundry.EXIT_FAILURE)

    print("✅ Commit message follows attribution standard")
    sys.exit(foundry.EXIT_SUCCESS)

if __name__ == "__main__":
    main()
```

#### Sustainability Guidance

**Always access Crucible assets through the helper library shim** rather than filesystem reads or network fetches. This ensures:

- ✅ **No crucible checkout required**: Tools work anywhere the helper library is installed
- ✅ **Version-tracked**: Documentation version matches helper library dependency
- ✅ **Offline access**: No network dependency or HTTP requests required
- ✅ **Consistent access pattern**: Same API across all languages and all ecosystem tools
- ✅ **Type safety**: Helper libraries provide typed APIs in statically-typed languages

**Anti-patterns to avoid:**

- ❌ Reading from local filesystem: `os.ReadFile("~/crucible/docs/standards/agentic-attribution.md")`
- ❌ HTTP requests to github: `http.Get("https://raw.githubusercontent.com/.../agentic-attribution.md")`
- ❌ Git cloning: Requiring `git clone` of crucible repository
- ❌ Vendoring docs: Copying documentation files into your repository

## Streaming Expectations

- `OpenAsset(id)` is RECOMMENDED (but not required) when the host language provides standard streaming abstractions.
- Python: context manager returning a binary file-like object.
- Go: `io.ReadCloser`.
- TypeScript/JavaScript: Node.js `Readable` stream (or async iterator).
- For languages that lack a native streaming abstraction, document that `OpenAsset` is unsupported and ensure `ReadAsset` remains available.

## Examples

```python
from pyfulmen import crucible, docscribe

# Enumerate logging docs
docs = crucible.list_assets("docs", prefix="standards/observability/")

# Fetch schema (parsed dict)
schema, meta = crucible.find_schema("observability/logging/v1.0.0/logger-config")
assert schema["properties"]["level"]["enum"] == ["trace", "debug", "info", "warn", "error"]

# Fetch config defaults (parsed dict)
defaults, defaults_meta = crucible.find_config("terminal/v1.0.0/terminal-overrides-defaults")

# Retrieve raw markdown and process via Docscribe
raw_doc, doc_meta = crucible.get_doc("standards/observability/logging.md")
content, frontmatter = docscribe.parse_frontmatter(raw_doc)
```

```go
doc, meta, err := crucible.GetDoc("standards/agentic-attribution.md")
if err != nil {
    return err
}
stats := docscribe.InspectDocument(string(doc))
```

```ts
const [config, meta] = await crucible.findConfig(
  "terminal/v1.0.0/terminal-overrides-defaults",
);
const [docBuffer, docMeta] = await crucible.getDoc(
  "standards/agentic-attribution.md",
);
const { content, metadata } = docscribe.parseFrontmatter(
  docBuffer.toString("utf8"),
);
```

## Implementation Notes

- **Go**: Embed assets via `//go:embed`. Provide packages `crucible/docs`, `crucible/schemas`. May precompute an index
  generated during `bun run sync:to-lang` for O(1) lookup, but on-demand discovery is acceptable for v0.1.x.
- **TypeScript**: Use generated `schemas.ts`/`docs.ts` modules under `lang/typescript/`. Export helper functions
  that resolve asset IDs to strings or Buffers.
- **Python**: Utilize `importlib.resources.files("pyfulmen.crucible")` to access embedded assets packaged with
  the distribution. Provide wrappers returning native types (`dict`, `list`, `str`, `bytes`).
- **Docscribe**: Helpers SHOULD re-export convenience utilities that delegate to the Docscribe module rather than
  re-implementing frontmatter parsing inside the shim.

### Templates Category

Templates are not yet populated in Crucible. Helper libraries SHOULD expose the category for future compatibility while returning an empty list from `ListAssets("templates")`. Once templates ship, update this standard with canonical ID patterns and payload expectations.

## Error Handling

- Throw/return `ErrAssetNotFound` (Go) or equivalent exceptions (`AssetNotFoundError` in Python/TypeScript) when
  the requested ID is missing.
- Throw/return `InvalidCategoryError` (or language equivalent) when `ListAssets` or find helpers receive an
  unsupported category. Error messages MUST enumerate the valid categories.
- Helper libraries SHOULD surface suggestion lists using the forthcoming Foundry similarity helpers once they
  land; until then, implement lightweight string similarity (`Did you mean …`) so consumers receive actionable
  feedback.
- Invalid IDs (incorrect separators, leading/trailing slashes, etc.) SHOULD be rejected with the same
  `AssetNotFound` family so consumers receive consistent guidance.
- All errors MUST preserve the original, case-sensitive ID that was requested.

## Testing Requirements

- Unit tests verifying asset enumeration matches manifest generated during sync.
- Tests ensuring version metadata reflects `VERSION` file and sync metadata.
- Integration tests confirming schema/doc helpers return expected content for sample IDs.

## Related Documents

- `scripts/sync-to-lang.ts`
- `docs/architecture/pseudo-monorepo.md`
- `docs/architecture/sync-model.md`
- `docs/standards/library/modules/docscribe.md`
