Wiki & Ontology - PyCharter
Description: Data Contract Management, ETL Pipelines, and Quality Assurance for Python

Skip to content

# Wiki & Ontology

The PyCharter wiki module adds semantic governance on top of data contracts: field-level annotations, a vocabulary of business concepts, lineage tracking, and collaborative governance workflows. It is the bridge between your technical schemas and the business meaning behind the data.

## What the wiki provides

Sub-module | Purpose
--- | ---
Ontology | Vocabulary of concepts, concept tiers, field annotations, relationship types, lineage
RDF / JSON-LD / SHACL | Export ontologies as RDF; validate shapes with SHACL
Knowledge graph | Graph-based discovery and lineage traversal across contracts
Collaboration | Proposals, reviews, conflict resolution, and permissions for governed changes
Versioning | Diff, history, and merge of ontology artifacts over time
Ingestion | Import external ontologies or vocabulary files
REST API | HTTP routes for all wiki operations

## Core concepts

### Concept tiers

Governance is structured around three tiers:

Tier | Enum value | Description
--- | --- | ---
Core | `ConceptTier.CORE` | Governed by the central data team; changes require approval
Domain | `ConceptTier.DOMAIN` | Governed by individual domain teams
Free | `ConceptTier.FREE` | No governance; used for experimental or team-local concepts

### Node kinds

Concepts in the knowledge graph are classified by node kind:

Kind | Example
--- | ---
`entity` | Customer, Product, Order
`process` | Checkout flow, Onboarding
`event` | Payment received, Item shipped
`attribute` | Price, Weight, Status
`rule` | Discount eligibility, Age restriction
`system` | Payment gateway, CRM
`concept` | Revenue, Churn
`metric` | Monthly active users, NPS

### Relationship types

Type | Meaning
--- | ---
`references` | Foreign key or pointer to another field
`derived_from` | Computed from another field
`same_as` | Two fields represent the same concept
`related_to` | General association
`is_a` | Taxonomic parent (X is a kind of Y)
`has` | Composition (X contains Y)
`depends_on` | X requires Y
`precedes` | X comes before Y (temporal)
`causes` | X triggers Y
`implements` | X is a concrete form of Y
`governs` | X constrains Y
`describes` | X is a property of Y
`equivalent_to` | Semantic equivalence

## Annotating fields in a contract

Add an `ontology` block inside any field definition in your contract YAML:

    # contract.yaml
    schema:
      title: orders
      version: "1.0.0"
      type: object
      properties:
        order_id:
          type: string
          ontology:
            concept: "order_identifier"
            broader: "identifier"
            definition: "Unique identifier for a customer order"
            tags: ["core", "immutable"]
            owner: "order-team"
        customer_id:
          type: string
          ontology:
            concept: "customer_identifier"
            relationships:
              - target: "customers.id"
                type: references
        total_amount:
          type: number
          ontology:
            concept: "monetary_value"
            lineage:
              derived_from: "order_items.unit_price"
              source_system: "order_service"

## Parsing and validating ontologies

    from pycharter.wiki import parse_ontology, parse_ontology_file, validate_ontology

    # Parse from a dict (e.g. loaded from YAML)
    artifact = parse_ontology({
        "field": "order_id",
        "concept": "order_identifier",
        "broader": "identifier",
        "tags": ["core", "immutable"],
        "owner": "order-team",
    })

    # Parse from a YAML file
    artifact = parse_ontology_file("ontologies/orders.yaml")

    # Validate (returns list of error strings, empty if valid)
    errors = validate_ontology(artifact)
    if errors:
        for err in errors:
            print(f"  ✗ {err}")
    else:
        print("Ontology is valid")

## Key data models

### `OntologyArtifact`

Top-level container for field annotations:

    from pycharter.wiki import OntologyArtifact, FieldOntology, Lineage, Relationship, RelationshipType

    artifact = OntologyArtifact(
        field="orders.total_amount",
        field_ontologies={
            "total_amount": FieldOntology(
                concept="monetary_value",
                broader="financial_metric",
                definition="Total order value in local currency",
                tags=["financial", "auditable"],
                owner="finance-team",
                lineage=Lineage(
                    derived_from="order_items.unit_price",
                    source_system="order_service",
                ),
                relationships=[
                    Relationship(target="invoices.amount", type=RelationshipType.SAME_AS),
                ],
            )
        },
    )

### `FieldOntology`

Per-field semantic metadata:

Attribute | Type | Description
--- | --- | ---
`concept` | `str` | Primary concept name from the vocabulary
`broader` | `str \| None` | Parent concept
`definition` | `str \| None` | Human-readable definition
`tags` | `list[str]` | Classification tags
`owner` | `str \| None` | Team or person responsible
`description` | `str \| None` | Additional prose
`deprecated` | `bool` | Whether this annotation is deprecated
`lineage` | `Lineage \| None` | Origin information
`relationships` | `list[Relationship]` | Relationships to other fields/concepts

### `Lineage`

Attribute | Description
--- | ---
`derived_from` | Field path this value is computed from
`source_system` | System that produces this field

### `Relationship`

Attribute | Description
--- | ---
`target` | Target field path (e.g. `"customers.id"`)
`type` | `RelationshipType` enum value

## Seed vocabulary

PyCharter ships a built-in vocabulary of ~250 business concepts in `data/seed/ontologies.yaml`. These cover common domains like finance, e-commerce, identity, and logistics. Load or extend them via the ingestion module:

    from pycharter.wiki.ingestion.importer import OntologyImporter

    importer = OntologyImporter()
    concepts = importer.load_from_file("data/seed/ontologies.yaml")
    print(f"Loaded {len(concepts)} concepts")

## RDF / JSON-LD export

Export ontology artifacts as RDF for interoperability with SPARQL endpoints, ontology editors, or SHACL validators:

    from pycharter.wiki.rdf.converter import RDFConverter

    converter = RDFConverter()
    json_ld = converter.to_json_ld(artifact)
    print(json_ld)  # JSON-LD document

## Collaborative governance

The wiki includes a collaboration layer for teams that need structured change management on ontology definitions:

- Proposals — submit a proposed change to a concept definition
- Reviews — accept, reject, or request changes on a proposal
- Conflict resolution — detect and resolve competing proposals
- Permissions — control who can propose, review, and approve changes per concept tier

These are primarily used through the REST API (see the wiki API routes at `/api/v1/wiki/`).

## Install

The wiki module is included in the core package. RDF features require an extra:

    pip install pycharter[rdf]

## See also

- Domain Models and Lifecycle — lifecycle contracts and FSM binding
- Data Contracts & Validation tutorial
- API Reference — wiki