# Example notebook

A full end-to-end notebook is available per environment:

- **cog-sail:** `config/environments/cog-sail/test_cogsail_cognite_data_quality.ipynb`

Run it from the repo root or from the environment directory so that paths to `settings.yaml`, `views/`, and `shacl_rules/` resolve correctly.

The notebook covers:

1. **Setup** – Installation from PyPI, credentials (TOML or manual).
2. **Verify** – Import and embedded function files (`_get_embedded_function_files()`).
3. **Client** – Load client from TOML and set timeout; prepare `function_secrets` for deployment.
4. **Optional cleanup** – Delete container, functions, triggers/workflows (for a clean redeploy).
5. **Run validation** – `run_validation()` with TTL rules, `DataModelConfig`, `instance_space`, `RecordsConfig`, `limit`, `print_output`, `post_to_records`.
6. **Records API** – List streams; filter records (e.g. failed validations, Violation/Warning in last hour).
7. **Deploy infrastructure** – `deploy_validation_infrastructure(settings_path, views_dir, function_secrets)`.
8. **Deploy validation pipeline** – `deploy_validation_pipeline(client, settings_path, view_external_id, wait=True)` and display orchestration ID, partitions, sync trigger, monitor schedule.

## Minimal script example

```python
from cognite_data_quality import load_cognite_client_from_toml, run_validation, DataModelConfig, RecordsConfig

client = load_cognite_client_from_toml("cog-sail.toml")

result = run_validation(
    client=client,
    rules_path="config/environments/cog-sail/shacl_rules/smallboat_shacl_rules.ttl",
    rules_format="ttl",
    datamodel=DataModelConfig(space="sailboat", external_id="sailboat", version="v1"),
    instance_space="sailboat",
    records_config=RecordsConfig(
        stream_id="dq_validation_stream",
        rule_set_id="SailboatSHACLv1",
        rule_set_version="1.0",
    ),
    limit=10,
    post_to_records=False,
)
print(result.conforms, result.instance_count, len(result.violations))
```

For YAML-based config (datamodel and instance space from the file):

```python
result = run_validation(
    client=client,
    rules_path="config/environments/cog-sail/views/smallboat.yaml",
    rules_format="yaml",
)
```
