# Validation Guide

Orxaq includes a validation pipeline that evaluates causal discovery algorithms against ground truth DAGs.

## Generate Synthetic Data

Create a 50,000-row credit portfolio with known causal structure:

```bash
orxaq validate --generate --rows 50000 --seed 42 --output data/validation/credit_portfolio.csv
```

The generator uses a linear structural causal model (SCM) with 20 variables and 25 causal edges matching the credit ontology. Edge weights define the causal mechanisms, and Gaussian noise is added at each node.

## Run Validation Pipeline

Compare discovery results against ground truth:

```bash
orxaq validate --data data/validation/credit_portfolio.csv \
               --ground-truth data/validation/ground_truth_dag.json \
               --algorithm pc --output validation_report.json
```

## Metrics

| Metric | Description |
|--------|-------------|
| SHD | Structural Hamming Distance: edge additions + deletions + reversals |
| Precision | Fraction of predicted edges that are correct |
| Recall | Fraction of true edges that were discovered |
| F1 | Harmonic mean of precision and recall |
| Orientation Accuracy | Of skeleton matches, fraction correctly oriented |
| FDR | False discovery rate: fraction of predictions that are wrong |

## CCAR Scenarios

The validation dataset ships with three CCAR-compliant scenarios in `data/validation/scenarios/`:

- **Baseline**: Normal growth trajectory (GDP +2.3%, unemployment 4.2%)
- **Adverse**: Moderate recession (GDP -1.5%, unemployment 7.5%, HPI -10%)
- **Severely Adverse**: Deep recession (GDP -4.5%, unemployment 10%, HPI -25%)

## Try It: Full Validation

```python
from orxaq.fabric.validation import ValidationDataGenerator, ValidationPipeline

gen = ValidationDataGenerator()
data = gen.generate(n=5000, seed=42)

pipe = ValidationPipeline(algorithm="pc")
metrics = pipe.run(
    data=data,
    true_edges=gen.true_edges,
    variables=["gdp_growth", "unemployment", "hpi"],
)

print(f"SHD: {metrics.shd}")
print(f"F1:  {metrics.f1:.3f}")
```
