# Advanced Topic – Validation Strategies

FoodSpec includes batch/group-aware and nested cross-validation to improve robustness.

## Batch/group-aware CV
- Keeps all samples from a batch/instrument together in either train or test to avoid leakage.
- Stratifies by class when possible to maintain balance.
- Configured via protocols (`validation_strategy: batch_aware`) or CLI flags.

## Nested CV
- Outer CV reports performance; inner CV performs feature selection/hyperparameter tuning.
- Useful when minimal panel selection or model tuning is needed without optimistic bias.
- Metrics from outer folds are reported; inner selections are logged.

## Metrics reported
- Balanced accuracy, per-class recall/sensitivity.
- Confusion matrix; ROC/PR curves when applicable.
- Warnings/QC flags when class counts are low or splits are under-powered.

## Where to configure
- `validation.py` provides `batch_aware_cv`, `nested_cv`, and split helpers.
- Protocols can specify `validation_strategy`; CLI flags may override.

See `03-cookbook/cookbook_validation.md` for quick recipes and `07-theory-and-background/chemometrics_and_ml_basics.md` for background.
