Metadata-Version: 2.4
Name: independent_validation
Version: 0.1.3
Summary: IV is a Python package designed to assess the accuracy of machine learning classifiers by generating a full probability distribution of their performance. This method goes beyond traditional cross-validation by using an iterative process and Markov Chain Monte Carlo (MCMC) to estimate uncertainty. By only using samples for training after they have been tested alpha inflation is prevented.
Project-URL: Homepage, https://github.com/jonasthedevonoertzen/IV
Author-email: Thede von Oertzen <thede.vonoertzen@thomasbayes.de>, Hannes Diemerling <hannes.diemerling@thomasbayes.de>, Timo von Oertzen <timo.vonoertzen@thomasbayes.de>
License-Expression: MIT
License-File: LICENSE
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Requires-Dist: matplotlib
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: pytest
Requires-Dist: scikit-learn
Requires-Dist: scipy
Requires-Dist: seaborn
Description-Content-Type: text/markdown

# IV: Independent Validation

IV is a Python package designed to assess the accuracy of machine learning classifiers by generating a full probability distribution of their performance. This method goes beyond traditional cross-validation by using an iterative process and Markov Chain Monte Carlo (MCMC) to estimate uncertainty.
By only using samples for training after they have been tested alpha inflation is prevented.

## Installation

```bash
pip install independent-validation
```


## Usage

An example of running the independent validation process with a classifier:

```python
from sklearn.datasets import load_wine
import independent_validation as iv
from sklearn.svm import SVC

wine = load_wine()
X, y = wine.data, wine.target

iv_svm = iv.IV(X, y, SVC(gamma='scale'))  # initiating
iv_svm.run_iv()  # classify samples
iv_svm.compute_posterior()  # use mcmc to compute posterior
bacc_svm_dist = iv_svm.get_bacc_dist()  # get desired output
print("Mode (MAP) value:", bacc_svm_dist.map())
```

## License

MIT License
