"""Package ConfMatrixCalc implements probabilistic Bayesian analysis
of phoneme identification results with forced-choice tests.
The analysis method was presented and validated in (Leijon et al., 2016).

This package estimates the *overall performance* by two measures:

1. Probability of Correct identification (PC), averaged across all presented phonemes.

2. The Mutual Information (MI) between stimulus and response,
    sometimes called "transmitted information".
    This measure was first applied to phoneme identification data by Miller and Nicely (1955).
    It indicates the average amount of information about the stimulus category,
    obtained by the listener for each presented phoneme.

The package also identifies credible confusion patterns as a set of
stimulus-response pairs where listeners' response probabilities are
jointly credibly different between test conditions.

*** Demo Usage
Copy the template scripts `run_sim.py' and `run_cm.py' to a working directory,
and edit data and result file paths if needed.
First run the simulation script `run_sim.py', then `run_cm.py'.
run_cm will use simulated data files created by run_sim.

*** Production Usage
Copy the template script `run_cm.py` to your working directory, rename it, and
edit the copy as suggested in the template, to specify
    - your experimental layout,
    - the top input data directory,
    - a directory where all output result files will be stored.

See module cm_data for information about input files and data format.
Simulated data files generated by `run_sim.py' can be used as examples.

Analysis results are saved as figures and tables in the designated result directory.
After running an analysis, the logging output briefly explains
the analysis results presented in figures and tables.

*** NOTE:

Estimation of entropy and MI is very difficult
if there are too few presentations for each stimulus (e.g., Nemenman et al., 2002).
The validation study (Leijon et al., 2016) indicated that
median estimates for the population are quite reliable,
but the range of variability in the population may be somewhat under-estimated.

*** References

A. Leijon, G. E. Henter, and M. Dahlquist (2016).
Bayesian analysis of phoneme confusion matrices.
IEEE Trans Audio, Speech, and Language Proc 24(3):469–482.
doi: 10.1109/TASLP.2015.2512039.

Arne Leijon. ConfMatrixCalc — Bayesian Analysis of Phoneme Discrimination Data, 2026.
Unpublished report with all math implementation details for the current package version.
Contact author for a copy.

G. A. Miller and P. E. Nicely (1955).
An analysis of perceptual confusions among some English consonants.
J Acoust Soc Amer 27(2):338–352, 1955.
doi: 10.1121/1.1907526.

H. Fletcher and J. Steinberg (1929). Articulation testing methods.
Bell System Technical Journal 8:806–854.
doi: 10.1002/j.1538-7305.1929.tb01246.x.

I. Nemenman, F. Shafee, and W. Bialek (2002).
Entropy and inference, revisited.
In T. Dietterich, S. Becker, and Z. Ghahramani, editors,
NIPS- Advances in Neural Information Processing Systems 14, 471–478.
[download](http://papers.nips.cc/paper/1965-entropy-and-inference-revisited.pdf)

This Python package is a re-implementation and generalisation of a similar MatLab package,
developed by Arne Leijon for ORCA Europe, Widex A/S, Stockholm, Sweden.
The MatLab development was financially supported by *Widex A/S, Denmark*.

*** Version History

* Version 0.8.0, 2026-02-21:

Generalized to handle tests with user-defined subsets of stimuli and forced-choice responses.
Full confusion matrices, as in Miller & Nicely (1955), can still be used,
by defining all stimuli and all allowed responses as one single subset.

The analysis model may be specified as a sampled (mixture) variant.
The point-estimation model described in Leijon et al. (2016) is still recommended and used by default.
Both the point- and sample-estimated variants have been validated by simulations like in Leijon et al. (2016),
but the point-estimation method still seems more reliable, and it is much faster.

* Version 0.7.1: Using the statistical model described in Leijon et al. (2016).


2018-09-15, first functional version 0.6.0
"""

__name__ = 'ConfMatrixCalc'
__version__ = '0.8.0'
__all__ = ['__version__', 'run_cm', 'run_sim']


