# gri_utils.stats

Statistical utilities for confidence intervals and scale factor analysis.

## Overview

This module provides confidence interval calculations commonly used in geospatial accuracy assessment. Some functions have scipy equivalents (documented below); others are implemented here because scipy lacks them.

## Functions

### Mean Confidence Interval

**Scipy equivalent:** `scipy.stats.t.interval()` + `scipy.stats.sem()`

Computes confidence interval for the mean of a sample using the t-distribution and the standard error of the mean (sem).

```python
from scipy import stats
import numpy as np

data = np.array([25, 28, 22, 27, 24, 26, 23, 29, 25, 28])
ci = stats.t.interval(0.95, df=len(data)-1, loc=np.mean(data), scale=stats.sem(data))
# ci = (24.22, 27.18)
```

### Quantile Confidence Interval

**Scipy equivalent (1D only):** `scipy.stats.quantile_test().confidence_interval()` (scipy >= 1.10)

Computes confidence interval for any quantile (e.g., median) of a distribution.

```python
from scipy import stats

data = np.random.randn(100)
result = stats.quantile_test(data, q=0.5, p=0.5)  # Test for median
ci = result.confidence_interval(confidence_level=0.95)
```

**This module provides:** `quantile_ci()` using normal approximation to binomial, with support for multi-dimensional arrays.

```python
from gri_utils.stats import quantile_ci

# 1D array - returns scalar floats
data = np.random.randn(100)
lower, upper = quantile_ci(data, quantile=0.5, confidence=0.95)

# 2D array - compute CI for each column (axis=0)
data_2d = np.random.randn(100, 3)  # 100 samples, 3 variables
lower, upper = quantile_ci(data_2d, axis=0)  # Returns arrays of shape (3,)

# 2D array - compute CI for each row (axis=1)
data_2d = np.random.randn(3, 100)  # 3 variables, 100 samples each
lower, upper = quantile_ci(data_2d, axis=1)  # Returns arrays of shape (3,)

# 3D array - operates along specified axis
data_3d = np.random.randn(50, 4, 3)
lower, upper = quantile_ci(data_3d, axis=0)  # Returns arrays of shape (4, 3)

# Different quantiles
lower, upper = quantile_ci(data, quantile=0.75)  # 75th percentile CI
```

**Parameters:**

- `data`: Array of sample data (1D or multi-dimensional)
- `quantile`: Quantile of interest in (0, 1). Default 0.5 for median.
- `confidence`: Confidence level in (0, 1). Default 0.95.
- `axis`: Axis along which to compute CI. Default is 0 (except for row vectors).
- `min_samples`: Minimum samples required. Default 10.

### Poisson Confidence Interval (Garwood Method)

Note: `scipy.stats.poisson.interval()` is NOT a confidence interval—it returns probability intervals (quantiles of the distribution).

Computes exact confidence interval for Poisson rate parameter using the Garwood method (1936), which uses the chi-squared distribution.

```python
from gri_utils.stats import poisson_ci

# Single count
lower, upper = poisson_ci(10)  # 95% CI for rate given 10 observed events
# lower ~= 4.80, upper ~= 18.39

# Multiple counts (vectorized)
lower, upper = poisson_ci([5, 10, 15, 20])

# Custom confidence level
lower, upper = poisson_ci(10, confidence=0.99)
```

**Mathematical basis:**

- Lower bound: chi2(alpha/2, 2k) / 2
- Upper bound: chi2(1-alpha/2, 2(k+1)) / 2

Where k is the observed count and alpha = 1 - confidence.

### Average Scale Factor (ASF/MSF)

Computes Average Scale Factor (ASF) or Median Scale Factor (MSF) to measure how well reported uncertainty ellipses match actual errors.

- **ASF/MSF = 1.0**: Perfect calibration
- **ASF/MSF > 1.0**: Underestimated uncertainties (ellipses too small)
- **ASF/MSF < 1.0**: Overestimated uncertainties (ellipses too large)

```python
from gri_utils.stats import asf
import numpy as np

# Example: ellipse norms from position errors
ellipse_norms = np.array([0.8, 1.2, 0.95, 1.1, ...])  # Mahalanobis-like distances

# Average Scale Factor (default)
asf_val, ci_lower, ci_upper = asf(ellipse_norms)

# Median Scale Factor (more robust to outliers)
msf_val, ci_lower, ci_upper = asf(ellipse_norms, use_median=True)

# Custom confidence level
asf_val, ci_lower, ci_upper = asf(ellipse_norms, confidence=0.90)
```

**How it works:**

1. Compares empirical ellipse norms to theoretical Rayleigh distribution
2. For each sample: SF = ideal_norm / empirical_norm
3. ASF = mean(SF), MSF = median(SF)
4. CI uses empirically-tuned formula (ASF) or quantile CI (MSF)

**Requirements:**

- Minimum 25 samples for reliable results
- Returns (nan, nan, nan) if insufficient samples

### Ratio Confidence Interval (Walters Method)

Computes confidence interval for the ratio A/B where A and B are counts (e.g., successes vs failures, detections vs misses). Uses the Walters logarithm method from Gart and Nam (1988).

```python
from gri_utils.stats import ratio_ci

# Single ratio: 10 successes, 20 failures -> ratio = 0.5
lower, upper = ratio_ci(10, 20)
# lower ~= 0.27, upper ~= 0.94

# Multiple ratios (vectorized)
lower, upper = ratio_ci([5, 10, 15], [15, 20, 25])

# Custom confidence level
lower, upper = ratio_ci(10, 20, confidence=0.99)
```

**Mathematical basis:**

The Walters logarithm method uses a log-transformation to produce asymmetric intervals:

- C = A + B (degrees of freedom)
- ratio = A / B
- expo = t(1-alpha/2, C) \* sqrt(1/A + 1/B + 2/C)
- CI = [ratio \* exp(-expo), ratio \* exp(expo)]

**Notes:**

- Requires positive counts (A > 0 and B > 0)
- Returns (nan, nan) for non-positive inputs
- Produces asymmetric intervals appropriate for ratios
