Mathematical background
=======================

Fisher information matrix
-------------------------

The Fisher information measures the amount of information that a set of
observables :math:`X_i` carries about unknown parameters :math:`\theta_i`, upon
which the probability distribution of the observables depends.

When there are :math:`N` parameters, i.e., :math:`\theta = (\theta_1, \ldots,
\theta_N)`, the Fisher information takes the form of an :math:`N \times N`
matrix. The Fisher Information Matrix (FIM) :math:`I(\theta)` for :math:`X_a`
has elements given by

.. math::

    I_{ij}(\theta) = \mathrm{E} \left[ \left( \frac{\partial \log p(X_a |
    \theta)}{\partial \theta_i} \right) \left( \frac{\partial \log p(X_a |
    \theta)}{\partial \theta_j} \right) \middle| \theta \right],

where :math:`p(X_a | \theta)` is the probability of the observable :math:`X_a`
given the parameters :math:`\theta`, and :math:`\mathrm{E}` is the expectation
value over the distribution of :math:`X_a` (here, evaluated at the true
parameter values).

Note that, under the `regularity conditions`_, the FIM can also be expressed as

.. math::

    I_{ij}(\theta) = - \mathrm{E} \left[ \frac{\partial^2 \log p(X_a |
    \theta)}{\partial \theta_i \partial \theta_j} \middle| \theta \right].

Note that these expressions are valid for any probability distribution of the
observables; however, they involve the expectation value over the distribution
of the observables, which is often not known analytically. In practice, the
FIM is often estimated by sampling from the distribution of the observables, for
example using Monte Carlo methods.

Matched-filter Gaussian-noise likelihood
----------------------------------------

Consider a set of observables :math:`X_a` defined as

.. math::

    X_a = h_a(\theta) + n_a,

where :math:`h_a(\theta)` is a template waveform (including the instrument
response) and :math:`n_a` is a noise term. If this noise is Gaussian, and under
the `regularity conditions`_, the components of the FIM no longer involve the
expectation value over the distribution of the observables; but instead can be
directly evaluated as

.. math::

    I_{ij}(\theta) = \left( \frac{\partial h}{\partial \theta_i} \middle|
    \frac{\partial h}{\partial \theta_j} \right),

where the frequency-domain noise-weighted inner product is defined as

.. math::

    (a|b) = 4 \Re \int_{f_{\min}}^{f_{\max}} \tilde{a}(f) \Sigma^{-1}(f)
    \tilde{b}^+(f) \, \mathrm{d}f,

with :math:`\tilde{a}(f)` and :math:`\tilde{b}(f)` the Fourier transforms
of the signals :math:`a(t)` and :math:`b(t)`, respectively, and
:math:`\Sigma(f)` the noise covariance matrix.

Here, we have assumed that the noise is independent of the model parameters
:math:`\theta`, and that the noise is stationary, such that there is no
correlation between the noise at different frequencies (i.e., the noise
covariance matrix is frequency-dependent and only characterize the noise
correlations between the various observables channels).



The inner product is implemented in :func:`wejax.fim.inner_product`, and the
matched-filter Gaussian-noise FIM is implemented in
:func:`wejax.fim.matched_filter_gaussian_fim`.

.. _regularity conditions: https://en.wikipedia.org/wiki/Fisher_information#Regularity_conditions

Error estimation
----------------

The inverse of the FIM provides an estimate of the covariance matrix
:math:`\Sigma` of the parameters, which can be used to estimate the errors on
the parameters. The diagonal elements of the covariance matrix provide the
variance of the parameters, while the off-diagonal elements provide the
covariance between the parameters,

.. math::

    \mathrm{Cov}(\theta_i, \theta_j) = \Sigma_{ij} = \left( I^{-1} \right)_{ij}.

The standard deviation of the parameters is given by the square root of the
diagonal elements of the covariance matrix,

.. math::

    \sigma(\theta_i) = \sqrt{\Sigma_{ii}} = \sqrt{\left( I^{-1} \right)_{ii}}.


The correlation between the parameters is given by the off-diagonal elements of
the covariance matrix divided by the square root of the product of the diagonal
elements,

.. math::

    \rho(\theta_i, \theta_j) = \frac{\Sigma_{ij}}{\sqrt{\Sigma_{ii}
    \Sigma_{jj}}} = \frac{\left( I^{-1} \right)_{ij}}{\sqrt{\left( I^{-1}
    \right)_{ii} \left( I^{-1} \right)_{jj}}}.

The correlation matrix :math:`P` is usually defined as the matrix containing
the correlation coefficients :math:`\rho(\theta_i, \theta_j)` as the
off-diagonal elements. Here we will also include the standard deviations :math:`\sigma(\theta_i)` as
the diagonal elements instead of the usual unit entries. This gives a matrix of the form:

.. math::

    P_{ij} =
    \begin{cases}
        \sigma(\theta_i) & \text{if } i = j, \\
        \rho(\theta_i, \theta_j) & \text{if } i \neq j.
    \end{cases}

This form combines both the scale (via standard deviations) and the correlation
structure between parameters, and can be helpful for visualization or uncertainty reporting.

To convert the FIM to a covariance or a correlation matrix, use
:func:`wejax.fim.fim_to_covariance` or :func:`wejax.errors.fim_to_correlation`.
You can plot the latter using :func:`wejax.errors.plot_correlation_matrix`.
