.. _linear_sensitivity:


===========================
Example: Linear Map Choose QoIs to minimize scale of inverses in measure
===========================
Here, we consider a simple example where a parameter space is given
by a 5-dimensional hypercube and the goal is to choose an optimal QoI
map from a space of possible QoI maps, denoted by :math:`\mathcal{Q}`,
where each QoI map is linear.
We use this simple example to demonstrate the use of the code to
optimally choose which possible QoI map does the best job of "scaling"
inverse sets to smaller sets.
The idea is that if we generally consider a set of high probability
in a particular data space defined by the range of a QoI map, we would
prefer that the inverse of this set is as small as possible in order to
try and identify the parameter responsible for the data.
This only makes sense for stochastic inverse problems framed within the
context of parameter identification under uncertainty.
In other words, when the problem is that the data are uncertain due to
measurement uncertainty and there is a true/exact parameter responsible
for whichever uncertain data is observed, then this is the type of problem
for which this optimization criteria is most appropriate.

This example generates uniform random samples in the unit hypercube and
corresponding QoIs (data) generated by a linear map Q.
We then calculate the
gradients using an RBF scheme and use the gradient information to choose the
optimal set of 2 (3, 4, ... input_dim) QoIs to use in the inverse problem.
This example also demonstrates how we can define :math:`\mathcal{Q}` implicitly
by creating a ``larger'' vector-valued QoI map in the case where the
cardinality is finite and determined by selecting any arbitrary subset of
a defined collection of scalar QoI.

Step (0): Setting up the environment
===========================
Import the necessary modules::

    import numpy as np
    import bet.sensitivity.gradients as grad
    import bet.sensitivity.chooseQoIs as cqoi
    import bet.calculateP.simpleFunP as simpleFunP
    import bet.calculateP.calculateP as calculateP
    import bet.postProcess.postTools as postTools
    import bet.Comm as comm
    import bet.sample as sample

Step (1): Define the space of possible QoI maps
==============================================
We have a 5 dimensional hypercube for the parameter space and we assume
that the space of possible QoI, :math:`\mathcal{Q}`, is defined by some random linear maps.
Specifically, we randomly generate 10 individual scalar linear QoI maps and
then consider :math:`\mathcal{Q}` to be defined by any 2, 3, ..., ``input_dim`` dimensional
vectors from this set of 10::

    # Setup the info for the spaces
    input_dim = 5
    QoI_scalar_map_num = 10
    num_samples = 1E5
    num_centers = 10

However, rather than define :math:`\mathcal{Q}` explicitly, we consider
a larger vector-valued map defined by ``Q`` that contains all of the scalar
QoI maps. We later determine the optimal QoI maps in terms of the components
of this larger map ``Q``::

    # Let the map Q be a random matrix of size (QoI_scalar_map_num, input_dim)
    np.random.seed(0)
    Q = np.random.random([QoI_scalar_map_num, input_dim])


Step (2): Create the discretization object from the input and output samples
============================================================================
We randomly sample the input space and generate the corresponding QoI values::

    # Initialize some sample objects we will need
    input_samples = sample.sample_set(input_dim)
    output_samples = sample.sample_set(QoI_scalar_map_num)

    # Choose random samples in parameter space to solve the model
    input_samples.set_values(np.random.uniform(0, 1, [num_samples, input_dim]))

    # Compute the output values with the map Q
    output_samples.set_values(Q.dot(input_samples.get_values().transpose()).\
            transpose())

    cluster_discretization = sample.discretization(input_samples, output_samples)

Step (3): Compute the gradients of all the maps in :math:`\mathcal{Q}`
======================================================================
We calculate the gradients using an RBF-FD scheme.
We set ``normalize=True`` because we assume that the sizes of the inverse sets to
be inverted in the data space are scaled relative to the size of the circumsribing box
of the data space.
Thus, all uncertainties are in a sense relative to the sensitivity
of the overall QoI map::

    center_discretization = grad.calculate_gradients_rbf(cluster_discretization,
            num_centers, normalize=True)


Step (4): Compute scaling properties of the maps and choose the best QoI
=========================================================================
With these gradient vectors, we are now ready to choose an optimal set of
QoIs to use in the inverse problem, based on minimizing the measures of
inverse sets defined by the scaling effect of the map.
The most robust method for this is
:meth:~bet.sensitivity.chooseQoIs.chooseOptQoIs_large which returns the
best set of 2, 3, 4 ... until input_dim.  This method returns a list of
matrices.  Each matrix has 10 rows, the first column representing the
expected scaling of the cross-sections of the contour event associated
with inverting a rectangular box, and the rest of the columns the corresponding
QoI indices.::

    input_samples_center = center_discretization.get_input_sample_set()
    best_sets = cqoi.chooseOptQoIs_large(input_samples_center, measure=True)


Step (5)[Optional]: Seeing the effect of the optimal QoI choice
================================================================
At this point we have determined the optimal set of QoIs to use in the inverse
problem.
Now we compare the support of the inverse solution using
different sets of these QoIs.
The user can select which QoI_indices to choose from each time::

    QoI_indices = best_sets[0][0, 1:].astype(int) # Chooses the optimal set of 2 QoI
    # QoI_indices = best_sets[0][1,1:].astype(int) # Chooses the second most optimal set of 2 QoI
    # QoI_indices = best_sets[0][9,1:].astype(int) # Chooses the least optimal set of 2 QoI
    # QoI_indices = best_sets[3][0,1:].astype(int) # Chooses the optimal set of 5 QoI
    # QoI_indices = best_sets[3][1,1:].astype(int) # Chooses the second most optimal set of 5 QoI

Setup the associated set of output samples from the choice of QoI defined by the
QoI_indices::

    output_samples._dim = len(QoI_indices)
    output_samples.set_values(output_samples.get_values()[:, QoI_indices])

Define the reference datum in the output space to correspond to the center of
the input space.
Q_ref = Q[QoI_indices, :].dot(0.5 * np.ones(input_dim))

``rect_scale`` defines the amount of uncertainty in the measured datum relative
to the scale of a circumscribing box of the data space::

    rect_scale = 0.25

Make the MC assumption and compute the volumes of each Voronoi cell::

    input_samples.estimate_volume_mc()

Create the discretization object::

    my_discretization = sample.discretization(input_sample_set=input_samples,
                                        output_sample_set=output_samples)


Compute the simple function approximation::

    simpleFunP.regular_partition_uniform_distribution_rectangle_scaled(
        data_set=my_discretization, Q_ref=Q_ref, rect_scale=rect_scale,
        cells_per_dimension=1)

Compute the solution to the stochastic inverse problem::

    calculateP.prob(my_discretization)

Determine which samples have non-zero probability, which, due to the MC assumption
on the volumes, can be used to approximate the ratio of the measure of the parameter
space defined by the support of the probability density function solving the inverse
problem::

    (_, _, indices_in_inverse) =\
        postTools.sample_highest_prob(top_percentile=1.0,
        sample_set=input_samples, sort=True)

    # Print the approximate percentage of the measure of the parameter space defined
    # by the support of the inverse density
    if comm.rank == 0:
        print 'The approximate percentage of the measure of the parameter space defined'
        print 'by the support of the inverse density associated with the choice of QoI map is'
        print np.sum(input_samples.get_volumes()[indices_in_inverse])