Description of Outputs from an ETC Calculation
==============================================

Exceptions:

  Normally expected exceptions are of base class etc3d.custom_exceptions.PandeiaException.
  The attribute .message contains a plain text description of the problem
  suitable for display to the user.

  There are numerous subclasses of PandeiaException that are not documented
  here, but can be seen in etc3d.custom_exceptions


The return value from perform_calculation() is a dict.  The rest of this document describes the keys in that dict.

error : string
  This string is not reported by the engine, but the name is reserved
  by the server.  See doc/transactions/ws_trans.txt

admin_data : string
  This string is not reported by the engine, but the name is reserved
  by the server.  See doc/transactions/ws_trans.txt

server_test : dict
  This data is not reported by the engine, but the name is reserved
  by the server.  See doc/transactions/ws_trans.txt

transform: dict
  This contains the information that describes the axes in the 2D
  and 3D images (e.g.  sizes, extents, and step sizes). This
  information is used to map pixel coordinates to world coordinates.
  There are two sets of numbers for the wavelength axis because the
  wavelength sampling of the spectrum used in the calculation is
  usually different from the wavelength sampling at the detector
  plane. The detector plane sampling is usually much finer. It would
  be inefficient to perform the ETC calculation at the full detector
  wavelength resolution if that's not required.

    x_refpix: int
        Reference pixel for the X axis. Not sure when it wouldn't be 0.
    x_refval: float
        Value of X at x_refpix.
    x_max: float
        Maximum X value.
    x_min: float
        Minimum X value.
    x_step: float
        Size of pixel in X direction in arcseconds/pixel.
    x_size: int
        Length of X axis.

    y_refpix: int
        Reference pixel for the Y axis.
    y_refval: float
        Value of Y at y_refpix.
    y_max: float
        Maximum Y value.
    y_min: float
        Minimum Y value.
    y_step: float
        Size of pixel in Y direction in arcseconds/pixel.
    y_size: int
        Length of Y axis.

    wave_refpix: int
        Reference pixel for the wavelength axis.
    wave_refval: float
        Value of wavelength at wave_refpix.
    wave_max: float
        Maximum wavelength value.
    wave_min: float
        Minimum wavelength value.
    wave_step: float
        Size of pixel in wavelength axis in microns/pixel
    wave_size: int
        Length of wavelength axis

    wave_det_refpix: int
        Reference pixel for the detector-sampled wavelength axis
    wave_det_refval: float
        Value of wavelength at wave_det_refval
    wave_det_max: float
        Maximum wavelength value sampled at the detector plane
    wave_det_min: float
        Minimum wavelength value sampled at the detector plane
    wave_det_step: float
        Size of detector pixels in wavelength axis in microns/pixel
    wave_det_size: int
        Length of the detector-sampled wavelength axis

.. image:: oapi_coords.gif

3d: dict
  The 3-D data products generated by the engine calculation

    flux: list of numpy.ndarray (3-D)
       The model flux cubes used in the calculation, excluding background. Units of mJy/pixel.
    flux_plus_background: list of numpy.ndarray (3-D)
       The model flux cubes used in the calculation, including signal from the background.
       The background only can be calculated as flux_plus_background - flux. Units of mJy/pixel.
    reconstructed: numpy.ndarray (3-D) (IFU calculations only)
       This is a reconstructed simulation of an observed flux cube
       generated by IFU calculations. Units of e-/s/pixel
    reconstructed_signal: numpy.ndarray (3-D) (IFU calculations only)
        The signal component of the reconstructed flux cube.  Units of e-/s/pixel
    reconstructed_noise: numpy.ndarray (3-D) (IFU calculations only)
        The noise component of the reconstructed flux cube.  Units of e-/s/pixel
    reconstructed_snr: numpy.ndarray (3-D) (IFU calculations only)
        reconstructed_signal divided by reconstructed_noise.
    reconstructed_saturation: numpy.ndarray (3-D) (IFU calculations only)
        Reconstructed saturation cube.

2d: dict
    The 2-D data products generated by the engine calculation.  Axis
    descriptions are given in 'transform'.

    detector: numpy.ndarray (2-D)
        ['2d']['detector'](x,y) is the count rate at the detector in e-/sec, including simulated noise.
        This is the product closest to what a "real" observation would look like.
        Shown in 'Detector' plot in GUI.

    snr: numpy.ndarray (2-D)
        ['2d']['snr'](x,y) is the S/N at the detector.
        Shown in '2D SNR' plot in GUI.

    saturation: numpy.ndarray (2-D)
        Saturation map at the detector. 0 -> no saturation, 1 -> partial saturation, 2 -> full saturation
        Shown in 'Saturation' plot in the GUI.

    ngroup_map: numpy.ndarray(2-D)
        Map of the maximum number of groups BEFORE a given pixel will saturate, with current instrument
        and detector setup.
        Shown in 'Groups Before Saturation' plot in the GUI.

1d: dict
  The 1-D data products generated by the engine calculation. Here
  we use 'Target' to refer to the position within the Scene defined
  by the Strategy.

    In these descriptions:
     - wave is wavelength in microns

    wave_pix: numpy.ndarray (1-D)
        Wavelength in microns as sampled in pixels on the detector.

    wave_calc: numpy.ndarray (1-D)
        Wavelength set used in the calculation before sampled and mapped
        onto the detector plane.

    target: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, Input integrated scene flux (in mJy). This includes flux from all defined sources
        within the scene, but excludes background.
        n.b. Target is the pixel specified as the target in the Strategy
        NOT currently displayed on GUI

    fp: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, integrated scene count rate at the focal plane in e-/s/pixel/micron.
        This excludes background, but includes signal from all defined sources within the scene.
        The calculation of fp includes throughputs from filters, dispersers, and internal optics as well as detector QE,
        and includes losses from pupil masks and focus apertures (e.g. slits). NOT currently displayed on GUI

    bg: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, input background surface brightness in MJy/sr
        This is the input background surface brightness binned to the wavelength
        scale used in the calculation, wave_calc.
        NOT currently displayed on GUI

    bg_rate: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, Background e- rate at focal plane
        This is the count rate per detector pixel in e-/sec/pixel
        and is spatially (but not necessarily spectrally) constant.
        NOT currently displayed on GUI

    sn: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, Signal/Noise Ratio
        Shown under 'SNR' in GUI.

    extracted_flux: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, Target count rate on the detector in e-/sec, as extracted by the defined Strategy.
        Shown under "ApFlux" in GUI

    extracted_flux_plus_bg: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, Target count rate on the detector including
        background in e-/sec, as extracted by the defined Strategy.
        If background subtraction is being performed, this should be the same as extracted_flux.
        NOT currently displayed on GUI

    total_flux: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, integrated scene count rate on the detector in e-/s. This includes signal from
        all sources within the scene, but does not include background.
        NOT currently displayed on GUI

    extracted_bg_total: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, total background count rate in e-/sec. This includes all flux contained within
        the background extraction region and is normalized to the target extraction region by the ratio
        of the areas of the regions.
        NOT currently displayed on GUI

    extracted_bg_only: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, background count rate in e-/sec. This includes only the flux due to the scene background
        and is normalized to the area of the target extraction region.
        Displayed in the GUI as "ApBackground"

    extracted_contamination: list [numpy.ndarray (1-D), numpy.ndarray (1-D)]
        Wavelength in microns, contamination count rate in e-/sec.  This is extracted_bg_total - extracted_bg_only and
        gives a measure of how much non-background flux (e.g. PSF wings or contamination sources) is contained within
        the background estimation region.  The count rate is normalized to the target extraction region area.
        NOT currently displayed on GUI

scalar: dict
  The scalar quantities generated by the engine Strategy.

    extracted_flux: float
        Total flux (e-/sec) within the extraction aperture. This is determined at the reference_wavelength for
        dispersed modes. Omitted for strategies that do not specify an extraction aperture.
    extracted_noise: float
        Noise (standard deviation, e-/sec) within the extraction aperture. This is determined at the
        reference_wavelength for dispersed modes. Omitted for strategies that do not specify an extraction aperture.
    sn: float
        Extracted signal-to-noise ratio (extracted_flux/extracted_noise). This is determined at the
        reference_wavelength for dispersed modes. Omitted for strategies that do not specify an extraction aperture.
    background_total: float
        Extracted background flux (e-/sec) including all components. This is determined at the reference_wavelength
        for dispersed modes.
    background_sky: float
        Extracted sky background flux (e-/sec) excluding any signal from the scene. This is determined at the
        reference_wavelength for dispersed modes.
    contamination: float
        Fraction of background_total that is due to signal from the scene:
        (background_total-background_sky)/background_total
    background: float
        Background surface brightness in MJy/sr at reference_wavelength.
    reference_wavelength: float
        Wavelength where scalar values are determined, if applicable (i.e. for dispersed modes) (microns).
    total_exposure_time: float
        Total exposure time for the calculation as determined by the exposure specification and strategy (seconds).
        This may include additional dithers introduced by certain strategies (for instance, IFUNodOffScene will include
        time for exposures pointing to a background region.)
    exposure_time: float
        Exposure time for a single exposure specification. Identical to the APT exposure time.
    sat_ngroups: int
        Highest number of groups (in current instrument and detector configuration) possible before the brightest pixel
        on the detector begins to saturate.
    fraction_saturation: float
        Fraction of saturation of the brightest pixel on the detector. Can be greater than 1 if saturated.
    duty_cycle: float
        Ratio between "measurement time" and "exposure time". Is a measure for the fraction of the exposure time being
        used to collect scientifically useful photons.
    cr_ramp_rate: float
        Assumed cosmic ray rate in events/ramp where ramp time is determined by the exposure specification.
    background_area: float or None
        Area in pixels of background estimation region. None if background subtraction is turned off.
    extraction_area: float
        Area in pixels of target extraction region.
    contrast_separation: float
        Radius in arcsec at which contrast is measured. Only applicable to coronagraphy.
    contrast_azimuth: float
        Azimuth in degrees where contrast is measured. Only applicable to coronagraphy.
    contrast: float
        Contrast measured at polar coords (contrast_separation, contrast_azimuth). Only applicable to coronagraphy.
    filter: string
        Instrument filter, if any, used in calculation.
    disperser: string
        Instrument disperser, if any, used in calculation.
    x_offset: float
        X offset in arcsec of source extraction aperture
    y_offset: float
        Y offset in arcsec of source extraction aperture
    aperture_size: float
        Radius in arcsec of source extraction aperture

information: dict
  This contains other information provided by the engine that isn't
  the result of any calculation.  This will also need to get fleshed
  out over time.

    calc_type: string
        Type of calculation as reported by the engine ('spec' or 'image')

    exposure_specification: dict
        nsample: int
           Number of samples of a pixel for each frame
        nsample_skip: int
           Number of samples of a pixel skipped for each frame
        nframe: int
            Number of frames read out and saved per group
        nskip: int
            Number of frames skipped per group
        ngroup: int
            Number of groups in a ramp
        nint: int
            Number of integrations per exposure
        nexp: int
            Number of exposures
        nramps: int
            Total number of ramps (nexp * nint)
        nprerej: int
            Number of groups rejected by the pipeline in the beginning of the ramp
        npostrej: int
            Number of groups rejected by the pipeline in the end of the ramp
        tframe: float
            Readout time per frame (seconds)
        tgroup: float
            Total time required per group (seconds)
        total_exposure_time: float
            Total exposure time for the calculation as determined by the exposure specification and strategy (seconds).
            This may include additional dithers introduced by certain strategies (for instance IFUNodOffScene will
            include time for exposures pointing to a background region.)
        exposure_time: float
            Exposure time for a single exposure specification. Identical to the APT exposure time.
        measurement_time: float
            Time in an exposure used to collect scientifically useful photons.
        saturation_time: float
            Time in a ramp relevant for saturation
        det_type: string
            Type of detector (currently h2rg and sias supported)
        subarray: string
            Portion of detector being read out
        pattern: string
            Name of readout pattern

input: dict
  A copy of the input dict used in the calculation.  This may be
  superfluous for most of our purposes.  The GUI won't use it and
  testing doesn't require it, either.

sub_reports: list of dicts
  Calculations involving multiple pointings will include this. It
  is a list of engine output API format dicts containing with one
  entry per pointing.

warnings: dict
  This contains any warnings or other messages, and is what appears in the 'Warnings' tab
  in the UI.
  Warnings include (among others):

  * Warnings for fully saturated pixels (where full saturation is defined as saturation in the first or second read of the ramp)

  * Warnings for partially saturated pixels (where partial saturation is defined as saturation after at least two reads)

  * Warnings for low SNR

  * Warnings for apertures extending outside the calculated region

  * Warnings for backgrounds and extraction apertures being more than 10% smaller than requested

  * Warnings for targets obscured behind a mask
