autowisp.data_reduction.data_reduction_file module — AutoWISP documentation

# autowisp.data_reduction.data_reduction_file module

## Class Inheritance Diagram

Define a class for worknig with data reduction files.

class autowisp.data_reduction.data_reduction_file.DataReductionFile(*args, **kwargs)

  Bases: `HDF5FileDatabaseStructure`

  Interface for working with the pipeline data reduction (DR) files.

  _product

    The pipeline key of the HDF5 product. In this case: ‘data_reduction’

    Type:

      str

  _key_io_tree_to_dr

    A dictionary specifying the correspondence between the keys used in astrowisp.IOTree to store quantities and the element key in the DR file.

    Type:

      dict

  _dtype_dr_to_io_tree

    A dictionary specifying the correspondence between data types for entries in DR files and data types in astrowisp.IOTree.

    Type:

      dict

  __init__(*args, **kwargs)

    Open or create a data reduction file.

    Parameters:

      - arguments (See HDF5File.__init__() for description of)

      - however

      - fname (instead of)

      - header (a DataReductionFile can be specified by the)

      - object (of the frame it corresponds to (or at least a dict-like)

      - template). (defining the header keywords required by the DR filename)

  classmethod _get_root_tag_name()

    The name of the root tag in the layout configuration.

  _prepare_source_iter(dataset_key, column_substitution_name, **path_substitutions)

    Return required head and tail of paths identifying source collection.

    Parameters:

      get_sources(). (See)

    Returns:

      The path to the parent group containing all source columns.

      str: The string that must be in the beginning of each path for it

        to be considered part of the source collection.

      str: The string that must be in the end of each path for it to be

        considered part of the source collection.

    Return type:

      str

  add_frame_header(header, **substitutions)

    Add the header of the corresponding FITS frame to DR file.

  add_hat_astrometry(filenames, configuration, **path_substitutions)

    Add astrometry derived by fistar, and anmatch to the DR file.

    Parameters:

      - filanemes (dict) – The files containing the astrometry results. Should have the following keys: ‘fistar’, ‘trans’, ‘match’, ‘catalogue’.

      - configuration – An object with attributes containing the configuraiton of how astormetry was performed.

      - path_substitutions – See get_source_count()

    Returns:

      None

  add_magnitude_fitting(*, fitted_magnitudes, fit_statistics, magfit_configuration, missing_indices, **path_substitutions)

    Add a magnitude fitting iteration to the DR file.

    Parameters:

      - fitted_magnitudes (numpy.array) – The differential photometry corrected magnitudes of the sources.

      - fit_statistics (dict) – Summary statistics about how the fit went. It should define at least the following keys: `initial_src_count`, `final_src_count`, and `residual`.

      - magfit_configuration – The configuration structure with which magnitude fitting was performed.

      - missing_indices – A list of indices within the file of sources for which no entries are included in fitted_magnitudes.

    Returns:

      None

  add_sources(data, dataset_key, column_substitution_name, *, parse_ids=False, ascii_columns=(), **path_substitutions)

    Creates datasets out of the fields in an array of sources.

    Parameters:

      - data (structured numpy.array) – The data about the sources to add.

      - dataset_key (str) – The pipeline key for the dataset to add.

      - column_substitution_name (str) – The %-subsittution variable to distinguish between the column in the array.

      - parse_ids (bool) – Should self.parse_hat_source_id() be used to translate string IDs to datasets to insert?

      - string_columns ([str]) – A list of column names to convert to ascii strings before saving.

    Returns:

      None

  delete_sources(dataset_key, column_substitution_name, **path_substitutions)

    Delete all columns of a given source collection.

  fname_template = None
  get_dataset_creation_args(dataset_key, **path_substitutions)

    See HDF5File.get_dataset_creation_args(), but handle srcextract.

  get_dtype(element_key)

    Return numpy data type for the element with by the given key.

  classmethod get_fname_from_header(header)

    Return the filename of the DR file for the given header.

  get_frame_header(**substitutions)

    Return the header of the corresponding FITS frame.

  get_matched_sources(**path_substitutions)

    Get combined catalogue and extracted matched sources.

  get_num_apertures(**path_substitutions)

    Return the number of apertures used for aperture photometry.

  get_num_magfit_iterations(**path_substitutions)

    Return how many magnitude fitting iterations are in the file.

    Parameters:

      path_substitutions – See get_source_count().

    Returns:

      The number of magnitude fitting iterations performed on the set of photometry measurements identified by the path_substitutions argument.

    Return type:

      int

  get_source_count(**path_substitutions)

    Return the number of sources for the given tool versions.

    Parameters:

      path_substitutions – Values to substitute in the paths to the datasets and attributes containing shape fit informaiton (usually versions of various components).

    Returns:

      The number of projected sources in the databasets reached by the given substitutions.

    Return type:

      int

  get_source_data(*, magfit_iterations='all', shape_fit=True, apphot=True, string_source_ids=True, all_numeric_source_ids=False, background=True, **path_substitutions)

    Extract available photometry from the data reduction file.

    Parameters:

      - magfit_iterations (iterable) – The set of magnitude fitting iterations to include in the result. `0` is the raw photometry (i.e. no magnitude fitting), 1 is single reference frame fit, 1 is the first re-fit etc. Use `'all'` to get all iterations. Negative numbers have the same interpretation as python list indices. For example `-1` is the final iteration.

      - shape_fit (bool) – Should the result include shape fit photometry measurements. If `True` and no shape fit is present, still excludes shape fit columns.

      - apphot (bool) – Should the result include aperture photometry measurements.

      - string_source_ids (bool) – Should source IDs be formatted as strings (True) or a set of integers (False)?

      - background (bool) – Should the result include information about the background behind the sources?

      - path_substitutions – See get_source_count().

    Returns:

      The photometry information in the current data reduction file.

        The columns always included are:

        > - ID(set as index): an array of sources IDs in the given DR file. Either a string (if string_source_ids) or 1- or 3-column composite index depending on ID type.

        > - <catalogue quantity> (dtype as needed): one entry for each catalogue column.

        > - x (numpy.float64): The x coordinates of the sources

        > - y (numpy.float64): The y coordinates of the sources

      The following columns are included if the corresponding input argument is set to True:

      > - bg (numpy.float64): The background estimates for the sources

      > - bg_err (numpy.float64): Error estimate for ‘bg’

      > - bg_npix (numpy.uint): The number of pixel background extraction was based on.

      > - mag (2-D numpy.float64 array): measured magnitudes. The first dimension is the index within the `magfit_iterations` argument and the second index iterates over photometry, starting with shape fitting (if the `shape_fit` argument is True), followed by the aperture photometry measurement for each aperture (if the `apphot` argument is True).

      > - mag_err (numpy.float64): Error estimate for `mag`. Same shape and order.

      > - phot_flag: The quality flag for the photometry. Same shape and order as `mag`.

    Return type:

      pandas.Dataframe

  get_source_ids(string_source_ids=True, **path_substitutions)

    Return the IDs of the sources in the given DR file.

    Parameters:

      - string_source_ids – Should source IDs be formatted as strings (True) or a set of integers (False)?

      - path_substitutions – See get_source_count().

    Returns:

      See ID field of result in get_source_data().

    Return type:

      numpy.array

  get_sources(dataset_key, column_substitution_name, **path_substitutions)

    Return a collection of sources previously stored in the DR file.

    Parameters:

      - dataset_key (str) – The pipeline key for the dataset to return.

      - column_substitution_name (str) – The %-subsittution variable to distinguish between the column in the array.

    Returns:

      The keys are the columns of the sources stored and the values are 1-D numpy arrays containing the data.

    Return type:

      dict

  has_shape_fit(accept_zeropsf=True, **path_substitutions)

    True iff shape fitting photometry exists for path_substitutions.

  parse_hat_source_id(source_id)

    Return the prefix ID, field number, and source number.

  save_source_extracted_psf_map(*, fit_results, fit_configuration, **path_substitutions)

    Create the datasets and attributes holding the fit results.

autowisp.data_reduction.data_reduction_file.init_dr_process(configuration)

  Initialize the process with the given configuration.