.. include:: ../references.txt

Astrometry Calibration
======================

Performing astrometric calibration is simple in |astropop|. It uses as backend the `astrometry.net`_ program.

In summary, `astrometry.net`_ searches a given field in the sky and computes a `World Coordinate System <https://fits.gsfc.nasa.gov/fits_wcs.html>`_ astrometric solution that can be used to relate a image coorditate ``(x, y)`` to physical sky coordinates, like ``(ra, dec)``.

For a more detailed WCS usage, see |astropy| documentation of |WCS|.

About astrometry.net
--------------------

From `astrometry.net`_ readme: Automatic recognition of astronomical images; or standards-compliant astrometric metadata from data.

So, `astrometry.net`_ creates robust |WCS| calibrations with almost none or no information about the field. It searches the solution around the sky using `index files <http://astrometry.net/doc/readme.html#getting-index-files>`_ and find optimal solutions using a very smart and fast algorithm based on 4-stars asterisms. Detailed explanation of `astrometry.net`_ is given in `Lang et al. (2010) <https://ui.adsabs.harvard.edu/abs/2010AJ....139.1782L/abstract>`_ atricle.

This code can be very fast if enough information about the field is needed. But, slowly, it can find the solution completely blind. It also can run locally in a common laptop. However, to run locally it needs the index files that, some times, can use a large disk space.

An online service is also available at `nova.astrometry.net <https://nova.astrometry.net>`_ and can be accessed by python using `astroquery.astrometry_net <https://astroquery.readthedocs.io/en/latest/astrometry_net/astrometry_net.html>`_ interface. This is recomended for who don't want to download the index files.

Installing astrometry.net
~~~~~~~~~~~~~~~~~~~~~~~~~

`astrometry.net`_ is a C written language that runs in command line. Currently, it only is supported on Linux and Mac. Windows is not supported by the developers. Due to this, its install is a bit tricky and, for most users, it may envolve building from the code. Detailed instructions about building and installing from the code or Linux distribution-specific installs can be found in `astrometry.net installing read-me <http://astrometry.net/doc/build.html#build>`_.

For `Anaconda <https://anaconda.com>`_ users, the project `conda-forge <https://conda-forge.org/>`_, that is a collection of community maintaned packages, provide a simplier way to install `astrometry.net`_ inside conda environments. You just need to:

.. code-block:: bash

    conda install -c conda-forge astrometry

or, withing a specific separated environment, use:

.. code-block:: bash

    conda install -n <environment_name> -c conda-forge astrometry

Index files
~~~~~~~~~~~

There are several sets of pre-built index files that can be used to run `astrometry.net`_. They are listed in `<http://data.astrometry.net/>`_. For more informations about how to download it, see `astrometry.net indexes read-me <http://astrometry.net/doc/readme.html#getting-index-files>`_.

If you want to download the index files and do not change anything to get it running, the files must be put in ``$INSTALL_DIR/data`` folder. If you download it to another location, you must change the ``$INSTALL_DIR/etc/astrometry.cfg``, adding the ``add_path <location of the indexes>`` line. `astrometry.net`_ will search all the folders listed in the configuration to seek index files.

In |astropop|, there is also an option in all the functions that compute astrometric solutions where you can set the index locations. See individual functions for more information.

Astropop wrapper usage
----------------------

|astropop| do not implement the solution by itself, it just wraps around the `astrometry.net`_ command-line code.

There is two ways to run the `astrometry.net`_ code inside |astropop|. Using a class-like interface and specific helper funcions. |astropop| will take care of prepare the inputs, monitor the process (with realtime logging output), read the outputs and organize them in a structure useful in Python. So you don't need to manually read the solution and deal with a ton of files produced by `astrometry.net`_.

All of them run the code in a similar way, giving the result in the same format.

Solution Format
~~~~~~~~~~~~~~~

The `astrometry.net`_ outputs several files, including plots, but, considering the astrometric solution itself, just a few of them are really useful. So, we decided to store this solution in a |AstrometricSolution| container. It contains:

- wcs: the |WCS| solved instance, created from the solved header.
- header: the FITS |Header| generated by `astrometry.net`_ containing the solution.
- correspondences: a |Table| containing the correspondences between stars seen in the field and the index file used to solve the astrometry by `astrometry.net`_.

To manually build an |AstrometricSolution| container, you just need to pass the solved header and, optionally, the correspondences table.

.. ipython:: python

    from astropop.astrometry import AstrometricSolution
    from astropy.utils.data import get_pkg_data_filename
    from astropy.io import fits
    fn = get_pkg_data_filename('data/j94f05bgq_flt.fits', package='astropy.wcs.tests')
    header = fits.open(fn)[1].header
    # Correspondeces is optional
    solution = AstrometricSolution(header)
    print(solution.wcs)

Class-like interface
~~~~~~~~~~~~~~~~~~~~

.. |solve_field| replace:: `~astropop.astrometry.AstrometrySolver.solve_field`

The most general wrapper interface implemented is trough a class-like object. The |AstrometrySolver| class can be used to run the ``solve-field`` `astrometry.net`_ interface and return the results in a very pythonic way.

Its job is to organize the inputs, options and configuration files, prepare and execute the ``solve-field`` command, check for solution and return a |AstrometricSolution| object if there is a solution.

To run the command, you must create a instance of |AstrometrySolver| and execute it using |solve_field| method. The |solve_field| method has, as main input, a ``filename`` that can be solved by ``solve-field`` `astrometry.net`_ command. There are several types accepted, like FITS files, common images (like .png, .jpg), and XYLS tables. For more information see the `astrometry.net solving documentation <http://astrometry.net/doc/readme.html#solving>`_.

.. ipython::
    :verbatim:

    # get the sample image
    In [2]: from astroquery.skyview import SkyView

    In [3]: from astropy.coordinates import Angle

    In [4]: filename = 'M20_test_file.fits'

    In [5]: s = SkyView.get_images('M20', radius=Angle('60 arcmin'),
       ...:                        pixels=2048, survey='DSS')

    In [6]: s[0][0].writeto(filename)

    # Run the solution
    In [8]: from astropop.astrometry import AstrometrySolver

    In [9]: solver = AstrometrySolver()

    In [10]: solution = solver.solve_field(filename,
       ....:                               options={'radius': 5.0})
    Out[10]: log messages appear here

    In [11]: solution.wcs
    Out[11]:
    WCS Keywords
    Number of WCS axes: 2
    CTYPE : 'RA---TAN-SIP'  'DEC--TAN-SIP'
    CRVAL : 270.675222111  -22.9719850171
    CRPIX : 1024.5  1024.5
    CD1_1 CD1_2  : -0.000976549933759  1.01589915847e-08
    CD2_1 CD2_2  : 7.53462015161e-08  0.00097650746147
    NAXIS : 0  0

    In [12]: solution.correspondences
    Out[12]:
    <Table length=253>
         field_x            field_y       ...    FLUX   BACKGROUND
          pixels             pixels       ...  unknown   unknown
         float64            float64       ...  float32   float32
    ------------------ ------------------ ... --------- ----------
     45.64460754394531  1968.970458984375 ... 21437.375   1706.625
    1065.0677490234375  1092.709228515625 ... 21186.135  2324.8652
       705.81591796875 1440.1300048828125 ... 21338.012  1834.9883
     1705.599365234375 158.44154357910156 ... 21252.873   1919.127
     787.1795654296875  640.0123291015625 ... 21140.668   1873.332
    1748.0179443359375  996.6746215820312 ... 20962.152  1870.8477
                   ...                ... ...       ...        ...
    1808.2672119140625 265.38702392578125 ... 12775.386  1874.6143
    241.27725219726562  670.1822509765625 ... 13805.926  1879.0742
    1636.2926025390625 113.85559844970703 ... 12501.093  1906.9072
      616.659423828125   44.4092903137207 ... 13424.324  1877.6758
     587.6741943359375   186.121826171875 ...  12751.13  1899.8701
     2021.401611328125 267.57586669921875 ...  9990.552  1847.4482


Using keyword arguments to be passed to |run_command| you can store ``stdout`` and ``stderr`` of ``solve-field`` in lists. Check the API from more details. Also, setting ``stdout_loglevel`` and ``stderr_loglevel`` you can print the live output of the code using `~logging` system.

.. ipython::
    :verbatim:

    In [2]: solver = AstrometrySolver()

    In [3]: stdout = []

    In [4]: stderr = []

    In [5]: solution = solver.solve_field(filename, options={'radius': 5.0},
       ...:                               stdout=stdout, stderr=stderr)

    In [6]: print("\n".join(stdout))
    Reading input file 1 of 1: "M20_test_file.fits"...
    Found an existing WCS header, will try to verify it.
    Extracting sources...
    simplexy: found 13193 sources.
    Solving...
    Reading file "/tmp/M20_test_file_6kk3quwg_astrometry.net/M20_test_file.axy"...
    Verifying WCS using indices with quads of size [6, 60] arcmin
    Verifying WCS with index 1 of 1 (/data/index-4107.fits)
    Verifying WCS of field 1.
      log-odds ratio 536.02 (6.17207e+232), 66 match, 0 conflict, 93 distractors, 68 index.
      RA,Dec = (270.675,-22.9719), pixel scale 1.75781 arcsec/pix.
      Hit/miss:   Hit/miss: +++++++++++++++++++++++++-++++-+++++++-+--+--+--++++--+++-+-+--+++-+++--+--------++-+--+-----++-----
     --> log-odds 536.02
    Got 1 solutions.
    Field 1 solved: writing to file /tmp/M20_test_file_6kk3quwg_astrometry.net/M20_test_file.solved to indicate this.
    Field: M20_test_file.fits
    Field center: (RA,Dec) = (270.675235, -22.972045) deg.
    Field center: (RA H:M:S, Dec D:M:S) = (18:02:42.056, -22:58:19.363).
    Field size: 59.9993 x 59.9921 arcminutes
    Field rotation angle: up is 0.00276933 degrees E of N
    Field parity: pos
    Creating new FITS file "/tmp/M20_test_file_6kk3quwg_astrometry.net/M20_test_file.new"...

Also, you can pass any parameter of ``astrometry.cfg`` file to the options. These parameters will be organized and a custom configuration file will be created, enabling these options only for that run. Manual edition of the config is also available using `~astropop.astrometry.AstrometrySolver.config` dictionary.

.. ipython:: python
    :verbatim:

    solver = AstrometrySolver(config={'index': 'index-401.fits',
                              'add_path': ['/data1', '/data2']})
    solver.config['inparallel'] = False
    solution = solver.solve_field(filename,
                                  options={'add_path': '/home/user/gaia-indexes',
                                           'autoindex': True})

Additional Python inputs, like |ImageHDU|, |FrameData|, and ``(x, y, flux)`` arrays are supported trough hepler functions, described in the section bellow.

Type-specific helper functions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Helper functions to handle some common Python formats are provided to make easy solving the fields directly from these objects, without manual save of files.

The helpers are:

- `~astropop.astrometry.solve_astrometry_xy`: for solving lists of sources with image coordinates and fluxes extracted outside `astrometry.net`_.
- `~astropop.astrometry.solve_astrometry_framedata`: for solving |FrameData| objects.
- `~astropop.astrometry.solve_astrometry_image`: image file saved on disk.
- `~astropop.astrometry.solve_astrometry_hdu`: for solving |ImageHDU| objects.

Each function has its own specific arguments, but all of them have the same optional arguments. They are:

- ``command``: used to set the ``solve-field`` CLI path.
- ``options``: custom options used in the run.
- Any ``**kwargs`` keyword argument: passed to |run_command| function, like ``stdout``, ``stderr`` and log levels.

Supported ``solve-field`` options
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The |astropop| `astrometry.net`_ wrapper don't support all `solve-field` command-line interface options.

Also, we added the possibility to add any ``astrometry.cfg`` configuration file parameter to options. These configuration parameters will be read and popped out from options, a new config file will be created and used just for that run.

The supported options can be print in the screen using `~astropop.astrometry.AstrometrySolver.print_options_help` or `~astropop.astrometry.astrometrynet.get_options_help` funtions. They are described bellow.

.. autofunction:: astropop.astrometry.astrometrynet.get_options_help

Astrometry API
--------------

.. automodapi:: astropop.astrometry
    :no-inheritance-diagram:
