User Guide
==========
This page summarizes how to use the main features of pyinaturalist.

.. contents::
    :local:  
    :depth: 1

Installation
------------
.. admonition:: Python version compatibility
    :class: toggle

    pyinaturalist currently requires **python 3.6+**. If you need to use an older version
    of python, here are the last compatible versions of pyinaturalist:

    * **python 2.7:** pyinaturalist 0.1
    * **python 3.4:** pyinaturalist 0.10
    * **python 3.5:** pyinaturalist 0.11
    * **python 3.6:** still supported, but expected to be dropped in a future release

Install the latest stable version with pip::

    pip install pyinaturalist

Or with Conda, if you prefer::

    conda install -c conda-forge pyinaturalist

If you would like to use the latest development (pre-release) version::

    pip install --pre pyinaturalist

See :ref:`contributing` for details on setup for local development.


Imports
-------
All of the main features can be imported from the top-level ``pyinaturalist`` namespace:

    >>> from pyinaturalist import Taxon, get_observations, pprint  # etc.

Or to just import everything (convenient for scripts and notebooks, but less so for applications):

    >>> from pyinaturalist import *

Requests
--------
Requests generally follow the same format as the `API <https://api.inaturalist.org/v1>`_
and `search URLs <https://forum.inaturalist.org/t/how-to-use-inaturalists-search-urls-wiki>`_.

For example, if you wanted to search observations by user, the following API request::

    https://api.inaturalist.org/v1/observations?user_id=tiwane%2Cjdmore

And search URL::

    https://www.inaturalist.org/observations?user_id=tiwane,jdmore

Are equivalent to the following pyinaturalist search:

    >>> get_observations(user_id=['tiwane', 'jdmore'])

There are some optional conveniences you can use, for example:

* Python lists instead of comma-separated strings
* Python booleans instead of JS-style boolean strings or 1/0
* Python file-like objects or file paths for photo and sound uploads
* Python :py:class:`~datetime.date` and :py:class:`~datetime.datetime` objects instead of date/time strings
* Some simplified data formats for create and update requests
* Simplified pagination
* Validation for multiple-choice parameters (for example, ``quality_grade``)

And more, depending on the function.
See the :ref:`reference` section for a complete list of functions available.

Responses
---------
API responses are returned as JSON, with some python type conversions applied (similar to the request
type conversions mentioned above). Example response data is shown in the documentation for each request
function. For example, here's what an observation response looks like:

.. admonition:: Observation response JSON
    :class: toggle

        .. literalinclude:: sample_data/get_observations_node.py

Previewing Responses
^^^^^^^^^^^^^^^^^^^^
These responses can contain large amounts of response attributes, making it somewhat cumbersome if you
just want to quickly preview results (for example, in a Jupyter notebook or REPL).
For that purpose, a handy :py:func:`~pyinaturalist.formatters.pprint` function is included that will
format and print responses and model objects as a condensed, colorized table.

**Examples:**

.. admonition:: Example observation table
    :class: toggle

    >>> from pyinaturalist import get_observations, pprint
    >>> observations = get_observations(user_id='niconoe', per_page=5)
    >>> pprint(observations)
    ID         Taxon ID   Taxon                                                  Observed on    User      Location                        
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 
    82974075   61546      Species: Nemophora degeerella (Yellow-barred Longhorn) Jun 14, 2021   niconoe   1428 Braine-l'Alleud, Belgique                                                                                  
    82827577   48201      Family: Scarabaeidae (Scarabs)                         Jun 13, 2021   niconoe   1428 Braine-l'Alleud, Belgique  
    82826778   48201      Family: Scarabaeidae (Scarabs)                         Jun 13, 2021   niconoe   1428 Braine-l'Alleud, Belgique  
    82696354   209660     Species: Chrysolina americana (Rosemary Beetle)        Jun 12, 2021   niconoe   1420 Braine-l'Alleud, Belgique  
    82696334   472617     Species: Tomocerus vulgaris                            Jun 07, 2021   niconoe   1428 Braine-l'Alleud, Belgique  
                                 

.. admonition:: Example place table
    :class: toggle

    >>> from pyinaturalist import get_places, pprint
    >>> places = get_places_autocomplete('Vale')
    >>> pprint(places)
     ID       Latitude    Longitude   Name                  Category   URL                                        
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 
    96877      49.5189     -2.5190   Vale                             https://www.inaturalist.org/places/96877   
    21951     -16.8960    -40.8349   Fronteira dos Vales              https://www.inaturalist.org/places/21951   
    23663      -6.3677    -41.8001   Valença do Piauí                 https://www.inaturalist.org/places/23663   
    24222     -27.2220    -53.6338   Pinheirinho do Vale              https://www.inaturalist.org/places/24222   
    24374     -29.8309    -52.1121   Vale Verde                       https://www.inaturalist.org/places/24374   
    24442     -10.3841    -62.0939   Vale do Paraíso                  https://www.inaturalist.org/places/24442   
    103902     44.7355     27.5412   Valea Ciorii                     https://www.inaturalist.org/places/103902  
    103905     44.7529     26.8481   Valea Macrisului                 https://www.inaturalist.org/places/103905  
    105015     44.6805     24.0224   Valea Mare                       https://www.inaturalist.org/places/105015  
    104268     46.7917     27.0905   Valea Ursului                    https://www.inaturalist.org/places/104268        


.. admonition:: Example place table (with terminal colors)
    :class: toggle

    .. figure:: images/pprint_table.png


Models
------
Data models (:py:mod:`pyinaturalist.models`) are included for all API response types. These allow
working with typed python objects instead of raw JSON. These are not used by default in the API query
functions, but you can easily use them as follows:

.. admonition:: Convert observation response JSON to Observation objects
    :class: toggle

    >>> from pyinaturalist import Observation, get_observations
    >>> response = get_observations(user_id='my_username)
    >>> observations = Observation.from_json_list(response)

In a future release, these models will be fully integrated with the API query functions.

Pagination
----------
Most endpoints support pagination, using the parameters:
    * ``page``: Page number to get
    * ``per_page``: Number of results to get per page
    * ``count_only=True``: This is just a shortcut for ``per_page=0``, which will return only the
      total number of results, not the results themselves.

The default and maximum ``per_page`` values vary by endpoint, but it's 200 for most endpoints.

To get all pages of results and combine them into a single response, use ``page='all'``.
Note that this replaces the ``get_all_*()`` functions from pyinaturalist<=0.12.

.. _auth:

Authentication
--------------
For any endpoints that create, update, or delete data, you will need to authenticate using an
OAuth2 access token. This requires both your iNaturalist username and password, and separate
"application" credentials.

.. note::
    Read-only requests generally don't require authentication; however, if you want to access
    private data visible only to your user (for example, obscured or private coordinates),
    you will need to use an access token.

**Summary:**

1. Create an iNaturalist application
2. Use :py:func:`.get_access_token` with your user + application credentials to get an access token
3. Pass that access token to any API request function that uses it

Creating an Application
^^^^^^^^^^^^^^^^^^^^^^^
.. admonition:: Why do I need to create an application?
    :class: toggle

    iNaturalist uses OAuth2, which provides several different methods (or "flows") to access the site.
    For example, on the `login page <https://www.inaturalist.org/login>`_, you have the option of logging
    in with a username/password, or with an external provider (Google, Facebook, etc.):

    .. figure:: images/inat-user-login.png
        :alt: Login form

    Outside of iNaturalist.org, anything else that uses the API to create or modify data is considered
    an "application," even if you're just running some scripts on your own computer.
    
    See `iNaturalist documentation <https://www.inaturalist.org/pages/api+reference#auth>`_
    for more details on authentication.

First, go to `New Application <https://www.inaturalist.org/oauth/applications/new>`_ and fill out the
following pieces of information:

* **Name:** Any name you want to come up with. For example, if this is associated with a GitHub repo,
  you can use your repo name.
* **Description:** A brief description of what you'll be using this for. For example,
  *"Data access for my own observations"*.
* **Confidential:** ✔️ This should be checked.
* **URL and Redirect URI:** Just enter the URL to your GitHub repo, if you have one; otherwise any
  placeholder like "https://www.inaturalist.org" will work.

.. figure:: images/inat-new-application.png
    :alt: New Application form

You should then see a screen like this, which will show your new application ID and secret. These will
only be shown once, so save them somewhere secure, preferably in a password manager.

.. figure:: images/inat-new-application-complete.png
    :alt: Completed application form

Basic Usage
^^^^^^^^^^^
There are a few different ways you can pass your credentials to iNaturalist. First, you can pass
them as keyword arguments to :py:func:`.get_access_token`:

    >>> from pyinaturalist import get_access_token
    >>> access_token = get_access_token(
    >>>     username='my_inaturalist_username',  # Username you use to login to iNaturalist.org
    >>>     password='my_inaturalist_password',  # Password you use to login to iNaturalist.org
    >>>     app_id='33f27dc63bdf27f4ca6cd95dd',  # OAuth2 application ID 
    >>>     app_secret='bbce628be722bfe2abde4',  # OAuth2 application secret
    >>> )

Environment Variables
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You can also provide credentials via environment variables instead of arguments. The
environment variable names are the keyword arguments in uppercase, prefixed with ``INAT_``:

* ``INAT_USERNAME``
* ``INAT_PASSWORD``
* ``INAT_APP_ID``
* ``INAT_APP_SECRET``

**Examples:**

.. admonition:: Set environment variables in python:
    :class: toggle

    >>> import os
    >>> os.environ['INAT_USERNAME'] = 'my_inaturalist_username'
    >>> os.environ['INAT_PASSWORD'] = 'my_inaturalist_password'
    >>> os.environ['INAT_APP_ID'] = '33f27dc63bdf27f4ca6cd95df'
    >>> os.environ['INAT_APP_SECRET'] = 'bbce628be722bfe283de4'

.. admonition:: Set environment variables in a POSIX shell (bash, etc.):
    :class: toggle

    .. code-block:: bash

        export INAT_USERNAME="my_inaturalist_username"
        export INAT_PASSWORD="my_inaturalist_password"
        export INAT_APP_ID="33f27dc63bdf27f4ca6cd95df"
        export INAT_APP_SECRET="bbce628be722bfe283de4"

.. admonition:: Set environment variables in a Windows shell:
    :class: toggle

    .. code-block:: bat

        set INAT_USERNAME="my_inaturalist_username"
        set INAT_PASSWORD="my_inaturalist_password"
        set INAT_APP_ID="33f27dc63bdf27f4ca6cd95df"
        set INAT_APP_SECRET="bbce628be722bfe283de4"

.. admonition:: Set environment variables in PowerShell:
    :class: toggle

    .. code-block:: powershell

        $Env:INAT_USERNAME="my_inaturalist_username"
        $Env:INAT_PASSWORD="my_inaturalist_password"
        $Env:INAT_APP_ID="33f27dc63bdf27f4ca6cd95df"
        $Env:INAT_APP_SECRET="bbce628be722bfe283de4"

Note that in any shell, these environment variables will only be set for your current shell
session. I.e., you can't set them in one terminal and then access them in another.

Keyring Integration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To handle your credentials more securely, you can store them in your system keyring.
You could manually store and retrieve them with a utility like
`secret-tool <https://manpages.ubuntu.com/manpages/xenial/man1/secret-tool.1.html>`_
and place them in environment variables as described above, but there is a much simpler option.

Direct keyring integration is provided via `python keyring <https://github.com/jaraco/keyring>`_. Most common keyring bakcends are supported, including:

* macOS `Keychain
  <https://en.wikipedia.org/wiki/Keychain_%28software%29>`_
* Freedesktop `Secret Service
  <http://standards.freedesktop.org/secret-service/>`_
* KDE `KWallet <https://en.wikipedia.org/wiki/KWallet>`_
* `Windows Credential Locker
  <https://docs.microsoft.com/en-us/windows/uwp/security/credential-locker>`_

To store your credentials in the keyring, run :py:func:`.set_keyring_credentials`:

    >>> from pyinaturalist.auth import set_keyring_credentials
    >>> set_keyring_credentials(
    >>>     username='my_inaturalist_username',
    >>>     password='my_inaturalist_password',
    >>>     app_id='33f27dc63bdf27f4ca6cd95df',
    >>>     app_secret='bbce628be722bfe283de4',
    >>> )

Afterward, you can call :py:func:`.get_access_token` without any arguments, and your credentials
will be retrieved from the keyring. You do not need to run :py:func:`.set_keyring_credentials`
again unless you change your iNaturalist password.

Password Manager Integration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Keyring integration can be taken a step further by managing your keyring with a password
manager. This has the advantage of keeping your credentials in one place that can be synced
across multiple machines. `KeePassXC <https://keepassxc.org/>`_ offers this feature for
macOS and Linux systems. See this guide for setup info:
`KeepassXC and secret service, a small walk-through
<https://avaldes.co/2020/01/28/secret-service-keepassxc.html>`_.

.. figure:: images/password_manager_keying.png

   Credentials storage with keyring + KeePassXC


Dry-run mode
------------
While developing and testing, it can be useful to temporarily mock out HTTP requests, especially
requests that add, modify, or delete real data. Pyinaturalist has some settings to make this easier.

Dry-run all requests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To enable dry-run mode, set the ``DRY_RUN_ENABLED`` variable. When set, requests will not be sent
but will be logged instead:

    >>> import logging
    >>> import pyinaturalist
    >>>
    >>> # Enable at least INFO-level logging
    >>> logging.basicConfig(level='INFO')
    >>>
    >>> pyinaturalist.DRY_RUN_ENABLED = True
    >>> get_taxa(q='warbler', locale=1)
    {'results': [], 'total_results': 0}
    INFO:pyinaturalist.api_requests:Request: GET, https://api.inaturalist.org/v1/taxa,
        params={'q': 'warbler', 'locale': 1},
        headers={'Accept': 'application/json', 'User-Agent': 'Pyinaturalist/0.9.1'}

You can also set this as an environment variable (case-insensitive):

.. code-block:: bash

    $ export DRY_RUN_ENABLED=true
    $ python my_script.py

Dry-run only write requests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you would like to send real ``GET`` requests but mock out any requests that modify data
(``POST``, ``PUT``, ``DELETE``, etc.), you can use the ``DRY_RUN_WRITE_ONLY`` variable
instead:

    >>> pyinaturalist.DRY_RUN_WRITE_ONLY = True
    >>> # Also works as an environment variable
    >>> import os
    >>> os.environ["DRY_RUN_WRITE_ONLY"] = 'True'


User Agent
----------
While not mandatory, it's good practice to include a `user-agent <https://en.wikipedia.org/wiki/User_agent>`_ in
your API calls. This field can be either something that identifies the project or its contact person.

You can either set this globally:

    >>> import pyinaturalist
    >>>
    >>> pyinaturalist.user_agent = "MyCoolAndroidApp/2.0 (using Pyinaturalist)"
    >>> # From now on, all API calls will use this user-agent.


To set this for individual requests, all API functions accept an optional ``user_agent`` parameter:

    >>> from pyinaturalist import get_observation
    >>> get_observation(observation_id=16227955, user_agent='Jane Doe <jane.doe@gmail.com>')

If not configured, ``Pyinaturalist/<VERSION>`` will be used.


API Recommended Practices
-------------------------
See `API Recommended Practices <https://www.inaturalist.org/pages/api+recommended+practices>`_
on iNaturalist for more general usage information and notes.
