=================
pytest-instrument
=================

.. image:: https://img.shields.io/pypi/v/pytest-instrument.svg
    :target: https://pypi.org/project/pytest-instrument
    :alt: PyPI version

.. image:: https://img.shields.io/pypi/pyversions/pytest-instrument.svg
    :target: https://pypi.org/project/pytest-instrument
    :alt: Python versions

.. image:: https://img.shields.io/pypi/dm/pytest-instrument
    :target: https://pypistats.org/packages/pytest-instrument
    :alt: PyPI - Downloads

.. image:: https://circleci.com/gh/j19sch/pytest-instrument/tree/master.svg?style=svg
    :target: https://circleci.com/gh/j19sch/pytest-instrument/tree/master
    :alt: See Build Status on Circle CI

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
    :target: https://github.com/psf/black

.. image:: https://img.shields.io/github/license/mashape/apistatus.svg
    :target: https://github.com/j19sch/pytest-logfest/blob/master/LICENSE
    :alt: MIT license

A `pytest`_ plugin to instrument tests and write the resulting records to file

----


Features
--------

* a log file with one record for each setup/call/teardown of each test containing:

    * session id and record id (UUIDs generated by plugin)
    * node_id (pytest's `nodeid`)
    * when (setup, call or teardown)
    * outcome (passed, failed or skipped)
    * start, stop and duration
    * labels (`array`) and tags (`object`) via :code:`@pytest.mark.instrument()` decorator
    * fixtures
* hooks to edit labels, tags and fixtures before the record is written to file
* a logger to emit records with session id, record id, node id to the same log file


Requirements
------------

* Python 3.6 or higher
* pytest 5.1.0 or higher


Installation
------------

You can install "pytest-instrument" via `pip`_ from `PyPI`_::

    $ pip install pytest-instrument


Usage
-----

Run your tests with::

    $ pytest --instrument=json

An `./artifacts` directory will be created if it doesn't exist yet. For each `pytest` session one `.json` file
will be written to that directory. The format of the filename is :code:`%Y%m%dT%H%M%S_<first group of session id>.json`.

To display the contents of the `.log` file, use `jq`: :code:`jq . <filename>`. Or, if for instance you only want to
see the message object of each record: :code:`jq '{message: .message}' <filename>`

Output to regular log file
~~~~~~~~~~~~~~~~~~~~~~~~~~
If you want to output a regular log file, use `--instrument=log` (or `--instrument=json,log` if you want both).
Note that the regular log file only contains the time, log node name, log level, test node id and log message
of each log record.


Loggers - instr.log and instr.report
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The logger reporting on the outcome of pytest's setup, call and teardown is named "instr.report".

The logger emitting records to the same file as the "instr.report" logger is named "instr.log".
You can use this logger like this:

.. code-block:: python

    logger = logging.getLogger("instr.log")
    logger.info("log record of level info")

You can also create children of that logger, which will emit records to that same file:

.. code-block:: python

    sublogger = logging.getLogger("instr.log").getChild("sublogger")
    logger.warning("log record of level warning")

The session id and node id of the "instr.log" logger and its children are set automatically in the :code:`getChild()` method.
The node id is updated automatically via pytests's :code:`pytest_runtest_setup()` hook.


Labels and tags
~~~~~~~~~~~~~~~
You can add labels and tags to tests with the plugin's `mark` decorator::

@pytest.mark.instrument("a_label", my_tag="tagged")

All `args`, such as :code:`"a_label"`, are put in the json array `labels`.
All `kwargs`, such as :code:`my_tag="tagged"`, are put in the json object `tags`.


Hooks
~~~~~
- :code:`pytest_instrument_labels`: edit list of labels after they've been parsed by the plugin
- :code:`pytest_instrument_tags`: edit dictionary of tags after they've been parsed by the plugin
- :code:`pytest_instrument_fixtures`: edit list of fixtures after they've been parsed by the plugin


Changelog
---------

Consult the `changelog <https://github.com/j19sch/pytest-instrument/blob/master/CHANGELOG.rst>`_ for fixes and enhancements of each version.


Issues/Requests
---------------

Please use the `GitHub issue tracker <https://github.com/j19sch/pytest-instrument/issues>`_ to submit bugs or request features.


Contributing
------------
Contributions are welcome. Tests can be run with `tox`_, please ensure
`black`_ code formatting and good test coverage before you submit a pull request.


License
-------

Distributed under the terms of the `MIT`_ license, "pytest-instrument" is free and open source software


Acknowledgements
----------------
Thank you to `Maaret Pyhäjärvi <https://maaretp.com>`_ for inspiring this plugin by sharing the story of how her team
started instrumenting tests at `TestCraftCamp`_ 2019.

Thank you to `Tony S Yu <https://github.com/tonysyu>`_ for backporting the :code:`stacklevel` argument from Python 3.8 logging.

This `pytest`_ plugin was generated with `Cookiecutter`_ along with `@hackebrot`_'s `cookiecutter-pytest-plugin`_ template.


.. _`Cookiecutter`: https://github.com/audreyr/cookiecutter
.. _`@hackebrot`: https://github.com/hackebrot
.. _`MIT`: http://opensource.org/licenses/MIT
.. _`BSD-3`: http://opensource.org/licenses/BSD-3-Clause
.. _`GNU GPL v3.0`: http://www.gnu.org/licenses/gpl-3.0.txt
.. _`Apache Software License 2.0`: http://www.apache.org/licenses/LICENSE-2.0
.. _`cookiecutter-pytest-plugin`: https://github.com/pytest-dev/cookiecutter-pytest-plugin
.. _`pytest`: https://github.com/pytest-dev/pytest
.. _`tox`: https://tox.readthedocs.io/en/latest/
.. _`pip`: https://pypi.org/project/pip/
.. _`PyPI`: https://pypi.org/project
.. _`black`: https://github.com/psf/black
.. _`TestCraftCamp`: https://testcraftcamp.nl/