Contributing
============

Here’s how you can get started if you want to help:

1. Fork the `repository <https://github.com/alexmojaki/birdseye>`_, and clone your fork.

2. Run ::

        pip install -e '.[tests]'

   in the root of the repo. This will install it
   using a symlink such that changes to the code immediately affect the
   installed library. In other words, you can edit a ``.py`` file in your copy of birdseye, then debug a
   separate program, and the results of your edit will be
   visible. This makes development and testing straightforward.
   The `[tests]` part installs extra dependencies needed for development.

   If you have one or more other projects that you’re working on where birdseye
   might be useful for development and debugging, install birdseye into
   the interpreter (so the virtualenv if there is one) used for that
   project.
3. Try using birdseye for a bit, ideally in a real
   scenario. Get a feel for what using it is like. Note any
   bugs it has or features you’d like added. `Create an issue`_ where
   appropriate or `ask questions on the gitter chatroom`_.
4. Pick an issue that interests you and that you’d like to work on,
   either one that you created or an existing one. An issue with the
   `easy label`_ might be a good place to start.
5. Read through the source code overview below to get an idea of how it all
   works.
6. :ref:`Run the tests <testing>` before making any changes just to verify that it all
   works on your computer.
7. Dive in and start coding! I’ve tried to make the code readable and
   well documented. Don’t hesitate to ask any questions on `gitter`_. If
   you installed correctly, you should find that changes you make to the
   code are reflected immediately when you run it.
8. Once you’re happy with your changes, `make a pull request`_.

.. _here: https://github.com/alexmojaki/birdseye#usage-and-features
.. _Create an issue: https://github.com/alexmojaki/birdseye/issues/new
.. _ask questions on the gitter chatroom: https://gitter.im/python_birdseye/Lobby
.. _easy label: https://github.com/alexmojaki/birdseye/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy
.. _gitter: https://gitter.im/python_birdseye/Lobby
.. _make a pull request: http://scholarslab.org/research-and-development/forking-fetching-pushing-pulling/

.. _source_overview:

Source code overview
--------------------

This is a brief and rough overview of how the core of this library
works, to assist anyone reading the source code for the first time.

See also ':doc:`how_it_works`' for a higher level view of the concepts
apart from the actual source code.

Useful background knowledge
~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. The ``ast`` module of the Python standard library, for parsing,
   traversing, and modifying source code. You don’t need to know the
   details of this in advance, but you should know that `this`_ is a
   great resource for learning about it if necessary, as the official
   documentation is not very helpful.
2. **Code objects**: every function in Python has a ``__code__``
   attribute pointing to a special internal code object. This contains
   the raw instructions for executing the function. A locally defined
   function (i.e. a ``def`` inside a ``def``) can have multiple separate
   instances, but they all share the same code object, so this is the
   key used for storing/finding metadata for functions.
3. **Frame objects**: sometimes referred to as the frame of execution,
   this is another special python object that exists for every function
   call that is currently running. It contains local variables, the code
   object that is being run, a pointer to the previous frame on the
   stack, and more. It’s used as the key for data for the current call.

.. _this: https://greentreesnakes.readthedocs.io/en/latest/index.html

When a function is decorated [``BirdsEye.trace_function``]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. [``TracedFile.__init__``] The entire file is parsed using the
   standard ``ast`` module. The tree is modified so that every
   expression is wrapped in two function calls
   [``_NodeVisitor.visit_expr``] and every statement is wrapped in a
   ``with`` block [``_NodeVisitor.visit_stmt``].
2. [``BirdsEye.compile``] An ``ASTTokens`` object is created so that the
   positions of AST nodes in the source code are known.
3. The modified tree is compiled into a code object. Inside this we find
   the code object corresponding to the function being traced.
4. The ``__globals__`` of the function are updated to contain references
   to the functions that were inserted into the tree in step 1.
5. A new function object is created that’s a copy of the original
   function except with the new code object.
6. An HTML document is constructed where the expressions and statements
   of the source are wrapped in ``<span>``\ s.
7. A ``Function`` row is stored in the database containing the HTML and
   other metadata about the function.
8. A ``CodeInfo`` object is kept in memory to keep track of metadata
   about the function.

When a function runs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. When the first statement of the function runs, the tracer notices
   that it’s the first statement and calls
   ``TreeTracerBase._enter_call``. A new ``FrameInfo`` is created and
   associated with the current frame.
2. [``BirdsEye.enter_call``] The arguments to the function are noted and
   stored in the ``FrameInfo``. If the parent frame is also being
   traced, this is noted as an inner call of the parent call.
3. A ``_StmtContext`` is created for every statement in the function.
   These lead to calling ``BirdsEye.before_stmt`` and
   ``BirdsEye.after_stmt``.
4. For every expression in the function call, ``BirdsEye.before_expr``
   and ``BirdsEye.after_expr`` are called. The values of expressions are
   expanded in ``NodeValue.expression`` and stored in an ``Iteration``,
   belonging either directly to the current ``FrameInfo`` (if this is at
   the top level of the function) or indirectly via an ``IterationList``
   (if this is inside a loop).
5. When the function call ends, ``BirdsEye.exit_call`` is called. The
   data from the current ``FrameInfo`` is gathered and stored in the
   database in a new ``Call`` row.

.. _testing:

Testing
-------

Run ``./misc/test.sh`` to run all
tests with a single Python interpreter. You will need to have
`chromedriver` installed.

Run `tox`_ (``pip install tox``) to run tests on all supported
versions of Python. You must install the interpreters
separately yourself.

``test_against_files``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

One of the tests involves comparing data produced by the debugger to the
contents of golden JSON files. This produces massive diffs when the
tests fail. To read these I suggest redirecting or copying the output to
a file and then doing a regex search for ``^[+-]`` to find the
actual differences.

If you’re satisfied that the code is doing the correct thing and the
golden files need to be updated, set the environment variable ``FIX_TESTS=1``,
then rerun the tests. This will write
to the files instead of comparing to them. Since there are files for
each version of python, you will need to run the tests on all supported
interpreters, so tox is recommended.

Browser screenshots for test failures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``test_interface.py`` runs a test using selenium and headless Chrome. If it
fails, it produces a file ``error_screenshot.png`` which is helpful for
debugging the failure locally.

.. _tox: https://tox.readthedocs.io/en/latest/


Linting
-------

None of this is strictly required, but may help spot errors to improve
the development process.

Linting Python using mypy (type warnings)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The code has type hints so that ``mypy`` can be used on it, but there
are many false warnings for various reasons. To ignore these, use the
``misc/mypy_filter.py`` script. The docstring explains in more detail.

Linting JavaScript using gulp and eslint
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Install ``npm``
2. Change to the ``gulp`` directory.
3. Run ``install-deps.sh``.
4. Run ``gulp``. This will lint the JavaScript continuously, checking
   every time the files change.
