Metadata-Version: 2.4
Name: pytest-fly
Version: 0.1.0
Summary: pytest runner and observer
Project-URL: Home, https://github.com/jamesabel/pytest-fly
Project-URL: Repository, https://github.com/jamesabel/pytest-fly
Project-URL: Documentation, https://github.com/jamesabel/pytest-fly#readme
Author-email: James Abel <j@abel.co>
Maintainer-email: James Abel <j@abel.co>
License: 
        The MIT License (MIT)
        
        Copyright (c) 2024 James Abel
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in
        all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
        THE SOFTWARE.
License-File: LICENSE
Keywords: observer,pytest,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.12
Requires-Dist: attrs
Requires-Dist: balsa
Requires-Dist: coverage
Requires-Dist: hashy
Requires-Dist: humanize
Requires-Dist: ismain
Requires-Dist: msqlite
Requires-Dist: numpy
Requires-Dist: platformdirs
Requires-Dist: pref
Requires-Dist: psutil
Requires-Dist: py-cpuinfo
Requires-Dist: pyside6
Requires-Dist: pytest
Requires-Dist: tobool
Requires-Dist: typeguard
Requires-Dist: uuid6
Requires-Dist: watchdog
Description-Content-Type: text/markdown

# pytest-fly

`pytest-fly` aides the development, debug, and execution of complex code bases and test suites.

## Features of `pytest-fly`

- Real-time monitor test execution in a GUI. Displays what tests are currently running, what tests have completed,
and what tests have failed. A time-based graph provides a visual representation of test progress.
- Resumable test execution. Only runs tests that have not yet been run or have failed.
- Graceful interruption of test execution. Allows the user to stop the test suite and then resume where it left off.
- Checks the code under test and restarts the test run from scratch if the code has changed.
- Optimally run tests in parallel. Monitors system resources and dynamically adjusts the number of 
parallel tests accordingly.
- Provides an estimate of the time remaining for the test suite to complete. Uses prior test run times to estimate 
the time remaining.
- Optional code coverage. Code coverage can also run tests in parallel.
- Run specific tests as a singleton via pytest markers (pytest.marker.singleton).

## Installation

You can install `pytest-fly` via `pip` from `PyPI`:

```
    pip install pytest-fly
```

## Parallelism

By default, `pytest-fly` executes *modules* (.py files) in parallel. 

Functions *inside* a module are executed serially with respect to each other. No parallelism is performed for 
functions inside a module. For example, if a set of tests use a shared resource that does not support concurrent 
access, putting those tests in the same module ensures the tests do not conflict.

Functions can optionally be run as a singleton via the `pytest.mark.singleton` marker. No other tests are run 
at the same time `singleton` functions are run.

In `pytest` terms, each module will be run as a separate `session`. Therefore, a pytest fixture with a `session` scope 
will actually be executed multiple times, once for each modulee.

Note that test concurrency in `pytest-fly` is different from `pytest-xdist`. `group-by` in `pytest-xdist` is
analogous to putting the tests in the same module in `pytest-fly`.

## Test Scheduling

`pytest-fly` attempts to order tests to make the best use of the test developer's time. The goal is for the more 
actionable and insightful information to appear earlier. In general, `pytest-fly` takes the following into account 
when scheduling tests:

1. Failed tests are re-run first. This is beneficial when a test developer is fixing failed tests. Note that if a 
test is *expected* to fail, it should be temporarily marked as such to avoid spending time re-running a test that 
will merely fail.
2. Tests with a higher coverage over time (i.e., lines/second) are run earlier. This way, if there is a problem in the 
code, it is more likely to be found earlier in the test run.
3. Tests are scheduled for maximum parallelism. Long-running tests are started earlier to minimize overall runtime. 
Also, tests that consume large amounts of particular resources (e.g., memory, file system, etc.) tend to be paired 
with other tests that do not use as much of those resources.
4. `singleton` tests are run last. This is to attempt to get higher coverage earlier in the test run.
