# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['waterfalls']

package_data = \
{'': ['*']}

install_requires = \
['matplotlib>=3.5.1,<4.0.0']

entry_points = \
{'console_scripts': ['waterfalls = waterfalls.viewer:main']}

setup_kwargs = {
    'name': 'waterfalls',
    'version': '0.2.1',
    'description': 'Visualizes duration and CPU usage of blocks of code to let you see bottlenecks in your applications.',
    'long_description': '[![PyPI](https://img.shields.io/pypi/v/waterfalls?style=flat-square)](https://pypi.org/project/waterfalls/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/waterfalls?style=flat-square)](https://pypi.org/project/waterfalls/)\n[![GitHub Workflow Status](https://img.shields.io/github/workflow/status/glenvorel/waterfalls/Lint%20and%20test?style=flat-square)](https://github.com/glenvorel/waterfalls/actions/workflows/lint_test.yml)\n\n# Waterfalls\n\nVisualizes duration and CPU usage of blocks of code to let you see bottlenecks in your applications.\n\nTo use Waterfalls, you mark the blocks of code in your application that you want to time and then run the application as you normally do.\nWaterfalls runs in the background and collects the data.\nWhen your application exits, Waterfalls automatically generates and saves a report.\n\nTo see the report, you use the `waterfalls` command which shows the timings as a waterfall diagram.\nIt is much easier to understand the application flow when you _see_ it, rather than trying to decipher timestamps printed to the console.\n\nWaterfalls handles also applications that use threading and multiprocessing.\n\nThe tool is very lightweight and has a negligible impact on the main program.\nAll slower tasks (e.g., saving reports to disk) are delayed until _after_ the main program finishes.\n\n## Basic example\n\n```python\nimport tarfile\nimport urllib.request\n\nfrom waterfalls import Timer\n\n\nURLS = [\n    "https://www.python.org/",\n    "https://pypi.org/",\n    "https://github.com/",\n]\n\n\nfor i, url in enumerate(URLS):\n\n    with Timer("Download site"):\n        with urllib.request.urlopen(url) as conn:\n            html = conn.read()\n\n    with Timer("Save as HTML"):\n        with open(f"{i}.html", "wb") as file:\n            file.write(html)\n\n    with Timer("Save as TAR"):\n        with tarfile.open(f"{i}-xz.tar", "w:xz") as tar:\n            tar.add(f"{i}.html")\n```\n\nThe program downloads sites from a list of URLs, saves each to an HTML file, and finally compresses the HTML file into a TAR file.\nWe use `waterfalls.Timer` to define each of the 3 logical steps.\n\nWhen we run the program, Waterfalls automatically saves a `waterfalls.json` file to the current working directory.\n\nWe can now type the `waterfalls` command which will display it as an interactive diagram.\n\n![](https://raw.githubusercontent.com/glenvorel/waterfalls/main/assets/diagram_download_sequential.png)\n\nLooking at the diagram, we immediately _see_ that the program spends a lot of time downloading each site.\nDuring these periods, nothing else is happening, the CPU usage is very low and the program is generally wasting time.\nWe can certainly improve on that.\nIn the section [**Advanced examples**](#advanced-examples), we will refactor the program to take advantage of multiple threads and use Waterfalls to analyze the refactored program.\n\n## Installation\n\nWaterfalls requires Python 3.8+ to run.\n\n```bash\npip install waterfalls\n```\n\n## Usage\n\n### Step 1/2: generating records\n\nMark the sections of the code you want to measure.\n\nFirst, create one or more _timers_. A _timer_ typically represents one logical activity, for example, `Download site`.\nThe _timer_ corresponds to one row in the diagram. It gives the row its name.\n\nOne _timer_ can have one or more _blocks_, for example, one for each site that is being downloaded.\nEach _block_ must be started (`start()`) and stopped (`stop()`).\nAll _blocks_ will be placed on the _timer_ row from left to right, showing exactly when they were started and stopped.\n\n```python\nfrom waterfalls import Timer\n\nt = Timer("Download site")\n\nt.start()\n# Download site A\nt.stop()\n\nt.start()\n# Download site B\nt.stop()\n```\n\nFor convenience, `Timer` can also be used as a context manager or as a decorator.\n\n```python\nwith Timer("Download site"):\n    # Download site C\n```\n\n```python\n@Timer("Download site")\ndef download_function():\n    # Download site D\n```\n\n#### Add informative text to timing blocks\n\nSometimes it can be useful to add custom _text_ to the timing _blocks_, for example, to record the concrete value that is being processed.\nThis _text_ will then be displayed in the diagram on top of its corresponding _block_.\n\nThe _text_ can be defined when calling the `start()` method.\n\n```python\nt = Timer("Download site")\n\nt.start(text="www.python.org")\n# Download www.python.org\nt.stop()\n\nt.start(text="pypi.org")\n# Download pypi.org\nt.stop()\n```\n\nIt can also be defined when using `Timer` as a context manager or decorator.\n\n```python\nwith Timer("Download site", text="www.python.org"):\n    # Download www.python.org\n\nwith Timer("Download site", text="pypi.org"):\n    # Download pypi.org\n```\n\n```python\n@Timer("Download site", text="www.python.org")\ndef download_function():\n    # Download www.python.org\n\n@Timer("Download site", text="pypi.org")\ndef download_function():\n    # Download pypi.org\n```\n\nSometimes the _text_ isn\'t known at the moment of the _block\'s_ start. For example, it may be obtained during the _block\'s_ execution.\nIn this case, you can define it in the `stop()` method.\n\n```python\nt = Timer("Download site")\nt.start()\n# Download github.com\n# Save response size to `bytes_len`\nt.stop(text=bytes_len)\n```\n\nWhen using `Timer` as a context manager, _text_ can also be set after initiation - using an instance attribute.\n\n```python\nwith Timer("Download site") as t:\n    # Download github.com\n    # Save response size to `bytes_len`\n    t.text = bytes_len\n```\n\n> Note that while `text` can be set at nearly every step of the _block\'s_ lifecycle, only the last value will be saved in the report.\n\n#### Change the reports directory\n\nBy default, the report file will be saved to the current working directory.\nTo choose a different directory, you can set `Timer.directory` anywhere in the code.\n\n```python\nTimer.directory = "/path/to/reports/"\n```\n\nYou can also use the `WATERFALLS_DIRECTORY` environment variable.\n\n```bash\nexport WATERFALLS_DIRECTORY=/path/to/reports/\n````\n\n### Step 2/2: viewing diagrams\n\nTo see the diagram, use the `waterfalls` command. By default, it will look for report files in the current working directory.\nIf you want to search in another location, you can specify the directory.\n\n```bash\nwaterfalls "/path/to/reports/"\n```\n\nThe command supports the following arguments.\n\nPositional arguments:\n- `directory`: Directory containing report file(s) generated by waterfalls.Timer. By default, the current working directory is used.\n\nOptional arguments:\n- `-u`, `--unit` `{nsec,usec,msec,sec,min,hour}`: Specifies time unit. By default, the time unit is determined automatically.\n- `-t`, `--thread_id`: Shows thread ID next to each timer. By default, thread ID is hidden and only shown to distinguish overlapping blocks belonging to the same timer.\n- `-l`, `--lines`: Shows horizontal lines in the graph.\n- `-i`, `--image`: Saves diagram as an image (waterfalls.svg) to the reports directory instead of launching interactive window.\n\n## Advanced examples\n\n### Multiple threads\n\nWhen we timed the program in the section [**Basic example**](#basic-example), we noticed that it spends a lot of time downloading the websites and the CPU usage is very low.\nThis type of program should benefit from using multiple threads so we will refactor it and use [`ThreadPoolExecutor`](https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor).\n\nWe will instruct Waterfalls to save the report file to the `./thread_pool_records/` directory.\n\nWe will also pass the `Download site` _timer_ a _text_ containing the domain that is being downloaded so the diagram becomes even clearer.\n\n```python\nimport concurrent.futures\nimport tarfile\nfrom urllib.parse import urlparse\nimport urllib.request\n\nfrom waterfalls import Timer\n\n\nURLS = [\n    "https://www.python.org/",\n    "https://pypi.org/",\n    "https://github.com/",\n]\n\n\nTimer.directory = "./thread_pool_records/"\n\n\ndef save_site(i, url):\n    netloc = urlparse(url).netloc\n\n    with Timer("Download site", text=netloc):\n        with urllib.request.urlopen(url) as conn:\n            html = conn.read()\n\n    with Timer("Save as HTML"):\n        with open(f"{i}.html", "wb") as file:\n            file.write(html)\n\n    with Timer("Save as TAR"):\n        with tarfile.open(f"{i}-xz.tar", "w:xz") as tar:\n            tar.add(f"{i}.html")\n\n\nwith concurrent.futures.ThreadPoolExecutor(max_workers=len(URLS)) as executor:\n    executor.map(save_site, range(len(URLS)), URLS)\n```\n\nAfter we run the program, there will be a new directory (`thread_pool_records`) in our current working directory.\nWe can now launch the Waterfalls viewer. We will also use the `--thread_id` flag to tell Waterfalls to show the thread ID belonging to each _timer_.\n\n```bash\nwaterfalls "./thread_pool_records/" --thread_id\n```\n\n![](https://raw.githubusercontent.com/glenvorel/waterfalls/main/assets/diagram_download_concurrent.png)\n\nMain observations:\n- We can _see_ that all sites are being downloaded concurrently, reducing the total time needed to run the program.\n- It takes around 350 ms to download, save and compress all 3 sites. (It was over 520 ms for the single-threaded version.)\n- `pypi.org` is the quickest site to be downloaded, `github.com` is the slowest.\n- Creating the TAR files is very CPU-intensive. However, multiprocessing would not help here since the TARs are created at different times and do not compete for the CPU.\n\n### Multiple processes\n\nWe have a program that determines if a number from a list is prime or not. We will add `waterfalls.Timer` called `Determine prime` to time the program.\n\n```python\nimport math\n\nfrom waterfalls import Timer\n\n\nNUMBERS = [112272535095293, 112582705942171, 115280095190773, 115797848077099, 1099726899285419]\n\n\nTimer.directory = "./prime_records/"\n\n\nfor number in NUMBERS:\n    with Timer("Determine prime") as t:\n        not_prime = False\n\n        if number > 1:\n            for i in range(2, math.floor(math.sqrt(number) + 1)):\n                if (number % i) == 0:\n                    not_prime = True\n                    break\n\n        t.text = f"{number}\\nnot prime" if not_prime else f"{number}\\nis prime"\n```\n\nAfter the program finishes, we will launch the viewer.\n\n```bash\nwaterfalls "./prime_records/"\n```\n\n![](https://raw.githubusercontent.com/glenvorel/waterfalls/main/assets/diagram_prime_sequential.png)\n\nWe can immediately _see_ that determining whether a number is prime or not uses almost 100% CPU.\nBut the computer has multiple CPU cores so we can take advantage of multiprocessing to speed up the program.\nWe will use the [`ProcessPoolExecutor`](https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor).\nWe will also change the reports directory to `./prime_multiprocessing_records/`.\n\n```python\nimport concurrent.futures\nimport math\n\nfrom waterfalls import Timer\n\n\nNUMBERS = [112272535095293, 112582705942171, 115280095190773, 115797848077099, 1099726899285419]\n\n\nTimer.directory = "./prime_multiprocessing_records/"\n\n\ndef determine_prime(number):\n    with Timer("Determine prime") as t:\n        not_prime = False\n\n        if number > 1:\n            for i in range(2, math.floor(math.sqrt(number) + 1)):\n                if (number % i) == 0:\n                    not_prime = True\n                    break\n\n        t.text = f"{number}\\nnot prime" if not_prime else f"{number}\\nis prime"\n\n\nif __name__ == "__main__":\n    with concurrent.futures.ProcessPoolExecutor(max_workers=2) as executor:\n        executor.map(determine_prime, NUMBERS)\n```\n\nAfter the program finishes, we can open the `./prime_multiprocessing_records/` directory where we will find 2 report files - one for each process.\nThey are differentiated by their process ID: `waterfalls.<process_id>.json`.\nWe use the viewer the same way we did before - the viewer automatically combines all record files in the directory into a unified view.\n\n```bash\nwaterfalls "./prime_multiprocessing_records/"\n```\n\n![](https://raw.githubusercontent.com/glenvorel/waterfalls/main/assets/diagram_prime_concurrent.png)\n\nMain observations:\n- We can _see_ that the refactored program can analyze 2 numbers in parallel.\n- It takes around 4.7 s to finish the analysis. (It was over 11 s for the version using a single process.)\n\n## License\n\n[MIT license](https://github.com/glenvorel/performancepy/blob/master/LICENSE)',
    'author': 'Glen Vorel',
    'author_email': 'glen.vorel@gmail.com',
    'maintainer': None,
    'maintainer_email': None,
    'url': None,
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'entry_points': entry_points,
    'python_requires': '>=3.8,<4.0',
}


setup(**setup_kwargs)
