Metadata-Version: 2.1
Name: manylog
Version: 0.1.0a2
Summary: Collect log messages and progress updates from multiple processes.
Author-email: Michael Ekstrand <mdekstrand@drexel.edu>
License: Copyright (c) 2023 Drexel University and contributors
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        > The above copyright notice and this permission notice shall be included in
        > all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Documentation, https://manylog.readthedocs.io/
Project-URL: GitHub, https://github.com/lenskit/manylog
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: progress-api>=0.1.0a5
Requires-Dist: pyzmq
Requires-Dist: typing-extensions~=4.11
Requires-Dist: msgpack==1.*
Provides-Extra: dev
Requires-Dist: setuptools>=64; extra == "dev"
Requires-Dist: setuptools_scm>=8; extra == "dev"
Requires-Dist: build==1.*; extra == "dev"
Requires-Dist: ruff>=0.2; extra == "dev"
Requires-Dist: pyright; extra == "dev"
Requires-Dist: copier==9.*; extra == "dev"
Requires-Dist: unbeheader~=1.3; extra == "dev"
Requires-Dist: ipython; extra == "dev"
Requires-Dist: pyproject2conda; extra == "dev"
Requires-Dist: sphinx-autobuild; extra == "dev"
Requires-Dist: msgpack-types; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Requires-Dist: pytest-cov>=2.12; extra == "test"
Requires-Dist: coverage>=5; extra == "test"
Provides-Extra: doc
Requires-Dist: sphinx>=4.2; extra == "doc"
Requires-Dist: sphinxext-opengraph>=0.5; extra == "doc"
Requires-Dist: furo; extra == "doc"

# manylog

The `manylog` package facilitates logging and progress reporting in Python
multiprocessing settings.  Right now, logging across process doesn't work
well, except for the natural interleaving that sometimes works acceptably
when using the `fork` multiprocessing context.  It supports log messages
through the Python standard library's logging module, and progress reported
with [progress-api][].

[progress-api]: https://progress-api.readthedocs.io/en/latest/

This package fixes that, through two parts:

-   A listener that runs in a parent process, listening for log messages and
    progress updates.  It republishes these log messages into the parent
    process's logging infrastructure so filters, handlers, etc. all apply.

-   Logging and progress backend for worker processes that forwards messages
    to the parent process.

It is agnostic to the specific multiprocessing framework in use, and will work
with stdlib multiprocessing, joblib (although Joblib lacks the setup API needed
to make it work easily), ipyparallel, Torch multiprocessing, and others. It uses
ZeroMQ to route messages between parent and child processes.

Currently, only single-machine multiprocessing is supported, but ZeroMQ will
make it easy to add cluster log and progress aggregation in the future.

## Example

In the parent:

```python
import multiprocessing.Pool
from manylog import LogListener, init_worker_logging

with LogListener() as ll, mp.Pool(4, init_worker_logging, (ll.address,), None, mp.get_context('spawn')) as pool:
    # use `pool` to schedule parallel work
```

Log messages from worker processes will be routed through whatever logging and
progress configuration you have set up.
