Metadata-Version: 2.1
Name: mpi-list
Version: 0.3
Summary: A Distributed List API over MPI
Home-page: https://github.com/frobnitzem/mpi_list/
Author: David M. Rogers
Author-email: predictivestatmech@gmail.com
License: MIT
Project-URL: Documentation, https://mpi-list.readthedocs.io/
Platform: any
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python
Classifier: Topic :: Database
Requires-Python: >=3.6
Description-Content-Type: text/x-rst; charset=UTF-8
Requires-Dist: numpy
Requires-Dist: mpi4py
Requires-Dist: importlib-metadata ; python_version < "3.8"
Provides-Extra: testing
Requires-Dist: setuptools ; extra == 'testing'
Requires-Dist: pytest ; extra == 'testing'
Requires-Dist: pytest-cov ; extra == 'testing'

========
mpi list
========

This package implements the `DFM` class.

The `DFM` is a useful abstraction for working with
lists distributed over a set of MPI ranks.
The acronym stands for distributed free monoid,
which is just a fancy way to say it's a list.

If you're familiar with spark, it's like an RDD,
but only holds a list.

Quick Start
===========

.. code-block::

    from mpi_list import Context, DFM

    C = Context() # calls MPI_Init via mpi4py

    # After each of the three lines below:
    #  1. each rank now has 1000//C.procs consecutive numbers
    #  2. each rank now has a list of strings
    #  3. only numbers containing a '2' remain
    dfm = C . iterates(1000) \
            . map(lambda i: f"String {i}") \
            . filter(lambda s: '2' in s)

    if C.rank == 0:
        # Caution! Uncommenting this will deadlock your program.
        # Collective calls must be called by all ranks!
        #print( dfm . head(10) )
        pass

    # This is OK, since all ranks now have 'ans'
    ans = dfm.head(10)
    if C.rank == 0:
        print( ans )

    ans = dfm . filter(lambda s: len(s) <= len("String nn")) \
              . collect()
    if ans is not None: # only rank 0 gets "collect"
        print( ans )

Launch your program with `mpirun python my_prog.py`.

If you're using a supercomputer, consider installing
`spindle <https://computing.llnl.gov/projects/spindle/software>`_,
and then use `spindle mpirun python my_prog.py`.

.. _pyscaffold-notes:

Note
====

This project has been set up using PyScaffold 4.0.1. For details and usage
information on PyScaffold see https://pyscaffold.org/.


