Metadata-Version: 2.1
Name: swim-protocol
Version: 0.6.2
Summary: SWIM protocol implementation for exchanging cluster membership status and metadata.
Project-URL: Homepage, https://github.com/icgood/swim-protocol/
Project-URL: API Documentation, https://icgood.github.io/swim-protocol/
Author-email: Ian Good <ian@icgood.net>
License: ## MIT License
        
        Copyright (c) 2021 Ian Good
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE.md
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: ~=3.10
Requires-Dist: typing-extensions
Provides-Extra: dev
Requires-Dist: autopep8; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pycodestyle; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-asyncio; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: doc
Requires-Dist: cloud-sptheme; extra == 'doc'
Requires-Dist: sphinx; extra == 'doc'
Requires-Dist: sphinx-autodoc-typehints; extra == 'doc'
Description-Content-Type: text/markdown

swim-protocol
=============

[SWIM protocol][0] implementation for exchanging cluster membership status and
metadata.

[![build](https://github.com/icgood/swim-protocol/actions/workflows/python-check.yml/badge.svg)](https://github.com/icgood/swim-protocol/actions/workflows/python-check.yml)
[![PyPI](https://img.shields.io/pypi/v/swim-protocol.svg)](https://pypi.python.org/pypi/swim-protocol)
[![PyPI](https://img.shields.io/pypi/pyversions/swim-protocol.svg)](https://pypi.python.org/pypi/swim-protocol)
![platforms](https://img.shields.io/badge/platform-linux%20%7C%20macOS%20%7C%20windows-blueviolet)
[![PyPI](https://img.shields.io/pypi/l/swim-protocol.svg)](https://pypi.python.org/pypi/swim-protocol)

This library is intended to fit into an [asyncio][1] event loop to help
synchronize a distributed group of processes.

#### [Introduction](https://icgood.github.io/swim-protocol/intro.html)

#### [API Documentation](https://icgood.github.io/swim-protocol/)

## Install and Usage

```console
$ pip install swim-protocol
```

### File Sync Tool

A basic tool for reporting cluster membership and synchronizing metadata as
files is provided:

```console
$ swim-protocol-sync --name 127.0.0.1:2001 --peer 127.0.0.1:2002 ~/node1
$ swim-protocol-sync --name 127.0.0.1:2002 --peer 127.0.0.1:2001 ~/node2
```

While running, the state of the cluster and the metadata of each member are
visible on the filesystem.

```console
$ tree -a ~/node1
node1
├── .available
│   └── 127.0.0.1:2002 -> ../127.0.0.1:2002
├── .local -> 127.0.0.1:2001
├── .offline
├── .online
│   └── 127.0.0.1:2002 -> ../127.0.0.1:2002
├── .suspect
├── .unavailable
├── 127.0.0.1:2001
│   └── file-one.txt
└── 127.0.0.1:2002
    └── file-two.txt
```

To change the metadata of the local cluster member, edit the files and issue a
SIGHUP to the process:

```console
$ vim ~/node1/.local/file-one.txt
$ pkill -HUP -f swim-protocol-sync
```

#### Running the Demo

There is a [demo][2] application included as a reference implementation. Try it
out by running the following, each from a new terminal window, and use _Ctrl-C_
to exit:

```console
$ swim-protocol-demo --name 127.0.0.1:2001 --peer 127.0.0.1:2003
$ swim-protocol-demo --name 127.0.0.1:2002 --peer 127.0.0.1:2001
$ swim-protocol-demo --name 127.0.0.1:2003 --peer 127.0.0.1:2001
$ swim-protocol-demo --name 127.0.0.1:2004 --peer 127.0.0.1:2003
```

Typing in any window will disseminate what has been typed across the cluster
with [eventual consistency][6].

![swim-protocol-demo](https://user-images.githubusercontent.com/438413/117895781-13f6b400-b28d-11eb-997d-d8b9dbc455cb.gif)

### Getting Started

First you should create a new [UdpConfig][100] object:

```python
from swimprotocol.udp import UdpConfig

config = UdpConfig(local_name='127.0.0.1:2001',
                   local_metadata={'name': b'one'},
                   peers=['127.0.0.1:2002'],
                   secret='my secret')
```

All other config arguments have default values, which are tuned somewhat
arbitrarily with a small cluster of 3-4 members in mind.

Now you can create the cluster members manager and transport layer, and enter
the event loop:

```python
from contextlib import AsyncExitStack
from swimprotocol.members import Member, Members
from swimprotocol.udp import UdpTransport
from swimprotocol.worker import Worker

members = Members(config)
worker = Worker(config, members)
transport = UdpTransport(config, worker)

async def run() -> None:
    async with AsyncExitStack() as stack:
        await stack.enter_async_context(transport)
        await stack.enter_async_context(worker)
        await stack.enter_async_context(
            members.listener.on_notify(on_member_change))
        await ...  # run your application

async def on_member_change(member: Member) -> None:
    ...  # handle a change in member status or metadata
```

These snippets demonstrate the UDP transport layer directly. For a more generic
approach that uses [argparse][11] and [load_transport][103], check out the
[demo][2] or the [sync tool][15].

If your application is deployed as a [Docker Service][13], the [UdpConfig][100]
`discovery=True` keyword argument can be used to discover configuration based
on the service name. See the [documentation][14] for more comprehensive usage.

### Checking Members

The [Members][101] object provides a few ways to check on the cluster and its
members:

```python
for member in members.non_local:
    # all other known cluster members
    print(member.name, member.status, member.metadata)

from swimprotocol.status import Status
for member in members.get_status(Status.AVAILABLE):
    # all cluster members except offline
    print(member.name, member.status, member.metadata)
```

Alternatively, listen for status or metadata changes on all members:

```python
from swimprotocol.member import Member

async def _updated(member: Member) -> None:
    print('updated:', member.name, member.status, member.metadata)

async with AsyncExitStack() as stack:
    # ...
    stack.enter_context(members.listener.on_notify(_updated))
```

### UDP Transport Security

The [UdpTransport][102] transport layer (the only included transport
implementation) uses salted [hmac][7] digests to sign each UDP packet payload.
Any UDP packets received that are malformed or have an invalid signature are
*silently* ignored. The eventual consistency model should recover from packet
loss.

The signatures rely on a [shared secret][8] between all cluster members, given
as the `secret=b'...'` argument to the [UdpConfig][100] constructor. If
`secret=None` is used, it defaults to [`uuid.getnode()`][9] but this is **not
secure** for production setups unless all sockets are bound to a local loopback
interface.

The cluster member metadata is **not** encrypted during transmission, so only
private networks should be used if metadata includes any secret data, or that
secret data should be encrypted separately by the application.

If member [metadata][12] is larger than can be transmitted in a single UDP
packet (hard-coded at 1500 bytes due to [MTU][10] sizes on public networks), a
TCP connection is used instead. There is no additional protocol for TCP; the
connection is opened, the oversized packet is transmitted, and then the
connection is closed without waiting for a response.

## Development

You will need to do some additional setup to develop and test plugins. Install
[Hatch][3] to use the CLI examples below.

Run all tests and linters:

```console
$ hatch run check
```

Because this project supports several versions of Python, you can use the
following to run the checks on all versions:

```console
$ hatch run all:check
```

### Type Hinting

This project makes heavy use of Python's [type hinting][4] system, with the
intention of a clean run of [mypy][5]:

```console
$ mypy
```

No code contribution will be accepted unless it makes every effort to use type
hinting to the extent possible and common in the rest of the codebase.

[0]: https://www.cs.cornell.edu/projects/Quicksilver/public_pdfs/SWIM.pdf
[1]: https://docs.python.org/3/library/asyncio.html
[2]: https://github.com/icgood/swim-protocol/blob/main/swimprotocol/demo/__init__.py
[3]: https://hatch.pypa.io/latest/install/
[4]: https://docs.python.org/3/library/typing.html
[5]: http://mypy-lang.org/
[6]: https://en.wikipedia.org/wiki/Eventual_consistency
[7]: https://docs.python.org/3/library/hmac.html
[8]: https://en.wikipedia.org/wiki/Shared_secret
[9]: https://docs.python.org/3/library/uuid.html#uuid.getnode
[10]: https://en.wikipedia.org/wiki/Maximum_transmission_unit
[11]: https://docs.python.org/3/library/argparse.html
[12]: https://icgood.github.io/swim-protocol/intro.html#term-metadata
[13]: https://docs.docker.com/engine/swarm/how-swarm-mode-works/services/
[14]: https://icgood.github.io/swim-protocol/swimprotocol.udp.html#docker-services
[15]: https://github.com/icgood/swim-protocol/blob/main/swimprotocol/sync.py

[100]: https://icgood.github.io/swim-protocol/swimprotocol.udp.html#swimprotocol.udp.UdpConfig
[101]: https://icgood.github.io/swim-protocol/swimprotocol.html#swimprotocol.members.Members
[102]: https://icgood.github.io/swim-protocol/swimprotocol.udp.html#swimprotocol.udp.UdpTransport
[103]: https://icgood.github.io/swim-protocol/swimprotocol.html#swimprotocol.transport.load_transport
