========================
DistKV's client protocol
========================

DistKV's native client protocol is based on MsgPack. The client sends
requests; the server sends one or more responses. You may (and indeed
should) run concurrent requests on the same connection.

Strings must be UTF-8, as per MsgPack specification.

Requests and replies are mappings.

The server initially sends a greeting, using sequence number zero. It will
not send any other unsolicited message.

Requests
========

Client requests are always mappings. ``seq`` and ``action`` must be
present. All other fields are request specific. The server will ignore
fields it doesn't understand.

seq
---

Every client request must contain a strictly increasing positive sequence
number. All replies associated with a request carry the same sequence
number.

action
------

The action which the server is requested to perform. Valid actions are
described below.

nchain
------

This field tells the DistKV server how many change entries to return.
The default is zero. If you want to update a value, retrieve the
original with ``nchain`` set to one. Synchronization between DistKV servers
requires the number of possible partitions plus one, in order to protect
against spurious conflict reports.


Replies
=======

Server replies are always mappings. At least one of ``seq`` and ``error``
must be present. The client must ignore fields it doesn't expect.

seq
---

The sequence number of the request which caused this reply.

Server messages which don't include a sequence number are errors and
will close the connection.

The server will either send exactly one reply with any given sequence number,
or a multi-reply sequence which starets with a ``state=start`` message.

error
-----

This field contains a human-readable error message. A request has failed if
this field is present.

value
-----

The value of the DistKV entry, assuming one was requested.

state
-----
May be ``start`` or ``end``

* ``start`` indicates the beginning of a multi-value result.

* ``end`` indicates that a multi-value result has finished. No more
  messages with this sequence number will be sent.

The ``start`` message will contain neither an error nor a value.

chain
-----

The change chain resulting from, or retrieved by, a command.

Change chains track which server last modified a value, so that replayed
updates can be ignored and conflicting updates can be recognized. A chain
never contains any one DistKV server more than once.

See the server protocol for a detailed description.

tick
----

The current server's change counter. This field can be used to ensure that
the local server is not restarted with old state.

tock
----

An always-increasing integer that's (supposed to be) shared within the
whole DistKV system. You can use it when you need to reconnect to a server,
to make sure that the system is (mostly) up-to-date.

Actions
=======

connect
-------

This is a pseudo-action with sequence number zero, which the server assumes
to have received after connecting. The server's first message will contain
``seq=0``, its ``node`` name, a ``version`` (as a list of integers), and
possibly its current ``tick`` and ``tock`` sequence numbers.

The ``auth`` parameter, if present, carries a list of configured
authorization methods. The first method in the list **should** be used to
authorize the client. If the list's first entry is ``None`` then
authorization is not required. Other entries **may** be used for
testing after a client is logged in.

auth
----

Tell the server about your identity. This method **must** be sent first if
the server requests authorization.

The ``identity`` parameter tells the server which user ID (or equivalent)
to use for logging in. ``typ`` contains the auth type to use; this
**must** be identical to the first entry in the ``connect`` reply's
``auth`` parameter.

If this is not the first message, the authorization is verified but the
resulting user identity is ignored.

test_acl
--------

Check whether the given ``path`` is accessible with the given  ``mode``.

The ``acl`` to test may be specified. The user's ACL, if any, is also
tested; the return message's ``access`` element may contain ``False``
(access not allowed), ``True`` (access allowed but no ACL details
available) or the actual ACL characters.

Access will not be granted if you try to check a specific ACL when your
own rights don't include 'a' (for accessing ACLs).

stop
----

Send this action to abort a running multi-value request. Set ``task`` to
the sequence number of the request to abort.

This action only works after you received a ``start`` state message.
It returns a :class:`bool` which is ``True`` if the command was still
running.

A positive reply does not indicate that no more messages with the stated
sequence number will arrive; this will be indicated by the ``state=end``
message.

get_value
---------

Retrieve a single value. The ``path`` to the value needs to be sent as a list.

If the value does not exist or has been deleted, you'll get ``None`` back.

Alternately, you can set ``node`` and ``tick``, which returns the entry
that has been set by this event (if the event is still available). The
entry will contain the current value even if the event has set a previous
value.

set_value
---------

Set a single value. The ``path`` to that ``value`` needs to be sent as a list.

If you are updating a known value, you should send a ``chain`` entry
to help ensure that no other node has changed it unexpectedly. (Of course,
due to the distributed nature of DistKV, this may happen anyway.) You can
also use ``prev`` to send an expected old value, but you really shouldn't.

This action returns the node's new change ``chain``. If you did not send a
``chain`` field, the previous value is returned in ``prev``.

delete_value
------------

Remove a single value. This is the same as setting it to ``None``.

get_state
---------

Retrieve the current system state. The following ``bool`` attributes can be
set to specify what is returned. The reply is stored in an attribute of the
same name.

* nodes

A dict of node ⇒ tick.

* known

A dict of node ⇒ ranges of ticks known. This contains current data as well
as events that have been superseded.

* current

A dict of node ⇒ ranges of ticks corresponding to the current state of
nodes. This is expensive to calculate. It is a superset of `'known``.

* missing

A dict of node ⇒ ranges of ticks not available locally. This is the inverse
of ``known``.

* remote_missing

A dict of node ⇒ ranges of ticks reported to be missing at some other node.

get_tree
--------

Retrieves all values with the prefix given in ``path``.

This is a multi-value reply; each reply contains ``path`` and ``value``
entries. Deleted nodes may or may not be reported.

If the path does not exist or does not have children, a single-value reply
is returned.

Optimization: if a reply contains a "depth" key, its path is shortened by
the request's path, plus that many elements from the previous reply's path.

Thus, if you request a path of ``['a','b','c']``, this reply::

    { seq=13, path=['a','b','c'], value="one" }
    { seq=13, path=['a','b','c','d','e'], value="two" }
    { seq=13, path=['a','b','c','d','f'], value="three" }

is equivalent to::

    { seq=13, depth=0, value="one" }
    { seq=13, depth=0, path=['d','e'], value="two" }
    { seq=13, depth=1, path=['f'], value="three" }

* min_depth

  Start reporting nodes at this depth.

* max_depth

  Limit recursion depth.

* add_empty

  Include empty nodes. This is useful when limiting the depth to non-leaf
  nodes without data.

root
----

Switch the client's root to the given path. This request returns the new
root node.

It is not possible to undo this request (other than to reconnect).
Tasks started before this action are not affected.

This action returns the new root node's value.

watch
-----

Monitor changes to this node (and those below it). Replies look like those from ``get_tree``.

The recommended way to run the ``watch`` call with ``fetch=True``. This
fetches the current state and guarantees that no updates are lost. To mark
the end of the static data, the server sends a ``state=uptodate`` message.
This process will not send stale data after an update, so your code may
safely replace an old entry's state with new data.

This task obeys ``min_depth`` and ``max_depth`` restrictions.

save
----

Instruct the server to save its state to the given ``path`` (a string with
a filename).

log
---

Instruct the server to continuously write change entries to the given ``path``
(a string with a filename). If ``fetch`` is ``True``, the server will also
write its current state to that file.

This command returns after the new file has been opened and the initial
state has been written, if so requested. If there was an old log stream,
there may be some duplicate entries. No updates are skipped.

msg_send
--------

Pass-through call to transmit a message. Parameters are ``type`` (the user
event to send to) and ``data`` (the data to send).

Raw binary data may be transmitted by using ``raw`` instead of ``data``.

msg_monitor
-----------

Pass-through call to receive brodcast messages. You'll get a
stream with ``data`` containing the decoded message. If decoding fails,
``raw`` contains the message's bytes and ``error`` holds a string
representation of the decoder problem.

Set ``raw`` to True if the incoming messages are not supposed to be
msgpack-encoded in the first place. In this case, ``data`` and ``error``
will always be missing.

Examples
========

You can turn on message debugging with 'distkv -vvv'.

Get and set a value
-------------------

If the value is not set::

    Send {'path': ('test',), 'nchain': 3, 'action': 'get_tree', 'seq': 1}
    Recv {'value': None, 'seq': 1}

Setting an initial value::

    Send {'value': 1234, 'path': ('test',), 'nchain': 2, 'chain': None, 'action': 'set_value', 'seq': 2}
    Recv {'changed': True, 'chain': {'node': 'test1', 'tick': 2, 'prev': None}, 'seq': 2}

Trying the same thing again will result in an error::

    Send {'value': 1234, 'path': ('test',), 'nchain': 2, 'chain': None, 'action': 'set_value', 'seq': 3}
    Recv {'error': 'This entry already exists', 'seq': 3}

To fix that, use the chain value you got when setting or retrieving the
previous value::

    Send {'value': 123, 'path': ('test',), 'nchain': 2, 'chain': {'node': 'test1', 'tick': 2}, 'action': 'set_value', 'seq': 4}
    Recv {'changed': True, 'chain': {'node': 'test1', 'tick': 3, 'prev': None}, 'seq': 4}

Sending no precondition would also work

After you set multiple values::

    Send {'value': 123, 'path': ('test', 'foo'), 'nchain': 0, 'action': 'set_value', 'seq': 5}
    Recv {'changed': True, 'prev': None, 'seq': 5}
    Send {'value': 12, 'path': ('test', 'foo', 'bap'), 'nchain': 0, 'action': 'set_value', 'seq': 6}
    Recv {'changed': True, 'prev': None, 'seq': 6}
    Send {'value': 1, 'path': ('test', 'foo', 'bar', 'baz'), 'nchain': 0, 'action': 'set_value', 'seq': 7}
    Recv {'changed': True, 'prev': None, 'seq': 7}
    Send {'value': 1234, 'path': ('test',), 'nchain': 0, 'action': 'set_value', 'seq': 8}
    Recv {'changed': True, 'prev': 123, 'seq': 8}

you can retrieve the whole subtree::

    Send {'path': ('test',), 'nchain': 0, 'action': 'get_tree', 'seq': 1}
    Recv {'seq': 1, 'state': 'start'}
    Recv {'value': 1234, 'depth': 0, 'seq': 1}
    Recv {'value': 123, 'path': ('foo',), 'depth': 0, 'seq': 1}
    Recv {'value': 12, 'path': ('bap',), 'depth': 1, 'seq': 1}
    Recv {'value': 1, 'path': ('bar', 'baz'), 'depth': 1, 'seq': 1}
    Recv {'seq': 1, 'state': 'end'}

Retrieving this tree with ``distkv client get -rd ':val' test`` would print::

    test:
      :val: 1
      foo:
        :val: 1
        bap: {':val': 12}
        bar:
          :val: 1
          baz: {':val': 1}


