Support for ibverbs
===================
Receiver performance can be significantly improved by using the Infiniband
Verbs API instead of the BSD sockets API. This is currently only tested on
Linux with Mellanox ConnectX®-3 NICs. It depends on device managed flow
steering (DMFS), which may require using the Mellanox OFED version of
libibverbs.

There are a number of limitations in the current implementation:

 - Only IPv4 is supported
 - VLAN tagging, IP optional headers, and IP fragmentation are not supported
 - Only multicast is supported

Within these limitations, it is quite easy to take advantage of this faster
code path. The main difficulty is that one *must* specify the IP address of
the interface that will receive the packets. The netifaces_ module can help
find the IP address for an interface by name.

.. _netifaces: https://pypi.python.org/pypi/netifaces

System configuration
--------------------
It is likely that some system configuration will be needed to allow this mode
to work correctly. For ConnectX®-3, add the following to
:file:`/etc/modprobe.d/mlnx.conf`::

   options ib_uverbs disable_raw_qp_enforcement=1
   options mlx4_core fast_drop=1
   options mlx4_core log_num_mgm_entry_size=-1

For more information, see the `libvma documentation_`.

.. _libvma documentation: https://github.com/Mellanox/libvma

Programmatic interface
----------------------
The ibverbs API can be used programmatically by using an extra method of
:py:class:`spead2.recv.Stream`:

.. py:method:: spead2.recv.Stream.add_udp_ibv_reader(multicast_group, port, interface_address, max_size=DEFAULT_UDP_IBV_MAX_SIZE, buffer_size=DEFAULT_UDP_IBV_BUFFER_SIZE, comp_vector=0, max_poll=DEFAULT_UDP_IBV_MAX_POLL)

      Feed data from a UDP port with multicast (IPv4 only).

      :param str multicast_group: Hostname/IP address of the multicast group to subscribe to
      :param int port: UDP port number
      :param str interface_address: Hostname/IP address of the interface which
        will be subscribed
      :param int max_size: Maximum packet size that will be accepted
      :param int buffer_size: Requested memory allocation for work requests. Note
        that this is used to determine the number of packets
        to buffer; if the packets are smaller than `max_size`,
        then fewer bytes will be buffered.
      :param int comp_vector: Completion channel vector (interrupt)
        for asynchronous operation, or
        a negative value to poll continuously. Polling
        should not be used if there are other users of the
        thread pool. If a non-negative value is provided, it
        is taken modulo the number of available completion
        vectors. This allows a number of readers to be
        assigned sequential completion vectors and have them
        load-balanced, without concern for the number
        available.
      :param int max_poll: Maximum number of times to poll in a row, without
        waiting for an interrupt (if `comp_vector` is
        non-negative) or letting other code run on the
        thread (if `comp_vector` is negative).

Environment variables
---------------------
An existing application can be forced to use ibverbs for all multicast IPv4
readers, by setting the environment variable :envvar:`IBV_INTERFACE` to the IP
address of the interface to receive the packets. Note that calls to
:py:meth:`spead2.recv.Stream.add_udp_reader` that pass an explicit interface
will use that interface, overriding :envvar:`IBV_INTERFACE`; in this case,
:envvar:`IBV_INTERFACE` serves only to enable the override.

It is also possible to specify :envvar:`IBV_COMP_VECTOR` to override the
completion channel vector from the default.
