/**
\page yarp_protocol Port and connection protocols

\tableofcontents

\section yarp_connection_protocol The connection protocol

This is the protocol used for a single connection from an output port to an
input port.
We discuss how this process gets initiated in the next section.
At the point of creation of a connection, the following information is needed:

- An address -- the machine name and socket-port at which the input port is
  listening.

- The name of the input port.

- The name of the output port associated with the connection.
  This name needs to be retained for proper disconnection in some cases.
  If the connection is not actually associated with a port, but is initiated by
  an external entity, then the name is not important and should be set to
  "external" (or any name without a leading slash characher).


\subsection connection_phases Connection phases

The connection protocol has several phases -- header, index, and body.

- **Initiation phase**

  - We begin once the sender has successfully opened a tcp socket connection to
    the receiver (assuming that is the carrier it is registered with).

- **Header phase**

  - This phase follows immediately after the initiation phase.
  - Transmission of protocol specifier
    - Sender transmits 8 bytes that identify the carrier that will be used.
      The header may be used to pass a few flags also.
    - Receiver expects 8 bytes, and attempts to find a carrier that is
      consistent with them.
  -  Transmission of sender name
     - Sender transmits the name of the output port it is associated with, in a
       carrier specific way.
     - Receiver expects the name of the output port, transmitted in a carrier
       specific way.
  - Transmission of extra header material
    - Sender may transmit extra information, depending on the carrier.
    - Receiver may expect extra information, depending on the carrier.

- **Header reply phase**

  - This phase follows immediately after the header phase, and concludes the
    preamble to actual data transmission.
    After this phase, the two ports are considered connected.
  - Receiver may transmit some data, depending on the carrier.
    Receiver then may switch from the initial network protocol used to something
    else (udp, mcast, etc), again depending on the carrier.
  - Sender may expect some data, depending on the carrier.
    Sender then may switch from the initial network protocol used to something
    else (udp, mcast, etc), again depending on the carrier.

- **Index phase**

  - Sender sends carrier-dependent data describing properties of the payload
    data to come.
  - Receiver expects carrier-dependent data describing properties of the payload
    data to come.

- **Payload data phase**

  - Sender sends carrier-dependent expression of user data (maybe none).
  - Receiver expects carrier-dependent expression of user data.

- **Acknowledgement phase**

  - Receiver sends carrier-dependent acknowledgement of receipt of payload data
    (maybe none).
  - Sender expects carrier-dependent acknowledgement of receipt of payload data
    (maybe none).

This is the basic pattern of YARP communication between ports.
Clearly different carriers have a lot of freedom in how they operate.


\subsection carrier_tcp The "tcp" carrier


- **Header and header reply**

  - The 8-byte protocol specifier for tcp is:
    {`Y` `A` `0xE4` `0x1E` `0x00` `0x00` `R` `P`}.
    Another possible variant is:
    {`Y` `A` `0x64` `0x1E` `0x00` `0x00` `R` `P`}.
    The first version identifies a connection that sends acknowledgements;
    the second is for connections that omit acknowledgements.

  - The sender name is transmitted and expected to be in the following format:
    a 4 byte integer (little endian) giving the length of port, followed by the
    port name in characters, followed by the null character.

  - There is no extra header material for this carrier.

  - The header reply is as 8 bytes long:
    {`Y` `A` `B1` `B2` `0x00` `0x00` `R` `P`},
    where (`B1`,`B2`) is a (little-endian) two-byte integer
    specifying a socket-port number (unused).

  - After the header reply, there is no switch in network protocol, the initial
    tcp connection continues to be used.

- **Index, payload, and acknowledgement**

  - The sender transmits 8 bytes:
    {`Y` `A` `0x0A` `0x00` `0x00` `0x00` `R` `P`}.
    This identifies the length of the "index header" as 10 (`0x0A`).

  - The sender transmits 10 bytes:
    {`LEN` `0x01` `0xFF` `0xFF` `0xFF` `0xFF` `0xFF` `0xFF` `0xFF` `0xFF`}.
    `LEN` is the number of blocks of user data need to be sent.
    This byte-sequence says there are `LEN` send blocks, 1 reply block expected,
    and that the sizes will be listed individually next (this odd format is for
    backward compatibility with older YARP versions).

  - The sender transmits `LEN` 4-byte little-endian integers, one for each of
    the `LEN` blocks of user data, giving the length of each block.

  - The sender transmits 4 bytes:
    {`0x00` `0x00` `0x00` `0x00`}.
    This asks for a reply length of 0.

  - If this is the variant of the tcp carrier that requires acknowledgments,
    then the receiver sends 8 bytes:
    {`Y` `A` `B1` `B2` `B3` `B4` `R` `P`}, where (`B1`,`B2`,`B3`,`B4`) is a
    little-endian integer giving a length (could be 0).
    It then sends that number of extra bytes.

\subsection carrier_udp The "udp" carrier


- **Header and header reply**

  - The 8-byte protocol specifier for udp is:
    {`Y` `A` `0x61` `0x1E` `0x00` `0x00` `R` `P`}.
    The following variant of this should also be accepted:
    {`Y` `A` `0xE1` `0x1E` `0x00` `0x00` `R` `P`}
    (it is the same thing).
  - Otherwise header and header reply are identical to the tcp case.
  - After the header reply, both sides switch to a udp connection to the
    socket-port specified in the header reply.

- **Index, payload, and acknowledgement**

  - Identical to tcp.
    Data is split arbitrarily to fit into datagrams.
    Datagrams have sequence numbers and checksums so that if part of message is
    lost or corrupted the entire message can be dropped cleanly.

  - Acknowledgments are not a possibility.

\subsection carrier_mcast The "mcast" carrier

- **Header and header reply**

  - The 8-byte protocol specifier for mcast is:
    {`Y` `A` `0x62` `0x1E` `0x00` `0x00` `R` `P`}.
    The following variant of this should also be accepted:
    {`Y` `A` `0xE2` `0x1E` `0x00` `0x00` `R` `P`}
    (it is the same thing).
  - The sender name is sent as for tcp.
  - Extra header material is send (6 bytes).
    The first 4 bytes specify a multicast IP address.
    Next 2 bytes are a (bigendian) integer giving a socket-port number.
    Note that producing these numbers can be helped by side communication with
    the name server.
  - There is no header reply for mcast.
  - Both sides switch to a multi-cast group on the specified IP and socket-port.

- **Index, payload, and acknowledgement**

  - Identical to udp.
  - But at most one connection from a given port with an mcast carrier should
    actually write to the multi-cast group.

\subsection carrier_text The "text" carrier

- This carrier is carefully designed to make it easy to type into a terminal.

- **Header and header reply**

  - The 8-byte protocol specifier for text is:
    {`C` `O` `N` `N` `E` `C` `T` ` `}.
  - The sender name is sent as plain text followed by the newline character
    `\n`.
  - There is no extra material.
  - There is no header reply expected for text.
  - There is no network protocol switch.

- **Index, payload, and acknowledgement**

  - There is no index.
  - The payload is expected to be a series of lines of text terminated by the
    newline character `\n`.
  - There is no acknowledgement expected for text.

\subsection proto_shmem The "shmem" carrier

This is essentially the same as the tcp carrier, except that there is no
header reply, and there is a shift in protocol after header transmission
on both sides to an ACE shared memory stream.  This carrier is
currently being reworked to make its specification independent of ACE,
and to further improve efficiency in an existing implementation.

The advantage of this carrier is that it is fast -- the best way to
send messages between processes on a single machine.  Of course, it
doesn't work for processes on different machines.


\subsection carrier_local The "local" carrier

This carrier is designed specifically for communication between threads in a
single process.
Giving a specification for the protocol it uses has low priority, since two such
threads are unlikely to be using different YARP implementations.


\subsection carrier_known Known protocol specifiers

Here are the currently known protocol specifiers.
The "shmem" carrier is not yet documented, but is
implemented in the C++ version of YARP.


| 8-byte magic number                       | protocol | variant    |
|-------------------------------------------|----------|------------|
|`Y` `A` `0x61` `0x1E` `0x00` `0x00` `R` `P`| udp      |            |
|`Y` `A` `0xE1` `0x1E` `0x00` `0x00` `R` `P`| udp      |            |
|`Y` `A` `0x62` `0x1E` `0x00` `0x00` `R` `P`| mcast    |            |
|`Y` `A` `0xE2` `0x1E` `0x00` `0x00` `R` `P`| mcast    |            |
|`Y` `A` `0x63` `0x1E` `0x00` `0x00` `R` `P`| shmem    |            |
|`Y` `A` `0xE3` `0x1E` `0x00` `0x00` `R` `P`| shmem    |            |
|`Y` `A` `0x64` `0x1E` `0x00` `0x00` `R` `P`| tcp      |without acks|
|`Y` `A` `0xE4` `0x1E` `0x00` `0x00` `R` `P`| tcp      |with acks   |
|`C` `O` `N`    `N`    `E` `C` `T` ` `      | text     |without acks|
|`C` `O` `N`    `N`    `A` `C` `K` ` `      | text     |with acks   |
|`L` `O` `C`    `A`    `L` `I` `T` `Y`      | local    |            |


\section yarp_protocol_port Port commands

Every port is always available for new connections from external
entities - to request that new connections between ports be created,
old connections be removed, to inquire after status, etc.
%
The protocol used for communicating with a port is layered on top of the
protocol described in the previous section.  Any carrier can be used.
The "payload data" is as follows:


- We send an 8 byte header {`0x00`,`0x00`,`0x00`,`0x00`,`~`,`CHAR`,`0x00`,`0x01`}.

- `CHAR` is a character that identifies what the message is about.

- `CHAR` = `d`: this header is used to signal that user data is arriving next,
    as opposed to a port command

- `CHAR` = anything else: this signals that a port command
    follows.


for the port command case (`CHAR` = `0x00`) the remainder of the message
is interpreted as a string `S`.

- `S` begins with `/`, e.g. `/read`: this is a request to add a
   Connection to the named InputPort.

- `S` begins with `!`, e.g. `!/read`: this is a request to remove
   a Connection to the named InputPort.

- `S` begins with `~, e.g. `~/read`: this is a
   request to remove a Connection from the named OutputPort.

- `S` is `*`: this is a request for the port to dump information
     about what it is connected to.

- `S` is `q`: the specific connection that the command is received on
   should now shut down.

- `S` is `a`: this is like `d`, signalling that data should be expected,
  but the data will not be passed on to the client of the port.
  Instead it will be processed internally according to \ref port_admin.
  This is a more user-friendly interface for communicating with a port.


Alternatively, with the "text" carrier, we send a string terminated
in `\n`.  This is the string `S`.  The first letter is copied
to be `CHAR`.



\section yarp_uri YARP URIs

Port names in YARP can contain multiple special elements.
We've seen names such as `/write`.
We can also have names such as `udp://write` which means "connect to the port
named `/write` using the udp carrier".

We can also prepend a network selector of the form `/net=NETNAME/`.
For example, a name such as `udp://net=196/write` means "connect to the port
named `/write` using the udp carrier, and make the connection on the network
with ip addresses beginning with `196`".
This is useful in scenarios with multiple networks, where it may be desirable to
route connections through particular networks (for example, to devote a network
to time-critical traffic).
This functionality is supported primarily with the help of the name server.
The ip it reports for a machine is usually a reasonable default, but the user
can choose using "net=" to request a name on a particular network.

Symbolic network names can be configured.
This process is not yet specified.
You can do it right now by setting properties of a fake port called `networks`
(no leading slash), where the properties are symbolic names and their values are
the numeric network IP prefix.
But this process will change.


\section yarp_carriers Carriers supported

An implementation of YARP2 must support at least the "tcp" carrier.
Other carriers that may be supported: "text", "udp", "mcast", "shmem", "local".

As a place to start an implementation, the "text" carrier is very simple to
implement, and can masquerade as "tcp" for the purposes of initial handshaking.

To see this, get the "netcat" program (available as debian package of the same
name).
In one terminal, run:
\verbatim
nc -l -p 9000
\endverbatim
This starts a tcp listener on socket-port 9000, and prints out any data that
arrives there.
Then tell the name server to create an entry for this listener, and tag it as
accepting text:
\verbatim
yarp name register /nc tcp ... 9000
yarp name set /nc accepts text
\endverbatim
Now lets write some data to that port.
\verbatim
yarp write /write text://nc
\endverbatim

Type something in, such as "hello world", and hit return.
On the terminal running netcat you should see:
\verbatim
CONNECT /write
d
0 "hello world"
\endverbatim
This is what text mode looks like, for the particular data type used by yarp
read and write ("bottles").
As we saw in an earlier section, we can also write to ports in text mode.
And if we were to restart nc and then try the following:
\verbatim
yarp connect text://nc /foo
\endverbatim
You should see:
\verbatim
CONNECT external
/foo
\endverbatim
This is what a command to connect looks like in YARP2.
If we omit the `text:/` then the tcp carrier may be used, which is compatible
with YARP1 but is a bit less trivial to work with.
Once our YARP implementations are up to date, the default command carrier will
be switched to text.


\section note_manually Manually interacting with ports

Suppose we have created ports as follows by typing the following in different
terminals:
\verbatim
  yarp server
  yarp write /write
  yarp read /read
  yarp read /read2
\endverbatim

We could connect and disconnect ports using the YARP companion utility, but
here's how we could do the same thing "manually":

\verbatim
command:  yarp where
response: Name server is available at ip 192.168.0.3 port 10000

command:  telnet 192.168.0.3 10000
type:     NAME_SERVER query /write
response: registration name /write ip 192.168.0.3 port 10001 type tcp
          *** end of message
          [connection closes]

command:  telnet 192.168.0.3 10001
type:     CONNECT anonymous
response: Welcome anonymous
type:     *
response: This is /write
          There are no outgoing connections
          There is this connection from anonymous to /write using protocol tcp
          *** end of message
type:     /read
response: Connected to /read
type:     *
response: This is /write
          There is a connection from /write to /read using protocol tcp
          There is this connection from anonymous to /write using protocol tcp
          *** end of message
type:     !/read
response: Removing connection from /write to /read
type:     /mcast://read
response: Connected to /read
type:     /read2
response: Connected to /read2
type:     *
response: This is /write
          There is a connection from /write to /read using protocol mcast
          There is a connection from /write to /read2 using protocol tcp
          There is this connection from anonymous to /write using protocol tcp
          *** end of message
type:     q
response: Bye bye
          [connection closes]

command:  telnet 192.168.0.3 10000
type:     NAME_SERVER query /read
response: registration name /write ip 192.168.0.3 port 10002 type tcp
          *** end of message
          [connection closes]

command:  telnet 192.168.0.3 10002
type:     CONNECT anonymous
response: Welcome anonymous
type:     *
response: This is /read
          There are no outgoing connections
          There is a connection from /write to /read using protocol mcast
          There is this connection from anonymous to /read using protocol tcp
          *** end of message
type:     q
response: Bye bye
          [connection closes]
\endverbatim


*/
