The Pipeline server — pylm 0.14.1 documentation

# The Pipeline server

The serial server presented in the previous section is intended to receive messages from a client. The pipeline server is just like a serial server, but it is designed to receive a stream of messages from another server, forming a pipeline. It can then redirect the messages to another server or it can route the messages back to the client to close the message loop.

The simplest architecture where a Pipeline server is useful is adding an additional step to a client-server call like the one presented in the following figure.

You can see an example how the Pipeline server to create a pipeline started by a Server in the examples section (A pipelined message stream). The big picture is simple. A Server starts a pipeline, and the pipeline servers are the steps of it.

One important detail in this first example is that the client gets a sequence of method calls, the server name and the method of each step, in a list. This of course means that the first argument of the `pylm.clients.Client.eval()` and `pylm.clients.Client.job()` methods in may be either a string or a list of strings.

Pipeline servers can be attached to Master servers too to attach a parallel-processing step to a serial-processing step

You can find the full example in Connecting a pipeline to a master

## Controlling the messages down the pipeline

One important feature of the pipelined message stream is that it can be controlled and diverted. If one connects multiple pipeline servers to a single server, the default behavior is to send all messages to all the connected pipelines.

If you take a look at the full example (A pipelined message stream forming a tee), you can see that the Pipeline needs an extra argument, which is the name of the server or the pipeline at the previous step. At the same time, one must tell the servers at its creation that the stream of messages will be sent to a Pipeline, and not sent back to the client.

If you want a finer-grain control over where each message is sent down the pipeline you can use the handle_stream method to manage the stream. This can be used in combination with the `previous` option to fully manage the routing of the messages on each step.

You can see the full example here (A pipelined message stream forming a tee and controls the stream of messages).