.. _devguide-overview:

================================
 Contributors Guide to the Code
================================

.. contents::
    :local:

Module Overview
===============

``faust.app``
    Defines the Faust application: configuration, sending messages, etc.

``faust.cli``
    Command-line interface.

``faust.exceptions``
    All custom exceptions are defined in this module.

``faust.models``
    Models describe how message keys and values are serialized/deserialized.

``faust.sensors``
    Sensors record statistics from a running Faust application.

``faust.serializers``
    Serialization using JSON, and codecs for encoding.

``faust.stores``
    Table storage: in-memory, RocksDB, etc.

``faust.streams``
    Stream and table implementation.

``faust.topics``
    Creating topic descriptions, and tools related to topics.

``faust.transport``
    Message transport implementations, e.g. aiokafka.

``faust.types``
    Public interface for static typing.

``faust.utils``
    Utilities.  Note: This package is not allowed to import from the
    top-level package.

``faust.web``
    Web abstractions and web applications served by the Faust web server.

``faust.windows``
    Windowing strategies.

``faust.worker``
    Deployment helper for faust applications: signal handling, graceful
    shutdown, etc.

Services
========

Everything in Faust that can be started/stopped and restarted, is a
:class:`~faust.utils.services.Service`.

Services can start other services, but they can also start :class:`asyncio.Task`
via `self.add_future`.  These dependencies will be started/stopped/restarted with
the service.

``Worker``
----------

The worker can be used to start a Faust application, and performs tasks like
setting up logging, installs signal handlers and debugging tools etc.

``App``
-------

The app configures the Faust instance, and is the entry point for just about
everything that happens in a Faust instance.  Consuming/Producing messages,
starting streams and agents, etc.

The app is usually started by ``Worker``, but can also be started alone if
less operating system interaction is wanted, like if you want to embed Faust
in an application that already sets up signal handling and logging.

``Monitor``
-----------

The monitor is a feature-complete sensor that collects statistics about
the running instance.  The monitor data can be exposed by the web server.

``Producer``
------------

The producer is used to publish messages to Kafka topics, and is started
whenever necessary. The App will always starts this when a Faust instance is starting,
in anticipation of messages to be produced.

``Consumer``
------------

The Consumer is responsible for consuming messages from Kafka topics, to be
delivered to the streams.  It does not actually fetch messages (the
``Fetcher`` services does that), but it handles everything to do with
consumption, like managing topic subscriptions etc.

``Agent``
---------

Agents are also services, and any async function decorated using ``@app.agent``
will start with the app.

``Conductor``
------------------

The topic conductor manages topic subscriptions, and forward messages
from the Kafka consumer to the streams.

``app.stream(topic)`` will iterate over the topic: ``aiter(topic)``.
The conductor feeds messages into that iteration, so the stream
receives messages in the topic::

    async for event in stream(event async for event in topic)

``TableManager``
----------------

Manages tables, including recovery from changelog and caching table contents.
The table manager also starts the tables themselves, and acts as a registry of
tables in the Faust instance.

``Table``
---------

Any user defined table.

``Store``
---------

Every table has a separate store, the store describes how the table is stored
in this instance.  It could be stored in-memory (default), or as a RocksDB
key/value database if the data set is too big to fit in  memory.

``Stream``
----------

These are individual streams, started after everything is set up.

``Fetcher``
-----------

The Fetcher is the service that actually retrieves messages from the kafka
topic.  The fetcher forwards these messages to the TopicManager, which in
turns forwards it to Topic's and streams.

``Web``
-------------

This is a local web server started by the app (see :setting:`web_enable`
setting).





