chaqum(1)

NAME
chaqum - [ˈkeɪkjuːm], the queue manager for chaotic job queues

SYNOPSIS
chaqum [-hv] [-e ENTRY] [-l LOG] DIRECTORY [ARGUMENT ...]

DESCRIPTION
Start a job manager with DIRECTORY as its job tree passing ARGUMENT ... as commandline parameters to the initial job script ENTRY. Then run the job manager as long as there are jobs running or repeats registered or it receives SIGINT, SIGHUP or SIGTERM.

The options are as follows:

-d, --daemonize PIDFILE[:USER:GROUP]
    Daemonize job manager. Use PIDFILE as lockfile to ensure only one instance is running. If USER and GROUP are specified run job manager under these credentials (this usually requires you to be root).

-e, --entry ENTRY
    Set the entry script to run as the first in the job tree. Defaults to 'entry'.

-h  Show a help message and exit.

-l, --log LOG
    Configure logging. Can be one of 'console' for logging to stderr or 'syslog[:[APPNAME][:FACILITY][:SOCKET_PATH]]' for sane defaults for logging to local Syslog. Otherwise the will be interpreted as the path to a JSON file containing the logging.config.dictConfig() format. Defaults to 'syslog' if running as a daemon 'console' otherwise.

-v  Turn on verbose logging. Can be repeated up to two times for even more verbosity.

JOB TREES
Job trees are simply directory trees with at least one executable somewhere inside. The only requirement for these executables is, that they either use the chaqum protocol to communicate with the job manager (see PROTOCOL if you're interested in the inner workings) or completly ignore file descriptors 3 and 4.

Having exclusivly executables of only the later type is unusual since such job trees can't make use of chaqum's job management, process spawning and inter-process messaging.

Most commonly job excutables will be written in an interpreted language and use one of the client libraries. References to those are in the SEE ALSO section.

The layout of the job tree directory as well as what exactly consitutes a single job (and thus executable within that job tree) is completly up to the user. There are simple examples demonstrating different features, available in the projects Git Repository: https://github.com/wagnerflo/chaqum/tree/main/examples.

LOGGING
chaqum captures everything the job processes write to stdout and stderr and redirects it to its interal logging mechanism (console, default local Syslog configuration or custom Python logging configuration: https://docs.python.org/3/library/logging.config.html?highlight=logging#dictionary-schema-details; see -l option). It generates one log message per line of output.

By default messages are logged with log level INFO: https://docs.python.org/3/library/logging.html#levels, but jobs can select the level by prepending the lines with one of
    C<US>
    E<US>
    W<US>
    I<US>
    D<US>

to log as level CRITICAL, ERROR, WARNING, INFO and DEBUG respectivly (in Syslog terminology these are LOG_CRIT, LOG_ERR, LOG_WARNING, LOG_INFO and LOG_DEBUG respectivly). <US> refers to the ASCII unit seperator character a.k.a. decimal 31, hex 0x1F or octal 037.

The client libraries supply functions that make logging using this protocol easy.

PROTOCOL
To allow jobs to communicate with the job manager chaqum implements a simple line based protocol. Communication is always initiated by the job; never by the server. Commands are handled one after the other and always follow the flow of: Job writes command to file descriptor 3, job reads managers reply by reading from file descriptor 4.

  Adding new jobs to be started

    > enqueue [-F] [-g GROUP] [-m MAXPROC] [-c MAXCPU] -- SCRIPT [ARGUMENT ...]<LF>
    < { S JOBIDENT<LF>,
        E<LF> }

  Adding new repeats to the scheduler

    > repeat [-c CRON] [-i INTERVAL] -- SCRIPT [ARGUMENT ...]<LF>
    < { S<LF>,
        E<LF> }

  Waiting for jobs to finish running

    > waitjobs [-t TIMEOUT] -- JOBIDENT [...]<LF>
    < { S JOBIDENT {T,N,EXITCODE} [...]<LF>,
        E<LF> }

  Forcibly terminating jobs and wait for them to be killed

    > killjobs [-t TIMEOUT] -- JOBIDENT [...]<LF>
    < { S JOBIDENT {T,N,EXITCODE} [...]<LF>,
        E<LF> }

  Queuing a inter-jobs message for delivery

    > sendmsg -- JOBIDENT LENGTH<LF>
      BYTES<LF>
    < { S MSGIDENT<LF>,
        E<LF> }

  Waiting for a message to be delivered

    > waitrecv [-t TIMEOUT] -- MSGIDENT [...]<LF>
    < { S MSGIDENT {T,R} [...]<LF>
        E<LF> }

  Receiving a message

    > recvmsg [-t TIMEOUT]<LF>
    < { S LENGTH<LF>
        BYTES<LF>,
        E<LF> }

SEE ALSO
chaqum.lib(3) — Python job library.

COPYRIGHT
Written by Florian Wagner.