💻 SysAdmin Network — COGNAC documentation

# 💻 SysAdmin Network

The Multi-agent SysAdmin problem is a widely used benchmark in the study of decision-making over networked systems. Originally introduced by Guestrin et al. in 2002 as a single-agent factored MDP benchmark [Guestrin et al.], it was later extended into a multi-agent formulation to evaluate coordinated reinforcement learning algorithms [Guestrin et al.]. Over the years, it has remained a standard reference in the field, with recent works reaffirming its relevance [Bargiacchi et al.], [Bianchi et al.].

This environment provides a modern and open-source implementation of the multi-agent version of the SysAdmin problem, specifically designed for multi-agent reinforcement learning (MARL). It maintains the original intent of testing structure-aware planning and coordination under uncertainty, while ensuring compatibility with modern MARL libraries.

## Environment Description

The environment models a network of computers performing tasks. Each computer (agent) can be in one of several health states and may also be processing a task. Over time, machines may become faulty, slowing down task completion, or dead, making the task impossible to complete. Faults can spread probabilistically to neighboring computers. At each timestep, agents may choose to reboot their machine, which resets it to a working state (good) with high probability, but also discards progress on any active task.

This decentralized setting models a Partially Observable Markov Decision Process (PoMDP), where coordination between agents is critical for maintaining system-wide performance and limiting fault propagation.

Illustration for one state update in the SysAdmin Network problem.

### Graph Topology

As in the Binary Consensus environment, the underlying graph defines the network topology. It determines which agents (computers) are neighbors and hence how faults can spread between them.

## State Space

Each agent’s state is defined by two categorical variables:

- Health status: one of good, faulty, or dead

- Task status: one of idle, loaded, or successful

Formally, the global joint state at time \(t\) is an element of:

\[S(t) \in \{\text{good}, \text{faulty}, \text{dead}\}^N \times \{\text{idle}, \text{loaded}, \text{successful}\}^N\]

This results in a total state space size of \(9^N\).

## Action Space

At each timestep, every agent selects one of two discrete actions:

- Do nothing (continue current operation)

- Reboot the machine (resets health to good with high probability but loses current task progress)

## Objective

The overall goal is to maximize the number of successfully completed tasks over time. This objective can be framed as either a finite-horizon or infinite-horizon cumulative reward problem. Performance is closely tied to how effectively agents collaborate and leverage the graph structure to mitigate cascading faults.

By modeling local and global trade-offs in a structured environment, the Multi-agent SysAdmin problem serves as an effective testbed for evaluating decentralized and semi-centralized MARL strategies under uncertainty and partial observability.

## Environment

class cognac.env.SysAdmin.env.SysAdminNetworkEnvironment(adjacency_matrix: ~numpy.ndarray, max_steps: int = 100, show_neighborhood_state: bool = True, reward_class: ~cognac.core.BaseReward.BaseReward = <class 'cognac.env.SysAdmin.rewards.SysAdminDefaultReward'>, is_global_reward: bool = False, base_arrival_rate: float = 0.5, base_fail_rate: float = 0.1, dead_rate_multiplier: float = 0.2, base_success_rate: float = 0.3, faulty_success_rate: float = 0.1)

  Bases: `ParallelEnv`

  Multi-agent environment simulating a network of machines managed by agents, based on the “SysAdmin” problem setting.

  Each agent controls one machine, which can be in states representing its operational condition (good, faulty, dead) and job state (idle, loaded, successful). Machines can influence each other’s failure rates based on a given network adjacency matrix.

  The environment follows a discrete timestep progression with reboot actions, task completions, faults propagation, and rewards computed per step.

  ### Parameters

  adjacency_matrixnp.ndarray

    Square matrix of shape (n_agents, n_agents) representing network connections. Entry (i, j) indicates influence of agent j on agent i. Must have zeros on diagonal initially (no self influence).

  max_stepsint, default=100

    Maximum number of timesteps before the environment is terminated.

  show_neighborhood_statebool, default=True

    If True, agents observe not only their own state but also their neighbors’ states.

  reward_classBaseReward subclass, default=SysAdminDefaultReward

    Class to compute the rewards. Should be derived from BaseReward.

  is_global_rewardbool, default=False

    If True, a single global reward is returned to all agents.

  base_arrival_ratefloat, default=0.5

    Probability of new job arriving for an available machine at each step.

  base_fail_ratefloat, default=0.1

    Base failure rate for machines without external influence.

  dead_rate_multiplierfloat, default=0.2

    Multiplier for probability of machine becoming dead influenced by faulty neighbors.

  base_success_ratefloat, default=0.3

    Probability that a working loaded machine completes its task successfully.

  faulty_success_ratefloat, default=0.1

    Probability that a faulty loaded machine completes its task successfully.

  ### Attributes

  adjacency_matrixnp.ndarray

    Original adjacency matrix representing the network structure.

  adjacency_matrix_probnp.ndarray

    Processed stochastic matrix of influence probabilities with self-failures included.

  n_agentsint

    Number of agents (machines) in the environment.

  possible_agentslist of int

    List of all agent IDs.

  statenp.ndarray

    Current state array of shape (n_agents, 2) with machine status and job state.

  timestepint

    Current timestep counter.

  max_stepsint

    Maximum allowed timesteps before termination.

  rewardBaseReward

    Instance of the reward class used to calculate rewards.

  is_global_rewardbool

    Flag indicating if rewards are global or individual.

  neighboring_masksnp.ndarray (bool)

    Boolean mask matrix indicating which agents observe each other.

  ### Methods

  reset(seed=None, options=None)

    Reset environment state and return initial observations.

  step(actions)

    Perform one timestep given agents’ actions; update state and return results.

  get_obs()

    Get the current observations dictionary for all agents.

  render()

    Print the current state of the environment (to be implemented).

  observation_space(agent)

    Return the observation space object for the specified agent.

  action_space(agent)

    Return the action space object for the specified agent.

  Warning

  Methods prefixed with an underscore (_) are for internal use only and should not be called directly by users.

  _available_mask()

    Warning

    Internal use.

    Boolean mask for machines that are available to receive new jobs (working and currently idle).

    #### Returns

    np.ndarray

      Boolean array where True indicates available machines.

  _check_adjacency_matrix()

    Warning

    Internal use.

    Validates the adjacency matrix to ensure: - Diagonal elements are zero (no self influence initially). - All entries are probabilities in [0, 1].

    #### Raises

    AssertionError

      If any of the validation checks fail.

  _done_mask()

    Warning

    Internal use.

    Boolean mask for machines that have completed their jobs.

    #### Returns

    np.ndarray

      Boolean array where True indicates machines done with their tasks.

  _faulty_loaded_mask()

    Warning

    Internal use.

    Boolean mask for machines that are faulty and currently loaded with a job.

    #### Returns

    np.ndarray

      Boolean array where True indicates faulty and loaded machines.

  _faulty_working_mask()

    Warning

    Internal use.

    Boolean mask for machines currently in faulty or dead state.

    #### Returns

    np.ndarray

      Boolean array where True indicates machine is faulty or dead (state code != 0).

  _scale_adjacency_matrix()

    Warning

    Internal use.

    Scales the adjacency_matrix_prob so that the maximum row or column sum is at most 1. This ensures the resulting matrix can be interpreted as a stochastic influence matrix.

  _working_loaded_mask()

    Warning

    Internal use.

    Boolean mask for machines that are working and currently loaded with a job.

    #### Returns

    np.ndarray

      Boolean array where True indicates working and loaded machines.

  _working_mask()

    Warning

    Internal use.

    Boolean mask for machines currently in working state.

    #### Returns

    np.ndarray

      Boolean array where True indicates the machine is working (state code 0).

  action_space(agent) → Discrete

    Return the action space for the given agent.

    Actions are discrete:

    - 0: do nothing

    - 1: reboot machine

    #### Parameters

    agentint

      Agent identifier.

    #### Returns

    gymnasium.spaces.Discrete

      Action space for the agent.

  get_obs() → dict

    Retrieve the current observation for each agent.

    Observations consist of each agent’s own machine state vector.

    #### Returns

    dict

      Mapping from agent IDs to their observations (np.ndarray of shape (2,)).

  metadata: dict[str, Any] = {'name': 'sysadmin_environment_v0'}
  observation_space(agent) → MultiDiscrete

    Return the observation space for the given agent.

    The observation space is a MultiDiscrete space describing machine status and job state with discrete values:

    - Machine status: 0=good, 1=faulty, 2=dead

    - Job state: 0=idle, 1=loaded, 2=successful

    #### Parameters

    agentint

      Agent identifier.

    #### Returns

    gymnasium.spaces.MultiDiscrete

      Observation space for the agent.

  render()

    Render the current environment state.

    Currently prints the raw state array.

    #### Notes

    This method is a placeholder and should be implemented to provide a graphical or structured visualization of the environment.

  reset(seed=None, options=None)

    Reset the environment to its initial state.

    #### Parameters

    seedint or None, optional

      Seed for the random number generator for reproducibility.

    optionsdict or None, optional

      Additional options for environment reset (currently unused).

    #### Returns

    observationsdict

      Dictionary mapping agent IDs to their initial observations.

    infosdict

      Dictionary mapping agent IDs to info dictionaries (empty in this implementation).

  state()

    Returns the state.

    State returns a global view of the environment appropriate for centralized training decentralized execution methods like QMIX

  step(actions)

    Advance the environment by one timestep given agents’ actions.

    #### Parameters

    actionsdict

      Dictionary mapping agent IDs to their actions. Each action is an integer: 0 for “do nothing”, 1 for “reboot”.

    #### Returns

    observationsdict

      Dictionary mapping agent IDs to their new observations.

    rewardsdict

      Dictionary mapping agent IDs to their rewards for this step.

    terminationsdict

      Dictionary mapping agent IDs to termination flags (bool).

    truncationsdict

      Dictionary mapping agent IDs to truncation flags (bool).

    infosdict

      Dictionary mapping agent IDs to info dictionaries (empty in this implementation).

    #### Notes

    - Rebooted machines are reset to working and idle state.

    - Machines working on tasks may complete successfully based on probabilities.

    - Faulty and dead states evolve probabilistically influenced by network

        neighbors.

    - Rewards are computed via the configured reward class.

    - Environment terminates after max_steps.

## Rewards

class cognac.env.SysAdmin.rewards.SysAdminDefaultReward

  Bases: `BaseReward`

  _abc_impl = <_abc._abc_data object>