PIDGIN-MONITOR(1)

NAME
pidgin-monitor - System-wide health monitor for running experiments

SYNOPSIS
pidgin monitor

DESCRIPTION
The monitor command provides a real-time, system-wide view of all active experiments. It displays a live dashboard that updates every 2 seconds, showing experiment progress, system load, convergence warnings, and completion estimates.

The monitor reads directly from JSONL event files, avoiding database locks and ensuring it doesn't interfere with running experiments. It provides a read-only view of system activity.

DISPLAY
The monitor shows:

  Experiment Overview

•   Experiment name and ID

•   Start time and duration

•   Number of conversations (completed/total)

•   Overall progress percentage

•   Estimated time to completion

  Active Conversations

•   Conversation IDs currently processing

•   Current turn number

•   Agent models in use

•   Latest convergence scores

  System Metrics

•   Total active conversations

•   System load indicator

•   Token usage rates

•   API response times

  Convergence Warnings

•   High convergence alerts (>0.95)

•   Stalled conversations

•   Repetition patterns detected

CONTROLS

Ctrl+C
    Exit the monitor

Space
    Pause/resume updates

R   Force refresh

FEATURES

  Lock-Free Operation
The monitor reads JSONL files directly without acquiring locks, ensuring it never blocks or slows down running experiments.

  Auto-Discovery
Automatically discovers all experiments in the output directory without requiring explicit configuration.

  Resource Efficient
Minimal CPU and memory usage through efficient file reading and caching strategies.

EXAMPLES

Start the system monitor:
    pidgin monitor

The monitor will display something like:
    ╭─ Pidgin System Monitor ────────────────────╮
    │ 3 Active Experiments │ Load: ████░░░░ 45%  │
    ╰────────────────────────────────────────────╯
    ▶ radiant-lake (experiment_a1b2c3d4)
      Started: 10:45 AM │ Duration: 0:23:45
      Progress: ████████░░ 78% (39/50)
      ETA: ~6 minutes

      Active: conv_12345678 (turn 18/20)
             Models: claude ↔ gpt-4
             Convergence: 0.89 ↑
    ▶ crystal-peak (experiment_e5f6g7h8)
      Started: 11:02 AM │ Duration: 0:06:30
      Progress: ██░░░░░░░░ 20% (2/10)
      ETA: ~26 minutes

    ⚠ High Convergence: conv_87654321 (0.97)
    [Updated: 11:08:45 AM]

ENVIRONMENT

PIDGIN_MONITOR_INTERVAL
    Update interval in seconds (default: 2)

PIDGIN_OUTPUT_DIR
    Directory to monitor (default: ./pidgin_output)

NOTES
The monitor is designed to be left running continuously during experiments. It has negligible performance impact and provides valuable real-time insight into experiment progress.

For detailed analysis of completed experiments, analyze the auto-generated Jupyter notebooks in the experiment directories.

SEE ALSO
pidgin(1), pidgin-run(1), pidgin-stop(1), pidgin-info(1)

AUTHOR
Nicholas Lange and contributors