PIDGIN-METRICS(7)

NAME
pidgin-metrics - Metrics calculated by Pidgin for conversation analysis

DESCRIPTION
Pidgin calculates a comprehensive set of metrics for each conversation turn, stored in the database for analysis. Metrics are calculated in real-time for operational needs (convergence monitoring) and comprehensively during import for research analysis.

METRIC CATEGORIES

  Message-Level Metrics (Per Agent)
Each metric is prefixed with a_ for agent A or b_ for agent B.

message_length
    Number of characters in the message.

word_count
    Number of space-separated words.

vocabulary_size
    Number of unique words used.

sentence_count
    Number of sentences detected.

avg_word_length
    Average length of words in characters.

avg_sentence_length
    Average number of words per sentence.

unique_word_ratio
    Ratio of unique words to total words (vocabulary diversity).

hapax_legomena_ratio
    Ratio of words used only once (lexical richness indicator).

question_count
    Number of questions in the message.

exclamation_count
    Number of exclamatory sentences.

emoji_count
    Number of emoji characters.

punctuation_diversity
    Number of distinct punctuation marks used.

formality_score
    Estimated formality level (0=informal, 1=formal).

  Information Theory Metrics

word_entropy
    Shannon entropy of word distribution (bits).

character_entropy
    Shannon entropy of character distribution.

compression_ratio
    Ratio indicating information density.

  Linguistic Complexity

lexical_diversity
    Type-token ratio adjusted for text length.

self_repetition
    Measure of phrase repetition within the message.

repeated_ngrams
    Count of repeated n-gram patterns.

  Convergence Metrics (Paired)

vocabulary_overlap
    Jaccard similarity of vocabulary between agents.
    overlap = |V_a ∩ V_b| / |V_a ∪ V_b|

length_convergence
    Similarity in message lengths.
    convergence = 1 - |len_a - len_b| / max(len_a, len_b)

structural_similarity
    Similarity in linguistic structure (punctuation, capitalization patterns).

style_similarity
    Composite measure of stylistic alignment.

mutual_mimicry
    Average of directional mimicry scores.

mimicry_score_a_to_b
    How much agent A mimics agent B's vocabulary.

mimicry_score_b_to_a
    How much agent B mimics agent A's vocabulary.

overall_convergence
    Weighted average of convergence metrics used for stopping criteria.
    convergence = 0.4 * vocabulary_overlap +
                  0.3 * structural_similarity +
                  0.2 * length_convergence +
                  0.1 * mimicry_score

  Cumulative Metrics

cumulative_vocabulary_a/b
    Total unique words used by each agent so far.

cumulative_convergence
    Vocabulary overlap considering all previous turns.

repetition_ratio
    Ratio of repeated content across the conversation.

  Placeholder Metrics
The following metrics are stored as placeholders (0.0) to maintain schema compatibility. They require heavy ML libraries and can be calculated post-hoc:

semantic_similarity
    Requires sentence transformers for embedding-based similarity.

sentiment_convergence
    Requires sentiment analysis libraries (TextBlob, VADER).

emotional_intensity
    Requires emotion lexicons and analysis tools.

topic_consistency
    Requires topic modeling (LDA, BERT-based models).

formality_convergence
    Requires detailed linguistic analysis.

CALCULATION DETAILS

  Vocabulary Overlap
Uses Jaccard similarity coefficient:
    def calculate_vocabulary_overlap(words_a, words_b):
        vocab_a = set(words_a)
        vocab_b = set(words_b)
        intersection = vocab_a & vocab_b
        union = vocab_a | vocab_b
        return len(intersection) / len(union) if union else 0

  Mimicry Score
Measures adoption of partner's vocabulary:
    def calculate_mimicry(current_words, partner_history):
        partner_vocab = set(word for turn in partner_history
                           for word in turn)
        current_vocab = set(current_words)
        shared = current_vocab & partner_vocab
        return len(shared) / len(current_vocab) if current_vocab else 0

  Word Entropy
Shannon entropy of word frequency distribution:
    H(X) = -Σ p(x) * log2(p(x))
Where p(x) is the probability of word x in the message.

DATABASE SCHEMA
Metrics are stored in the conversation_turns table with appropriate data types:

•   Counts: TINYINT or SMALLINT

•   Ratios/scores: DOUBLE (0.0-1.0)

•   Text metrics: INTEGER

•   Identifiers: VARCHAR

ANALYSIS QUERIES

  Average convergence by turn
    SELECT turn_number, AVG(overall_convergence) as avg_conv
    FROM conversation_turns
    WHERE experiment_id = ?
    GROUP BY turn_number
    ORDER BY turn_number;

  High convergence conversations
    SELECT conversation_id, MAX(overall_convergence) as peak
    FROM conversation_turns
    GROUP BY conversation_id
    HAVING peak > 0.9
    ORDER BY peak DESC;

  Vocabulary growth
    SELECT turn_number,
           AVG(cumulative_vocabulary_a) as avg_vocab_a,
           AVG(cumulative_vocabulary_b) as avg_vocab_b
    FROM conversation_turns
    WHERE experiment_id = ?
    GROUP BY turn_number;

NOTES
All metrics are calculated deterministically from message text, ensuring reproducibility. The computational cost is O(n) for most metrics where n is message length.

Metrics are designed to capture both individual message characteristics and paired conversation dynamics, enabling analysis of how communication patterns evolve over time.

SEE ALSO
pidgin(1), pidgin-run(1), pidgin-info(1)

REFERENCES
Shannon, C. E. (1948). "A Mathematical Theory of Communication"

Pickering, M. J., & Garrod, S. (2004). "Toward a mechanistic psychology of dialogue"

AUTHOR
Nicholas Lange and contributors