# Lafleur Developer Documentation: 05. State and Data Formats

### Introduction

During a fuzzing campaign, `lafleur` generates several files to store its persistent state, track statistics, and save its findings. This document provides a detailed reference for the structure and purpose of each of these files.

---

### `coverage/coverage_state.pkl`

This is the **most critical file** in the fuzzer. It is a binary file serialized using Python's `pickle` module and acts as the fuzzer's complete, persistent memory. To save space and reduce memory usage, it uses integer IDs to represent coverage items (uops, edges, and rare events). The file is saved atomically to prevent corruption.

It contains a single dictionary with the following top-level keys:

* **`uop_map`**, **`edge_map`**, **`rare_event_map`**: Dictionaries that map the string representation of a coverage item (e.g., `"('OPTIMIZED', '_LOAD_ATTR->_STORE_ATTR')"`) to a unique integer ID.
* **`next_id_map`**: A small dictionary that tracks the next available integer ID for each coverage type.
* **`global_coverage`**: The master "bitmap" of all unique coverage points ever seen. It maps the integer IDs of uops, edges, and rare events to their total hit counts.
* **`per_file_coverage`**: A dictionary where each key is a filename in the corpus (e.g., `"123.py"`) and the value is a rich metadata object describing that file. The metadata dictionary for each file contains the following keys:
    * `parent_id`: The filename of the parent test case that this file was mutated from. `None` for initial seed files.
    * `lineage_depth`: An integer representing how many generations of successful mutations led to this file.
    * `content_hash`: The SHA256 hash of the file's "core code," used for content duplicate detection.
    * `coverage_hash`: The SHA256 hash of the file's coverage profile, used to detect unique behaviors from non-deterministic code.
    * `discovery_time`: An ISO 8601 timestamp of when the file was added to the corpus.
    * `execution_time_ms`: The time in milliseconds the test case took to run during its discovery.
    * `file_size_bytes`: The size of the core code in bytes.
    * `total_finds`: A "fertility" score; the number of interesting children this parent has produced.
    * `subsumed_children_count`: An integer representing how many other corpus files have been subsumed (and pruned) by this one. This metric is generated by the `--prune-corpus` tool.
    * `is_sterile`: A boolean flag that is set to `True` if the parent has been mutated many times without producing any new discoveries.
    * `baseline_coverage`: A dictionary representing the coverage this specific file generated when it was discovered. It includes `uops`, `edges`, `rare_events`, and structural metrics like `trace_length` and `side_exits`.
    * `lineage_coverage_profile`: A set-based representation of the union of all coverage found in this file's entire ancestry.
    * **`mutation_info`**: A dictionary containing extra data about the mutation process that created this file. Crucially, it now stores **JIT Vitals** (e.g., `max_exit_density`, `zombie_traces`) used for differential scoring in future generations. Note that metrics like `max_exit_density` may be "clamped" here to ensure reachable targets for offspring.

---

### `coverage/mutator_scores.json`

This file is the persistent memory for the **adaptive mutation engine**. It is a human-readable JSON file that stores the learned effectiveness of each mutator and strategy, allowing the fuzzer to resume a campaign with its learned knowledge intact.

* **`scores`**: A dictionary mapping the name of each mutator or strategy (e.g., `"SideEffectInjector"`, `"havoc"`) to its current floating-point score. This score is increased on success and periodically decayed to favor recently successful mutators.
* **`attempts`**: A dictionary mapping each mutator or strategy name to an integer count of how many times it has been tried. This is used by the learning algorithm to give new mutators a "grace period" before their scores are used for selection.

---

### `fuzz_run_stats.json`

This is a human-readable JSON file that acts as a high-level dashboard for the entire fuzzing campaign. It tracks cumulative statistics across all sessions.

Key fields include:
* `total_sessions`: The total number of parents selected for mutation.
* `total_mutations`: The total number of child test cases executed.
* `crashes_found`: The total number of unique crashes discovered.
* `timeouts_found`: The total number of unique timeouts discovered.
* `divergences_found`: The total number of unique correctness divergences found.
* `new_coverage_finds`: The total number of times a new, unique, interesting test case was added to the corpus.
* `average_mutations_per_find`: An efficiency metric calculated as `total_mutations / new_coverage_finds`.
* `global_seed_counter`: A persistent counter used to ensure every mutation attempt across all runs has a unique, deterministic seed.
* `corpus_file_counter`: A persistent counter used to generate unique integer filenames for new corpus files.

---

### `logs/timeseries_... .jsonl`

This is a time-series log file in the **JSON Lines** (`.jsonl`) format, meaning each line in the file is a complete, independent JSON object.

At regular intervals (e.g., after every 10 new discoveries), the orchestrator takes a complete snapshot of the `fuzz_run_stats.json` file and appends it as a new line to this log. This creates a detailed historical record of the fuzzer's performance over the course of a single run.

---

### `logs/mutator_effectiveness.jsonl`

This is a new time-series log, also in the JSON Lines (`.jsonl`) format, that provides telemetry for the **adaptive learning engine**.

At the same time a snapshot is saved to the main time-series log, the `MutatorScoreTracker` saves a snapshot of its current state to this file. Each line is a JSON object containing:
* **`timestamp`**: An ISO 8601 timestamp for the snapshot.
* **`scores`**: The complete dictionary of all mutator and strategy scores at that moment.
* **`attempts`**: The complete dictionary of attempt counts.
* **`success_rates`**: A calculated dictionary showing the `score / attempts` ratio for each mutator, providing a clear view of its effectiveness.

This file is essential for post-run analysis to understand how the fuzzer's strategy evolved over time.

---

### Output Directories

The fuzzer saves its findings—the valuable test cases that reveal bugs—into several output directories.

* **`corpus/`**: Contains the main collection of "interesting" test cases that have discovered new JIT coverage.
* **`crashes/`**: Stores test cases that caused the Python interpreter to crash or that contained a keyword indicating a fatal error.
    * **Standard Mode:** Contains individual files like `crash_123.py`.
    * **Session Mode:** Contains **Session Bundles**—directories (e.g., `session_crash_123/`) that preserve the full context needed to reproduce the bug. These include:
        * `00_polluter.py`, `01_warmup.py`: The context scripts that set up the JIT state.
        * `02_attack.py`: The mutated child script that triggered the crash.
        * `reproduce.sh`: A generated script to run the bundle in the correct order.
* **`timeouts/`**: Stores test cases that caused the child process to hang. Large log files in this directory are automatically compressed with `zstd`.
* **`divergences/`**: When running in differential testing mode, this directory stores test cases that revealed a correctness bug.