# Benchmark Plan

## Purpose

Every Rust migration in Phase 3 must prove it's faster. This document defines the
benchmarks, how to run them, and what "faster" means.

---

## Benchmark Scenarios

### Scenario 1: Read and Iterate (the common case)

Open a MARC file, iterate all records, access title field. This is what most pymarc
users do.

```python
def bench_read_iterate(path):
    from rmarc import MARCReader
    with open(path, "rb") as fh:
        reader = MARCReader(fh)
        for record in reader:
            if record:
                _ = record.title
```

**Measures:** Full pipeline — file I/O, binary parsing, field construction, property
access.

### Scenario 2: Decode Only (isolates parsing)

Parse a pre-read byte blob into a Record. No file I/O.

```python
def bench_decode(data_blob):
    from rmarc import Record
    record = Record(data_blob)
```

**Measures:** `decode_marc()` in isolation.

### Scenario 3: Encode / Round-Trip

Parse then serialize back to bytes.

```python
def bench_roundtrip(data_blob):
    from rmarc import Record
    record = Record(data_blob)
    _ = record.as_marc()
```

**Measures:** `decode_marc()` + `as_marc()`.

### Scenario 4: MARC-8 Conversion

Convert MARC-8 encoded data to Unicode.

```python
def bench_marc8(marc8_lines):
    from rmarc import marc8_to_unicode
    for line in marc8_lines:
        _ = marc8_to_unicode(line)
```

**Measures:** `marc8_to_unicode()` throughput on the 1515-line test corpus.

### Scenario 5: Field Access (micro-benchmark)

Create a record, access subfields by code. Tests Python object overhead.

```python
def bench_field_access(record):
    for field in record.get_fields("245", "100", "650"):
        _ = field.get("a")
        _ = field.value()
```

**Measures:** Python object traversal. Baseline for "is Rust even relevant here?"

### Scenario 6: Bulk File Processing

Process a large (multi-MB) MARC file end-to-end. This is the "real world" scenario.

```python
def bench_bulk(path):
    from rmarc import MARCReader
    count = 0
    with open(path, "rb") as fh:
        for record in MARCReader(fh):
            if record:
                count += 1
    return count
```

**Measures:** Overall throughput in records/second.

---

## Test Data

### Small (ships with repo)
- `test_pymarc/test.dat` — 10 records, 6.4 KB
- `test_pymarc/marc.dat` — ~30 records, 20 KB
- `test_pymarc/marc8.dat` — 1 record with MARC-8 encoding

### Medium (generated)
- `bench_data/medium.dat` — ~1,000 records, ~1 MB
  Generated by repeating `test.dat` records.

### Large (generated)
- `bench_data/large.dat` — ~100,000 records, ~100 MB
  Generated by repeating `test.dat` records.

### MARC-8 corpus
- `test_pymarc/test_marc8.txt` — 1,515 MARC-8 strings for conversion benchmarks

### Data Generation Script

```python
#!/usr/bin/env python3
"""Generate benchmark data files by repeating test records."""
import os

def generate(src, dst, target_records):
    with open(src, "rb") as f:
        data = f.read()
    # each record in MARC is self-delimiting (ends with 0x1D)
    records = []
    pos = 0
    while pos < len(data):
        length = int(data[pos:pos+5])
        records.append(data[pos:pos+length])
        pos += length
    os.makedirs(os.path.dirname(dst), exist_ok=True)
    with open(dst, "wb") as f:
        for i in range(target_records):
            f.write(records[i % len(records)])

if __name__ == "__main__":
    generate("test_pymarc/test.dat", "bench_data/medium.dat", 1_000)
    generate("test_pymarc/test.dat", "bench_data/large.dat", 100_000)
    print("Generated bench_data/medium.dat and bench_data/large.dat")
```

---

## Benchmark Runner

Use `pytest-benchmark` for automated, statistically rigorous benchmarks.

### Installation

```bash
uv add --dev pytest-benchmark
```

### Benchmark File: `bench/bench_core.py`

```python
"""Core benchmarks for rmarc performance tracking."""
import pytest


@pytest.fixture
def test_dat():
    with open("test_pymarc/test.dat", "rb") as f:
        return f.read()


@pytest.fixture
def one_record_bytes(test_dat):
    """Extract first record from test.dat."""
    length = int(test_dat[:5])
    return test_dat[:length]


@pytest.fixture
def marc8_lines():
    with open("test_pymarc/test_marc8.txt", "rb") as f:
        return [line.strip(b"\r\n") for line in f if line.strip(b"\r\n")]


def test_bench_decode_single(benchmark, one_record_bytes):
    from rmarc import Record
    benchmark(Record, one_record_bytes)


def test_bench_roundtrip_single(benchmark, one_record_bytes):
    from rmarc import Record
    def roundtrip():
        r = Record(one_record_bytes)
        return r.as_marc()
    benchmark(roundtrip)


def test_bench_read_iterate_small(benchmark, test_dat):
    from rmarc import MARCReader
    def iterate():
        count = 0
        for record in MARCReader(test_dat):
            if record:
                count += 1
        return count
    result = benchmark(iterate)
    assert result == 10


def test_bench_marc8(benchmark, marc8_lines):
    from rmarc import marc8_to_unicode
    def convert_all():
        for line in marc8_lines:
            marc8_to_unicode(line)
    benchmark(convert_all)


def test_bench_field_access(benchmark, one_record_bytes):
    from rmarc import Record
    record = Record(one_record_bytes)
    def access():
        for field in record.get_fields():
            _ = field.value()
    benchmark(access)


def test_bench_as_dict(benchmark, one_record_bytes):
    from rmarc import Record
    record = Record(one_record_bytes)
    benchmark(record.as_dict)


def test_bench_str(benchmark, one_record_bytes):
    from rmarc import Record
    record = Record(one_record_bytes)
    benchmark(str, record)
```

### Running Benchmarks

```bash
# Run all benchmarks
uv run pytest bench/ --benchmark-only

# Compare against saved baseline
uv run pytest bench/ --benchmark-compare=baseline

# Save current results as baseline (do this BEFORE Rust migration)
uv run pytest bench/ --benchmark-save=baseline

# Save after Rust migration
uv run pytest bench/ --benchmark-save=after_step1

# Compare
uv run pytest bench/ --benchmark-compare=0001_baseline,0002_after_step1
```

---

## Comparison Against Original pymarc

To compare rmarc against pymarc directly, install pymarc in a separate venv and run
the same benchmarks with the import swapped:

```bash
# In a separate venv with pymarc installed:
BENCH_MODULE=pymarc pytest bench/ --benchmark-save=pymarc_baseline

# In the rmarc venv:
BENCH_MODULE=rmarc pytest bench/ --benchmark-save=rmarc_current

# Compare
pytest bench/ --benchmark-compare=pymarc_baseline,rmarc_current
```

The benchmark file can support this with:

```python
import os
MODULE = os.environ.get("BENCH_MODULE", "rmarc")
if MODULE == "pymarc":
    import pymarc as marc
else:
    import rmarc as marc
```

---

## What "Faster" Means

### Minimum bar for Rust migration

A Rust component replaces its Python equivalent only if:

1. **Correctness:** All 171 tests pass.
2. **Speed:** At least **2x faster** on the targeted benchmark scenario.
3. **No regression:** No benchmark scenario gets more than 10% slower.

### Targets by step

| Step | Benchmark | Target vs Python baseline |
|---|---|---|
| Step 1 (MARC codec) | decode single record | ≥5x faster |
| Step 1 (MARC codec) | roundtrip single record | ≥5x faster |
| Step 1 (MARC codec) | read+iterate 10 records | ≥3x faster |
| Step 2 (MARC-8) | convert 1515 lines | ≥10x faster |
| Step 3 (full reader) | read+iterate 10 records | ≥5x faster |
| Step 3 (full reader) | bulk 100K records | ≥5x faster |

### Reporting

After each Rust step, record results in a table:

```
| Benchmark              | Python (ms) | Rust (ms) | Speedup |
|------------------------|-------------|-----------|---------|
| decode_single          |       X.XX  |     X.XX  |   XX.Xx |
| roundtrip_single       |       X.XX  |     X.XX  |   XX.Xx |
| read_iterate_10        |       X.XX  |     X.XX  |   XX.Xx |
| marc8_1515_lines       |       X.XX  |     X.XX  |   XX.Xx |
| bulk_100k              |       X.XX  |     X.XX  |   XX.Xx |
```

---

## Profiling (When Benchmarks Aren't Enough)

### Python profiling

```bash
# cProfile on the bulk scenario
python -m cProfile -s cumtime -c "
from rmarc import MARCReader
with open('bench_data/large.dat', 'rb') as f:
    for r in MARCReader(f):
        pass
"
```

### Line-level profiling

```bash
uv add --dev line_profiler
kernprof -l -v bench/profile_decode.py
```

### Rust profiling

```bash
# flamegraph via cargo-flamegraph (Linux/macOS)
cargo flamegraph --bench marc_decode
```

---

## CI Integration

Add a benchmark job to CI that:

1. Runs `pytest bench/ --benchmark-json=bench_results.json`
2. Compares against the committed baseline
3. Fails if any benchmark regresses by more than 20%
4. Posts results as a PR comment (optional, via `pytest-benchmark` GitHub action)

---

## Directory Layout

```
bench/
├── bench_core.py        # main benchmark file
├── conftest.py          # shared fixtures
└── generate_data.py     # data generation script

bench_data/              # gitignored, generated locally
├── medium.dat
└── large.dat

.benchmarks/             # pytest-benchmark saved results (committed)
└── ...
```

Add to `.gitignore`:
```
bench_data/
```
