# Benchmark Guide

browser-hybrid is optimized for low-latency, real-world interactions. To maintain this performance and provide transparent comparisons, the project includes a performance benchmarking suite.

The benchmark suite compares `browser-hybrid` against common automation tools:
*   [Playwright](https://playwright.dev/)
*   [Selenium](https://www.selenium.dev/)
*   [agent-browser](https://github.com/google/agent-browser) (if available)

## 🚀 Setup

Before running benchmarks, ensure you have the necessary dependencies:

```bash
# Install browser-hybrid in editable mode
pip install -e .

# Install competitors
pip install playwright && playwright install chromium
pip install selenium
# Optional: pip install agent-browser
```

Your primary Chrome instance **must** be running with `--remote-debugging-port=9222`.

## 📈 Running Benchmarks

### Quick Start

Run the entire benchmark suite with default settings:

```bash
python -m benchmarks.run_benchmarks
```

### Options

The benchmark runner supports various CLI flags for customization:

*   `--scenarios` : Filter specific scenarios (e.g., `--scenarios navigation click`)
*   `--runs` : Number of iterations per tool (default: 10)
*   `--results-dir` : Custom directory for JSON results
*   `--output` : Format (table, json, markdown)

## 🔍 Available Scenarios

We measure across several critical paths:

| Scenario | Description |
| :--- | :--- |
| `navigation` | Cold navigation to simple and JS-heavy pages. |
| `accessibility_tree` | Time to fetch and parse the semantic AX tree. |
| `click_performance` | Latency of text-based and ref-based clicks. |
| `form_fill` | Speed of typing and filling multiple complex forms. |
| `screenshot` | Time to capture both viewport and full-page screenshots. |

## 📊 Comparison Tool

Use the comparison script to generate a summary table comparing `browser-hybrid` to its competitors:

```bash
cd browser-hybrid
python benchmarks/compare.py results/*.json
```

This will output a formatted table showing the relative performance gains (or trade-offs) for each scenario.

## 🛠 Adding New Scenarios

To add a custom benchmark:
1.  Create a new file in `benchmarks/scenarios/`.
2.  Inherit from `BaseScenario`.
3.  Implement the `run()` and `teardown()` methods.
4.  Add the scenario to the `SCENARIOS` registry in `benchmarks/run_benchmarks.py`.

Example of a custom scenario:

```python
from benchmarks.scenarios.base import BaseScenario

class CustomScenario(BaseScenario):
    def run(self, browser, page_url):
        with self.timer("total_interaction"):
            browser.click_by_text("Special Interaction")
```

## 📉 Interpreting Results

*   `mean_ms` : Average latency. Focus on this for overall responsiveness.
*   `p95_ms` : 95th percentile latency. Critical for identifying inconsistent "jitter".
*   `success_rate` : Reliability measure. Ensure tools are actually completing the task.
