# Browser Hybrid — Testing Strategy

## Overview

Browser-hybrid requires a browser with remote debugging enabled. This creates unique testing challenges compared to typical Python packages.

## Test Categories

### 1. Unit Tests (Mocked)

**Purpose:** Test internal logic without Chrome.

**Location:** `tests/test_*.py`

**Run:** `pytest tests/ -v`

**Status:** ✅ All passing (46 tests)

**Coverage:**
- AXNode tree parsing and traversal
- Error classes and context
- Connection pool logic (no real WebSocket)
- HTTP API client (mocked urllib)

### 2. Integration Tests (Real Chrome)

**Purpose:** Test against actual Chrome DevTools Protocol.

**Location:** `tests/test_*.py` with `@pytest.mark.integration`

**Run:** `pytest tests/ -m integration`

**Requirement:** Chrome running with `--remote-debugging-port=9222`

**Status:** ⚠️ Skipped by default (need Chrome)

**What they test:**
- Real tab creation/closure
- Real accessibility tree parsing
- Real navigation and clicks
- Real screenshots

### 3. Smoke Tests (Quick Validation)

**Purpose:** Fast check that core functionality works.

**Location:** `tests/test_smoke.py`

**Run:** `pytest tests/test_smoke.py -v`

**Requirement:** Chrome with debug port

**What they test:**
- Can connect to Chrome
- Can create/navigate/close tab
- Can get accessibility tree
- Can take screenshot

### 4. Live Tests (Full Workflow)

**Purpose:** End-to-end scenarios like real usage.

**Location:** `tests/test_live.py`

**Run:** `pytest tests/test_live.py -v --live`

**Requirement:** Chrome with debug port + internet

**What they test:**
- Login flow (simulated on test site)
- Form filling
- Multi-page navigation
- Error recovery

### 5. Performance Tests

**Purpose:** Measure latency and connection reuse.

**Location:** `tests/test_perf.py`

**Run:** `pytest tests/test_perf.py -v --benchmark`

**What they measure:**
- Time per operation
- Connection reuse vs new connection
- Memory usage over time

---

## Running Tests

### Unit Tests Only (No Chrome needed)

```bash
pytest tests/ -v -m "not integration"
```

### With Chrome Debug Port

```bash
# Start Chrome with debugging
~/scripts/chrome-debug.sh

# Run all tests
pytest tests/ -v

# Run only integration tests
pytest tests/ -m integration -v

# Run smoke tests
pytest tests/test_smoke.py -v
```

### CI Pipeline

The CI runs:
1. **Lint job** - ruff check, ruff format, mypy
2. **Test job** - Unit tests on Python 3.10/3.11/3.12 with coverage
3. **Integration job** - Smoke tests with headless Chrome (main branch only)

To test locally before pushing:
```bash
# Quick check (no Chrome)
make test-unit

# Full check (Chrome required)
make test
```

---

## Test Fixtures

### Chrome Fixture (`tests/conftest.py`)

```python
import pytest
from browser_hybrid import Browser

@pytest.fixture
def chrome():
    """Provide a Browser connected to Chrome."""
    try:
        browser = Browser()
        yield browser
    except ConnectionError:
        pytest.skip("Chrome not running with --remote-debugging-port=9222")
    finally:
        browser.close()

@pytest.fixture
def tab(chrome):
    """Provide a fresh tab for each test."""
    tab = chrome.new_tab("about:blank")
    yield tab
    chrome.close_tab(tab.id)
```

### Test Site Fixture

For live tests that need a predictable target:
- Use `https://example.com` for simple navigation
- Use `https://httpbin.org` for form/HTTP testing
- Use local HTML files for controlled scenarios

---

## Testing Anti-Patterns

### ❌ Don't

- Test against real accounts (Gmail, GitHub)
- Hardcode timeouts
- Assume Chrome is available
- Ignore connection errors

### ✅ Do

- Use fixtures with skip markers
- Set reasonable timeouts
- Clean up tabs after tests
- Test error paths

---

## EVAL Integration

**Status:** Not implemented yet.

**What EVAL would provide:**
- Standardized test scenarios
- Pass/fail metrics
- Regression tracking
- A/B comparison of changes

**How it would work:**
```bash
# Hypothetical future
browser-hybrid eval --suite form-fill
browser-hybrid eval --suite navigation
browser-hybrid eval --all
```

For now, manual testing of real-world scenarios is required before releases.

---

## Development Workflow

### Before Committing

```bash
# 1. Format and lint
make lint

# 2. Run unit tests
make test-unit

# 3. (Optional) Run with Chrome
make test
```

### Before Release

```bash
# 1. All unit tests pass
pytest tests/ -v

# 2. Smoke tests pass (Chrome required)
pytest tests/test_smoke.py -v

# 3. Manual test against real sites
python3 -c "
from browser_hybrid import Browser
b = Browser()
b.goto('https://github.com')
print(b.snapshot()[:500])
"

# 4. Update version, changelog
```

---

## Current Gaps

| Gap | Priority | Solution |
|-----|----------|----------|
| No live tests | High | Create `test_live.py` |
| No smoke tests | High | Create `test_smoke.py` |
| No performance tests | Medium | Create `test_perf.py` |
| No EVAL system | Low | Future work |
| CI skips integration | Expected | Document local testing |