# Browser Header Profiles Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Capture real browser headers during login via CDP, classify them into profiles (navigation/xhr/form), store on the session, and replay them automatically in programmatic requests — so every HTTP request looks like it came from the browser that created the session.

**Architecture:** The observe capture backends already record CDP `Network.requestWillBeSent` events with full request headers in `_request_map`. This feature adds: (1) pure functions to classify requests into profiles and extract header dicts, (2) a `get_header_profiles()` method on capture backends, (3) a `GraftpunkSession(requests.Session)` subclass that auto-applies profiles in `prepare_request()`, (4) integration in `session.py` to always capture headers during login and store `_gp_header_profiles` on the session before pickling, and (5) `load_session_for_api()` returning the new `GraftpunkSession`.

**Tech Stack:** Python 3.11+, requests, CDP Network events, dill/pickle, pytest

---

## Task 1: Header classification pure functions

**Files:**
- Create: `src/graftpunk/observe/headers.py`
- Test: `tests/unit/test_observe_headers.py`

**Step 1: Write the failing tests**

Create `tests/unit/test_observe_headers.py` with tests for:

```python
from graftpunk.observe.headers import classify_request, extract_header_profiles, EXCLUDED_HEADERS

# Test classify_request()
def test_classify_navigation_by_sec_fetch_mode():
    headers = {"sec-fetch-mode": "navigate", "Accept": "text/html"}
    assert classify_request(headers) == "navigation"

def test_classify_xhr_by_sec_fetch_mode():
    headers = {"sec-fetch-mode": "cors", "Accept": "application/json"}
    assert classify_request(headers) == "xhr"

def test_classify_xhr_by_x_requested_with():
    headers = {"X-Requested-With": "XMLHttpRequest", "Accept": "*/*"}
    assert classify_request(headers) == "xhr"

def test_classify_xhr_by_accept_json():
    headers = {"Accept": "application/json, text/javascript, */*; q=0.01"}
    assert classify_request(headers) == "xhr"

def test_classify_form_by_content_type_urlencoded():
    headers = {"Content-Type": "application/x-www-form-urlencoded"}
    assert classify_request(headers) == "form"

def test_classify_form_by_content_type_multipart():
    headers = {"Content-Type": "multipart/form-data; boundary=----"}
    assert classify_request(headers) == "form"

def test_classify_returns_none_for_image():
    headers = {"Accept": "image/webp,image/png,*/*;q=0.8"}
    assert classify_request(headers) is None

def test_classify_returns_none_for_css():
    headers = {"Accept": "text/css,*/*;q=0.1"}
    assert classify_request(headers) is None

def test_classify_sec_fetch_mode_takes_priority():
    """sec-fetch-mode: navigate overrides Content-Type: form."""
    headers = {"sec-fetch-mode": "navigate", "Content-Type": "application/x-www-form-urlencoded"}
    assert classify_request(headers) == "navigation"

# Test EXCLUDED_HEADERS
def test_excluded_headers_contains_cookie():
    assert "cookie" in EXCLUDED_HEADERS

def test_excluded_headers_contains_host():
    assert "host" in EXCLUDED_HEADERS

def test_excluded_headers_contains_pseudo_headers():
    assert ":authority" in EXCLUDED_HEADERS
    assert ":method" in EXCLUDED_HEADERS

# Test extract_header_profiles()
def test_extract_header_profiles_from_request_map():
    """Build profiles from a realistic _request_map."""
    request_map = {
        "1": {
            "url": "https://example.com/",
            "method": "GET",
            "headers": {
                "sec-fetch-mode": "navigate",
                "User-Agent": "Mozilla/5.0 ...",
                "Accept": "text/html,application/xhtml+xml",
                "Cookie": "session=abc",
                "Host": "example.com",
                ":authority": "example.com",
            },
        },
        "2": {
            "url": "https://example.com/api/data",
            "method": "GET",
            "headers": {
                "sec-fetch-mode": "cors",
                "User-Agent": "Mozilla/5.0 ...",
                "Accept": "application/json",
                "X-Requested-With": "XMLHttpRequest",
                "Cookie": "session=abc",
            },
        },
        "3": {
            "url": "https://example.com/logo.png",
            "method": "GET",
            "headers": {"Accept": "image/webp,image/png"},
        },
    }
    profiles = extract_header_profiles(request_map)
    assert "navigation" in profiles
    assert "xhr" in profiles
    assert "Cookie" not in profiles["navigation"]
    assert "Host" not in profiles["navigation"]
    assert ":authority" not in profiles["navigation"]
    assert profiles["navigation"]["User-Agent"] == "Mozilla/5.0 ..."
    assert profiles["xhr"]["X-Requested-With"] == "XMLHttpRequest"

def test_extract_header_profiles_first_match_wins():
    """Only the first matching request per profile is stored."""
    request_map = {
        "1": {"headers": {"sec-fetch-mode": "navigate", "User-Agent": "First"}},
        "2": {"headers": {"sec-fetch-mode": "navigate", "User-Agent": "Second"}},
    }
    profiles = extract_header_profiles(request_map)
    assert profiles["navigation"]["User-Agent"] == "First"

def test_extract_header_profiles_empty_map():
    assert extract_header_profiles({}) == {}
```

**Step 2: Run tests to verify they fail**

Run: `uv run pytest tests/unit/test_observe_headers.py -v`
Expected: FAIL with ImportError (module doesn't exist yet)

**Step 3: Write minimal implementation**

Create `src/graftpunk/observe/headers.py`:

```python
"""Header profile classification and extraction from CDP request data."""

from __future__ import annotations

from typing import Any

# Headers excluded from profiles — managed by requests or request-specific
EXCLUDED_HEADERS: frozenset[str] = frozenset({
    "cookie",
    "host",
    "content-length",
    "referer",
    "origin",
    ":authority",
    ":method",
    ":path",
    ":scheme",
})


def classify_request(headers: dict[str, str]) -> str | None:
    """Classify a request into a header profile based on its headers.

    Classification priority:
    1. sec-fetch-mode (most reliable — Chrome always sends it)
    2. Content-Type (for form detection)
    3. X-Requested-With / Accept heuristics (for XHR detection)

    Args:
        headers: Request headers dict (case-insensitive keys expected from CDP).

    Returns:
        Profile name ("navigation", "xhr", or "form"), or None if not classifiable.
    """
    # Normalize header keys to lowercase for comparison
    lower = {k.lower(): v for k, v in headers.items()}

    # 1. sec-fetch-mode (highest priority)
    sec_fetch = lower.get("sec-fetch-mode", "")
    if sec_fetch == "navigate":
        return "navigation"
    if sec_fetch == "cors":
        return "xhr"

    # 2. Content-Type for form detection
    content_type = lower.get("content-type", "")
    if "application/x-www-form-urlencoded" in content_type or "multipart/form-data" in content_type:
        return "form"

    # 3. XHR heuristics
    if lower.get("x-requested-with", "").lower() == "xmlhttprequest":
        return "xhr"
    accept = lower.get("accept", "")
    if "application/json" in accept:
        return "xhr"

    # 4. Navigation heuristic (Accept contains text/html without sec-fetch-mode)
    if "text/html" in accept:
        return "navigation"

    return None


def extract_header_profiles(
    request_map: dict[str, dict[str, Any]],
) -> dict[str, dict[str, str]]:
    """Extract header profiles from a capture backend's request map.

    Iterates all captured requests, classifies each, and stores the full
    header set (minus excluded headers) from the first matching request
    per profile.

    Args:
        request_map: The capture backend's _request_map dict. Each entry
            has at least a "headers" key with the request header dict.

    Returns:
        Dict mapping profile name to header dict, e.g.
        {"navigation": {"User-Agent": "...", ...}, "xhr": {...}}.
    """
    profiles: dict[str, dict[str, str]] = {}

    for _request_id, data in request_map.items():
        headers = data.get("headers", {})
        if not headers:
            continue

        profile = classify_request(headers)
        if profile is None or profile in profiles:
            continue

        # Store all headers except excluded ones
        filtered = {
            k: v for k, v in headers.items()
            if k.lower() not in EXCLUDED_HEADERS
        }
        profiles[profile] = filtered

    return profiles
```

**Step 4: Run tests to verify they pass**

Run: `uv run pytest tests/unit/test_observe_headers.py -v`
Expected: All PASS

**Step 5: Commit**

```bash
git add src/graftpunk/observe/headers.py tests/unit/test_observe_headers.py
git commit -m "$(cat <<'EOF'
feat: add header profile classification and extraction

Pure functions for classifying CDP requests into navigation/xhr/form
profiles and extracting filtered header dicts. First step toward
replaying real browser headers in programmatic sessions.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: stavxyz <hi@stav.xyz>
EOF
)"
```

---

## Task 2: Add `get_header_profiles()` to capture backends

**Files:**
- Modify: `src/graftpunk/observe/capture.py`
- Test: `tests/unit/test_observe_headers.py` (extend)

**Step 1: Write the failing tests**

Add to `tests/unit/test_observe_headers.py`:

```python
from graftpunk.observe.capture import SeleniumCaptureBackend, NodriverCaptureBackend

def test_selenium_backend_get_header_profiles():
    """SeleniumCaptureBackend exposes header profiles from its request map."""
    backend = SeleniumCaptureBackend.__new__(SeleniumCaptureBackend)
    backend._request_map = {
        "1": {"headers": {"sec-fetch-mode": "navigate", "User-Agent": "Test/1.0"}},
    }
    profiles = backend.get_header_profiles()
    assert "navigation" in profiles
    assert profiles["navigation"]["User-Agent"] == "Test/1.0"

def test_nodriver_backend_get_header_profiles():
    """NodriverCaptureBackend exposes header profiles from its request map."""
    backend = NodriverCaptureBackend.__new__(NodriverCaptureBackend)
    backend._request_map = {
        "1": {"headers": {"sec-fetch-mode": "cors", "Accept": "application/json"}},
    }
    profiles = backend.get_header_profiles()
    assert "xhr" in profiles

def test_backend_get_header_profiles_empty():
    backend = SeleniumCaptureBackend.__new__(SeleniumCaptureBackend)
    backend._request_map = {}
    assert backend.get_header_profiles() == {}
```

**Step 2: Run tests to verify they fail**

Run: `uv run pytest tests/unit/test_observe_headers.py::test_selenium_backend_get_header_profiles -v`
Expected: FAIL with AttributeError

**Step 3: Write minimal implementation**

Add `get_header_profiles()` method to both backend classes in `capture.py`:

In `CaptureBackend` protocol, add:
```python
def get_header_profiles(self) -> dict[str, dict[str, str]]:
    """Return header profiles classified from captured requests."""
    ...
```

In `SeleniumCaptureBackend`, add after `get_console_logs()`:
```python
def get_header_profiles(self) -> dict[str, dict[str, str]]:
    """Return header profiles classified from captured network requests."""
    from graftpunk.observe.headers import extract_header_profiles
    return extract_header_profiles(self._request_map)
```

In `NodriverCaptureBackend`, add after `get_console_logs()`:
```python
def get_header_profiles(self) -> dict[str, dict[str, str]]:
    """Return header profiles classified from captured network requests."""
    from graftpunk.observe.headers import extract_header_profiles
    return extract_header_profiles(self._request_map)
```

**Step 4: Run tests to verify they pass**

Run: `uv run pytest tests/unit/test_observe_headers.py -v`
Expected: All PASS

**Step 5: Commit**

```bash
git add src/graftpunk/observe/capture.py tests/unit/test_observe_headers.py
git commit -m "$(cat <<'EOF'
feat: add get_header_profiles() to capture backends

Both SeleniumCaptureBackend and NodriverCaptureBackend now expose
header profiles extracted from their request maps. Delegates to
the pure classification functions in observe.headers.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: stavxyz <hi@stav.xyz>
EOF
)"
```

---

## Task 3: Create `GraftpunkSession` subclass

**Files:**
- Create: `src/graftpunk/graftpunk_session.py`
- Test: `tests/unit/test_graftpunk_session.py`

**Step 1: Write the failing tests**

Create `tests/unit/test_graftpunk_session.py`:

```python
import requests
from graftpunk.graftpunk_session import GraftpunkSession

SAMPLE_PROFILES = {
    "navigation": {
        "User-Agent": "Mozilla/5.0 Test",
        "Accept": "text/html,application/xhtml+xml",
        "Accept-Language": "en-US,en;q=0.9",
    },
    "xhr": {
        "User-Agent": "Mozilla/5.0 Test",
        "Accept": "application/json",
        "X-Requested-With": "XMLHttpRequest",
    },
    "form": {
        "User-Agent": "Mozilla/5.0 Test",
        "Accept": "text/html",
        "Content-Type": "application/x-www-form-urlencoded",
    },
}


class TestProfileDetection:
    """Test auto-detection of profile from request characteristics."""

    def test_get_without_json_uses_navigation(self):
        session = GraftpunkSession(header_profiles=SAMPLE_PROFILES)
        req = requests.Request("GET", "https://example.com/page")
        prepared = session.prepare_request(req)
        assert prepared.headers["User-Agent"] == "Mozilla/5.0 Test"
        assert "text/html" in prepared.headers["Accept"]

    def test_post_with_json_uses_xhr(self):
        session = GraftpunkSession(header_profiles=SAMPLE_PROFILES)
        req = requests.Request("POST", "https://example.com/api", json={"key": "val"})
        prepared = session.prepare_request(req)
        assert prepared.headers.get("X-Requested-With") == "XMLHttpRequest"

    def test_post_with_data_string_uses_form(self):
        session = GraftpunkSession(header_profiles=SAMPLE_PROFILES)
        req = requests.Request("POST", "https://example.com/submit", data="key=val")
        prepared = session.prepare_request(req)
        assert "form" in prepared.headers.get("Content-Type", "").lower() or \
               prepared.headers.get("Accept") == "text/html"

    def test_explicit_accept_json_uses_xhr(self):
        session = GraftpunkSession(header_profiles=SAMPLE_PROFILES)
        req = requests.Request(
            "GET", "https://example.com/api",
            headers={"Accept": "application/json"},
        )
        prepared = session.prepare_request(req)
        assert prepared.headers.get("X-Requested-With") == "XMLHttpRequest"


class TestExplicitOverride:
    """Test explicit profile override mechanisms."""

    def test_headers_for_returns_profile_dict(self):
        session = GraftpunkSession(header_profiles=SAMPLE_PROFILES)
        xhr_headers = session.headers_for("xhr")
        assert xhr_headers["X-Requested-With"] == "XMLHttpRequest"

    def test_headers_for_unknown_profile_returns_empty(self):
        session = GraftpunkSession(header_profiles=SAMPLE_PROFILES)
        assert session.headers_for("nonexistent") == {}

    def test_gp_default_profile_overrides_detection(self):
        session = GraftpunkSession(header_profiles=SAMPLE_PROFILES)
        session.gp_default_profile = "xhr"
        req = requests.Request("GET", "https://example.com/page")
        prepared = session.prepare_request(req)
        assert prepared.headers.get("X-Requested-With") == "XMLHttpRequest"


class TestCallerHeadersPrecedence:
    """Caller-supplied headers must override profile headers."""

    def test_caller_headers_override_profile(self):
        session = GraftpunkSession(header_profiles=SAMPLE_PROFILES)
        req = requests.Request(
            "GET", "https://example.com/",
            headers={"User-Agent": "CustomAgent"},
        )
        prepared = session.prepare_request(req)
        assert prepared.headers["User-Agent"] == "CustomAgent"


class TestNoProfiles:
    """Sessions without profiles should work normally."""

    def test_no_profiles_no_injection(self):
        session = GraftpunkSession(header_profiles={})
        req = requests.Request("GET", "https://example.com/")
        prepared = session.prepare_request(req)
        # Should still have requests default User-Agent
        assert "python-requests" in prepared.headers.get("User-Agent", "").lower() or \
               "User-Agent" in prepared.headers

    def test_none_profiles_no_injection(self):
        session = GraftpunkSession(header_profiles=None)
        req = requests.Request("GET", "https://example.com/")
        prepared = session.prepare_request(req)
        assert prepared.headers is not None
```

**Step 2: Run tests to verify they fail**

Run: `uv run pytest tests/unit/test_graftpunk_session.py -v`
Expected: FAIL with ImportError

**Step 3: Write minimal implementation**

Create `src/graftpunk/graftpunk_session.py`:

```python
"""GraftpunkSession — requests.Session subclass with browser header replay."""

from __future__ import annotations

from typing import Any

import requests

from graftpunk.logging import get_logger

LOG = get_logger(__name__)


class GraftpunkSession(requests.Session):
    """A requests.Session that auto-applies captured browser header profiles.

    Header profiles are dicts of real browser headers captured during login,
    classified into profiles: "navigation", "xhr", "form". This session
    auto-detects which profile to apply based on request characteristics,
    or allows explicit override.

    Profile headers are applied as defaults — any headers explicitly passed
    by the caller take precedence and are not overwritten.
    """

    def __init__(
        self,
        header_profiles: dict[str, dict[str, str]] | None = None,
        **kwargs: Any,
    ) -> None:
        super().__init__(**kwargs)
        self._gp_header_profiles: dict[str, dict[str, str]] = header_profiles or {}
        self.gp_default_profile: str | None = None

    def headers_for(self, profile: str) -> dict[str, str]:
        """Get the header dict for a specific profile.

        Args:
            profile: Profile name ("navigation", "xhr", or "form").

        Returns:
            Header dict for the profile, or empty dict if not found.
        """
        return dict(self._gp_header_profiles.get(profile, {}))

    def _detect_profile(self, request: requests.Request) -> str:
        """Auto-detect the appropriate header profile for a request.

        Args:
            request: The request about to be sent.

        Returns:
            Profile name ("navigation", "xhr", or "form").
        """
        if self.gp_default_profile:
            return self.gp_default_profile

        method = (request.method or "GET").upper()

        # Check for explicit Accept: application/json in caller headers
        caller_headers = request.headers or {}
        caller_accept = caller_headers.get("Accept", "")
        if "application/json" in caller_accept:
            return "xhr"

        # POST/PUT/PATCH with json= → xhr
        if method in ("POST", "PUT", "PATCH") and request.json is not None:
            return "xhr"

        # POST with data= (string or dict) → form
        if method == "POST" and request.data:
            return "form"

        # Default: navigation
        return "navigation"

    def prepare_request(self, request: requests.Request, **kwargs: Any) -> requests.PreparedRequest:
        """Prepare a request with auto-detected profile headers.

        Profile headers are applied as session-level defaults so that
        any headers explicitly set by the caller take precedence.
        """
        if not self._gp_header_profiles:
            return super().prepare_request(request, **kwargs)

        profile_name = self._detect_profile(request)
        profile_headers = self._gp_header_profiles.get(profile_name)

        if not profile_headers:
            # Fall back to navigation if requested profile not available
            profile_headers = self._gp_header_profiles.get("navigation", {})

        if profile_headers:
            # Apply profile headers as session defaults (lowest priority).
            # Caller headers > request headers > session headers > profile headers.
            # We inject into self.headers temporarily, then restore.
            original_session_headers = dict(self.headers)
            for key, value in profile_headers.items():
                self.headers.setdefault(key, value)

            try:
                prepared = super().prepare_request(request, **kwargs)
            finally:
                # Restore original session headers
                self.headers.clear()
                self.headers.update(original_session_headers)

            return prepared

        return super().prepare_request(request, **kwargs)
```

**Step 4: Run tests to verify they pass**

Run: `uv run pytest tests/unit/test_graftpunk_session.py -v`
Expected: All PASS

**Step 5: Commit**

```bash
git add src/graftpunk/graftpunk_session.py tests/unit/test_graftpunk_session.py
git commit -m "$(cat <<'EOF'
feat: add GraftpunkSession with browser header replay

requests.Session subclass that auto-detects the appropriate header
profile (navigation/xhr/form) per outgoing request and applies
captured browser headers as defaults. Supports explicit overrides
via headers_for() and gp_default_profile.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: stavxyz <hi@stav.xyz>
EOF
)"
```

---

## Task 4: Capture headers during login, store on session

**Files:**
- Modify: `src/graftpunk/session.py` (add `_gp_header_profiles` to `__getstate__`/`__setstate__`)
- Modify: `src/graftpunk/plugins/login_engine.py` (always create capture backend during login, extract profiles before caching)
- Test: `tests/unit/test_session.py` (extend)
- Test: `tests/unit/test_login_engine.py` (extend)

**Step 1: Write the failing tests**

Add to `tests/unit/test_session.py`:

```python
def test_getstate_includes_gp_header_profiles():
    """__getstate__ should include _gp_header_profiles if set."""
    # Create a minimal session mock that has __getstate__
    session = create_test_session()  # use existing test helper
    session._gp_header_profiles = {"navigation": {"User-Agent": "Test"}}
    state = session.__getstate__()
    assert state.get("_gp_header_profiles") == {"navigation": {"User-Agent": "Test"}}

def test_setstate_restores_gp_header_profiles():
    """__setstate__ should restore _gp_header_profiles."""
    session = create_test_session()
    profiles = {"xhr": {"Accept": "application/json"}}
    session._gp_header_profiles = profiles
    state = session.__getstate__()

    new_session = create_test_session()
    new_session.__setstate__(state)
    assert getattr(new_session, "_gp_header_profiles", None) == profiles
```

Add to `tests/unit/test_login_engine.py`:

```python
def test_login_stores_header_profiles_on_session():
    """After successful login, session should have _gp_header_profiles set."""
    # Test that _generate_*_login extracts header profiles from capture backend
    # and sets them on the session before caching
    ...
```

Note: The exact test structure will depend on existing test patterns. The implementer should follow the existing mocking patterns in `test_login_engine.py` and `test_session.py`.

**Step 2: Run tests to verify they fail**

Run: `uv run pytest tests/unit/test_session.py::test_getstate_includes_gp_header_profiles -v`
Expected: FAIL

**Step 3: Implement**

**In `session.py` `__getstate__`** (both nodriver and selenium paths):
Add `state["_gp_header_profiles"] = getattr(self, "_gp_header_profiles", {})` before the return.

**In `session.py` `__setstate__`** (both paths):
Add `self._gp_header_profiles = state.get("_gp_header_profiles", {})` after restoring other attrs.

**In `login_engine.py` `_generate_nodriver_login`**:
After `await backend.stop_capture_async()` (which we need to add), before `cache_session()`:
```python
# Always capture headers during login for header profile extraction
from graftpunk.observe.capture import create_capture_backend
capture = create_capture_backend("nodriver", session.driver, get_tab=lambda: tab)
await capture.start_capture_async()
# ... existing login flow ...
await capture.stop_capture_async()
session._gp_header_profiles = capture.get_header_profiles()
```

Actually — login_engine already uses `BrowserSession` as a context manager, which calls `_start_observe()` and `_stop_observe()`. The issue is that `_start_observe()` only runs when `observe_mode != "off"`.

The cleanest approach: **always create a lightweight capture backend during login** purely for header extraction, independent of the observe mode. Add it directly in the login functions after all network activity is done but before caching.

In `_generate_nodriver_login`, before `cache_session()`:
```python
# Extract header profiles from capture backend if available
capture = getattr(session, "_capture", None)
if capture is not None:
    session._gp_header_profiles = capture.get_header_profiles()
else:
    session._gp_header_profiles = {}
```

But wait — the context manager only starts capture when `observe_mode != "off"`, and login doesn't currently pass `observe_mode="full"`. We need to ensure headers are always captured. Two options:

**Option A**: Always force `observe_mode="headers"` during login (the issue spec says "Login always implicitly enables headers mode"). This requires the granular observe modes refactor.

**Option B** (simpler, do now): Create a separate lightweight capture backend in the login functions just for header extraction. This doesn't require refactoring the observe system.

**Go with Option B** — it's self-contained and the granular observe modes can be added later as a separate feature.

In both `_generate_nodriver_login` and `_generate_selenium_login`, after the browser session is opened but before the login flow, create a capture backend and start it. After login completes, extract profiles.

For nodriver in `_generate_nodriver_login`:
```python
async with BrowserSession(backend="nodriver", headless=False) as session:
    # Start header capture for profile extraction
    from graftpunk.observe.capture import create_capture_backend
    header_capture = create_capture_backend(
        "nodriver", session.driver, get_tab=lambda: tab
    )
    await header_capture.start_capture_async()

    tab = await session.driver.get(f"{base_url}{login_url}")
    # ... existing login flow ...

    # Extract header profiles before caching
    session._gp_header_profiles = header_capture.get_header_profiles()

    await session.transfer_nodriver_cookies_to_session()
    cache_session(session, plugin.session_name)
```

For selenium in `_generate_selenium_login`:
```python
with BrowserSession(backend="selenium", headless=False) as session:
    # Selenium capture backend parses perf log on stop, so just create it
    from graftpunk.observe.capture import create_capture_backend
    header_capture = create_capture_backend("selenium", session.driver)
    header_capture.start_capture()

    session.driver.get(f"{base_url}{login_url}")
    # ... existing login flow ...

    # Stop capture to parse perf log, then extract profiles
    header_capture.stop_capture()
    session._gp_header_profiles = header_capture.get_header_profiles()

    session.transfer_driver_cookies_to_session()
    cache_session(session, plugin.session_name)
```

**Step 4: Run tests to verify they pass**

Run: `uv run pytest tests/unit/test_session.py tests/unit/test_login_engine.py -v`
Expected: All PASS

**Step 5: Commit**

```bash
git add src/graftpunk/session.py src/graftpunk/plugins/login_engine.py tests/unit/test_session.py tests/unit/test_login_engine.py
git commit -m "$(cat <<'EOF'
feat: capture browser headers during login for profile storage

Login engine now creates a capture backend during login to record
CDP network events. After login completes, header profiles are
extracted and stored as _gp_header_profiles on the session before
pickling. BrowserSession.__getstate__/__setstate__ preserve the
profiles through serialization.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: stavxyz <hi@stav.xyz>
EOF
)"
```

---

## Task 5: Wire up `load_session_for_api()` to return `GraftpunkSession`

**Files:**
- Modify: `src/graftpunk/cache.py`
- Test: `tests/unit/test_cache.py` (extend)

**Step 1: Write the failing tests**

Add to `tests/unit/test_cache.py`:

```python
def test_load_session_for_api_returns_graftpunk_session(tmp_path, monkeypatch):
    """load_session_for_api should return GraftpunkSession when profiles exist."""
    from graftpunk.graftpunk_session import GraftpunkSession

    # Mock load_session to return a session with _gp_header_profiles
    mock_session = MagicMock()
    mock_session.cookies = requests.cookies.RequestsCookieJar()
    mock_session.headers = {"User-Agent": "test"}
    mock_session._gp_header_profiles = {"navigation": {"User-Agent": "Browser"}}
    monkeypatch.setattr("graftpunk.cache.load_session", lambda name: mock_session)

    api_session = load_session_for_api("test")
    assert isinstance(api_session, GraftpunkSession)
    assert api_session._gp_header_profiles == {"navigation": {"User-Agent": "Browser"}}

def test_load_session_for_api_no_profiles_returns_graftpunk_session(tmp_path, monkeypatch):
    """load_session_for_api returns GraftpunkSession even without profiles."""
    mock_session = MagicMock()
    mock_session.cookies = requests.cookies.RequestsCookieJar()
    mock_session.headers = {"User-Agent": "test"}
    mock_session._gp_header_profiles = {}
    monkeypatch.setattr("graftpunk.cache.load_session", lambda name: mock_session)

    api_session = load_session_for_api("test")
    assert isinstance(api_session, GraftpunkSession)
    assert api_session._gp_header_profiles == {}
```

**Step 2: Run tests to verify they fail**

Run: `uv run pytest tests/unit/test_cache.py::test_load_session_for_api_returns_graftpunk_session -v`
Expected: FAIL (returns plain requests.Session)

**Step 3: Implement**

Modify `load_session_for_api()` in `cache.py`:

```python
def load_session_for_api(name: str) -> requests.Session:
    """..."""
    try:
        browser_session = load_session(name)
    except FileNotFoundError as exc:
        LOG.error("session_not_found_for_api", name=name)
        raise SessionNotFoundError(
            f"No cached session found for '{name}'. Please login first."
        ) from exc

    # Extract header profiles if present
    header_profiles = getattr(browser_session, "_gp_header_profiles", {})

    # Create GraftpunkSession with header profiles for auto-detection
    from graftpunk.graftpunk_session import GraftpunkSession

    api_session = GraftpunkSession(header_profiles=header_profiles)

    # Copy cookies from browser session
    if hasattr(browser_session, "cookies"):
        api_session.cookies = browser_session.cookies
        LOG.debug(
            "copied_cookies_from_session",
            cookie_count=len(browser_session.cookies),
        )

    # Copy headers from browser session
    if hasattr(browser_session, "headers"):
        api_session.headers.update(browser_session.headers)
        LOG.debug("copied_headers_from_session")

    LOG.info(
        "created_api_session_from_cached_session",
        name=name,
        has_header_profiles=bool(header_profiles),
        profile_count=len(header_profiles),
    )
    return api_session
```

**Step 4: Run tests to verify they pass**

Run: `uv run pytest tests/unit/test_cache.py -v`
Expected: All PASS

**Step 5: Commit**

```bash
git add src/graftpunk/cache.py tests/unit/test_cache.py
git commit -m "$(cat <<'EOF'
feat: load_session_for_api returns GraftpunkSession with header profiles

load_session_for_api() now returns a GraftpunkSession that auto-applies
captured browser headers on every request. Sessions cached before this
feature work normally (empty profiles, no header injection).

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: stavxyz <hi@stav.xyz>
EOF
)"
```

---

## Task 6: Remove `DEFAULT_BROWSER_HEADERS` from `http_commands.py`

**Files:**
- Modify: `src/graftpunk/cli/http_commands.py`
- Test: `tests/unit/test_http_commands.py` (update)

**Step 1: Write the failing tests**

Update existing tests that reference `DEFAULT_BROWSER_HEADERS` to verify the dict no longer exists and that `_make_request` no longer applies hardcoded headers (since `GraftpunkSession` handles it automatically now).

Add test:
```python
def test_default_browser_headers_removed():
    """DEFAULT_BROWSER_HEADERS should no longer exist."""
    import graftpunk.cli.http_commands as mod
    assert not hasattr(mod, "DEFAULT_BROWSER_HEADERS")
```

**Step 2: Run tests to verify they fail**

Run: `uv run pytest tests/unit/test_http_commands.py::test_default_browser_headers_removed -v`
Expected: FAIL (attribute still exists)

**Step 3: Implement**

In `http_commands.py`:
1. Delete the `DEFAULT_BROWSER_HEADERS` dict (lines 34-44)
2. In `_make_request`, remove the `browser_headers` parameter and the block that applies `DEFAULT_BROWSER_HEADERS`:
   ```python
   # DELETE these lines:
   if browser_headers:
       for key, value in DEFAULT_BROWSER_HEADERS.items():
           session.headers.setdefault(key, value)
   ```
3. In `_http_command`, keep the `--no-browser-headers` flag but change its behavior: when set, clear the `_gp_header_profiles` on the session so no profile headers are injected:
   ```python
   if no_browser_headers and hasattr(session, '_gp_header_profiles'):
       session._gp_header_profiles = {}
   ```

**Step 4: Run tests to verify they pass**

Run: `uv run pytest tests/ -v`
Expected: All PASS

**Step 5: Commit**

```bash
git add src/graftpunk/cli/http_commands.py tests/unit/test_http_commands.py
git commit -m "$(cat <<'EOF'
refactor: remove DEFAULT_BROWSER_HEADERS in favor of captured profiles

gp http now relies on GraftpunkSession's auto-detected header profiles
from the cached session instead of a hardcoded browser headers dict.
--no-browser-headers still works by clearing the session's profiles.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: stavxyz <hi@stav.xyz>
EOF
)"
```

---

## Task 7: Add brotli as core dependency

**Files:**
- Modify: `pyproject.toml`
- Test: verify import works

**Step 1: Add brotli to dependencies**

In `pyproject.toml`, add to the `dependencies` list:
```toml
"brotli>=1.1.0",
```

**Step 2: Sync deps**

Run: `uv sync`

**Step 3: Verify import**

Run: `uv run python -c "import brotli; print(brotli.__version__)"`
Expected: prints version number

**Step 4: Commit**

```bash
git add pyproject.toml uv.lock
git commit -m "$(cat <<'EOF'
feat: add brotli as core dependency

Browsers send Accept-Encoding: br (brotli) by default. With header
profiles now replaying real browser Accept-Encoding headers, the
requests library needs brotli installed to decompress responses.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: stavxyz <hi@stav.xyz>
EOF
)"
```

---

## Task 8: Final verification

**Step 1: Run full test suite**

```bash
uv run pytest tests/ -v
```
Expected: All tests pass

**Step 2: Run quality checks**

```bash
uvx ruff check .
uvx ruff format --check .
uvx ty check src/
```
Expected: Clean (8 pre-existing ty diagnostics acceptable)

**Step 3: Push**

```bash
git push
```
