# gsearch-mcp-cli Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Build a Python CLI and MCP server for Google Search (regular + AI Mode) and page fetching, using headless Chrome via CDP.

**Architecture:** Async CDP over WebSocket to a headless Chromium browser with a persistent warmed profile. Single shared backend (`shared.py`) serves both Click CLI and FastMCP server. Pydantic models for all data. Rate-limited, SSRF-safe, with crash recovery.

**Tech Stack:** Python 3.10+, hatchling/uv, click/rich-click, fastmcp, websockets (async), httpx, pydantic, trafilatura, html-to-markdown, rich, orjson

**Spec:** `docs/specs/2026-03-21-gsearch-design.md`

---

## Dependency Graph & Execution Order

Tasks are numbered for reference but should be executed in risk-first order. The dependency graph:

```
Task 1 (Scaffold)
└── Task 2 (Constants, Exceptions, Models)
    ├── Task 3 (Config) ─────────────────────────┐
    ├── Task 4 (Browser Detection/Launch) ────────┤
    │   └── Task 5 (CDP Connection) ──────────────┤
    │       ├── Task 7 (Warmup) ──────────────────┤
    │       └── Task 8 (Extractor) ───────────────┤
    ├── Task 6 (Rate Limiter) ← NO browser dep ──┤
    └── Task 9 (Fetcher) ← needs Task 5 for CDP ─┤
                                                  │
    Task 10 (Shared Backend) ← depends on ALL ────┘
    ├── Task 11 (CLI Main + Search) ──┐
    ├── Task 12a (CLI: ai + fetch) ───┤ parallel OK
    ├── Task 12b (CLI: setup/cfg/dr) ─┤
    ├── Task 12c (CLI: skill) ────────┤
    ├── Task 13 (MCP Server) ─────────┤
    └── Task 14 (Skill File) ─────────┘
        └── Task 15 (Integration Tests)
```

**Recommended execution order** (front-loads highest-risk tasks):
```
1 → 2 → 4 → 5 → [3, 6 parallel] → [7, 8, 9 parallel] → 10 → [11, 13, 14 parallel] → 12a → 12b → 12c → 15
```

**Parallelization groups** (for multi-agent execution):
- Group B: Tasks 3, 4, 6 (after Task 2)
- Group D: Tasks 7, 8, 9 (after Task 5)
- Group E: Tasks 11, 13, 14 (after Task 10)

**Rollback checkpoints:**
| After Task | State | Risk if next task fails |
|-----------|-------|------------------------|
| Task 2 | Models + constants compile | Nothing — no runtime code yet |
| Task 5 | CDP connects to Chrome | If CDP fails, Tasks 7-10 are blocked |
| Task 10 | Full backend works | CLI/MCP have interfaces but no working backend |
| Task 13 | Complete application | Integration tests may reveal issues |

---

## File Structure

```
src/gsearch_mcp_cli/
├── __init__.py                  # version
├── constants.py                 # browser paths, ports, URLs, timeouts, SSRF blocklists
├── exceptions.py                # error hierarchy
├── models.py                    # Pydantic models (SearchResult, AIResult, etc.)
├── config.py                    # config load/save/get/set with validation
├── core/
│   ├── __init__.py
│   ├── browser.py               # detect browsers, launch Chrome, port map, health check, shutdown
│   ├── cdp.py                   # async CDP: connect, message dispatcher, tab context manager
│   ├── warmup.py                # profile warmup with event-based waits + cookie verification
│   ├── extractor.py             # JS extraction scripts for search results + AI Mode
│   ├── fetcher.py               # URL validation (SSRF, IPv4+IPv6) + page fetch + MD conversion
│   └── rate_limiter.py          # async token bucket with asyncio.Lock
├── shared.py                    # search(), ai(), fetch() — single backend with asyncio.Lock init guard
├── cli/
│   ├── __init__.py
│   ├── main.py                  # Click group, global flags, default command
│   ├── ai_doc.py                # --docs flag content
│   ├── search_cmd.py            # gsearch search / gsearch "query"
│   ├── ai_cmd.py                # gsearch ai
│   ├── fetch_cmd.py             # gsearch fetch
│   ├── setup_cmd.py             # gsearch setup
│   ├── config_cmd.py            # gsearch config
│   ├── doctor_cmd.py            # gsearch doctor
│   └── skill_cmd.py             # gsearch skill
├── mcp/
│   ├── __init__.py              # run_server()
│   ├── server.py                # FastMCP tools with sanitize_error()
│   └── __main__.py              # python -m gsearch_mcp_cli.mcp
└── data/
    ├── SKILL.md
    ├── AGENTS_SECTION.md
    └── references/
        ├── commands.md
        └── mcp-tools.md
tests/
├── conftest.py                  # shared fixtures: tmp dirs, mock CDP, mock tabs, HTML fixtures
├── test_models.py
├── test_config.py
├── test_browser.py
├── test_cdp.py
├── test_warmup.py
├── test_extractor.py
├── test_fetcher.py
├── test_rate_limiter.py
├── test_shared.py
├── test_cli.py
├── test_cli_commands.py         # tests for ai, fetch, setup, config, doctor, skill commands
├── test_mcp.py
└── test_integration.py
```

---

### Task 1: Project Scaffold

**Files:**
- Create: `pyproject.toml`
- Create: `src/gsearch_mcp_cli/__init__.py`
- Create: `tests/conftest.py`
- Create: `README.md`
- Create: `.gitignore`

- [ ] **Step 1: Create pyproject.toml**

```toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "gsearch-mcp-cli"
version = "0.1.0"
description = "Google Search CLI and MCP server for AI agents"
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
dependencies = [
    "click>=8.0",
    "rich-click>=1.7",
    "rich>=13.0",
    "pydantic>=2.0",
    "orjson>=3.9",
    "websockets>=12.0",
    "httpx>=0.27",
    "trafilatura>=1.8",
    "html-to-markdown>=1.0",
    "fastmcp>=0.1",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-asyncio>=0.23",
    "pytest-mock>=3.12",
    "pytest-cov>=4.1",
]

[project.scripts]
gsearch = "gsearch_mcp_cli.cli.main:main"
gsearch-mcp = "gsearch_mcp_cli.mcp:run_server"

[tool.hatch.build.targets.wheel]
packages = ["src/gsearch_mcp_cli"]

[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
markers = [
    "integration: end-to-end tests requiring browser and network",
]

[tool.ruff]
target-version = "py310"
line-length = 120

[tool.ruff.lint]
select = ["E", "F", "I", "W"]
```

- [ ] **Step 2: Create .gitignore**

```gitignore
__pycache__/
*.pyc
*.egg-info/
dist/
build/
.venv/
.pytest_cache/
.ruff_cache/
*.egg
.coverage
htmlcov/
```

- [ ] **Step 3: Create directory structure**

```bash
mkdir -p src/gsearch_mcp_cli/{cli,core,mcp,data/references}
mkdir -p tests
touch src/gsearch_mcp_cli/__init__.py
touch src/gsearch_mcp_cli/{cli,core,mcp}/__init__.py
```

- [ ] **Step 4: Create __init__.py with version**

```python
# src/gsearch_mcp_cli/__init__.py
__version__ = "0.1.0"
```

- [ ] **Step 5: Create conftest.py with comprehensive fixtures**

```python
# tests/conftest.py
import asyncio
import json
from unittest.mock import AsyncMock, MagicMock
import pytest


@pytest.fixture
def tmp_config_dir(tmp_path):
    """Temporary config directory for tests."""
    config_dir = tmp_path / ".gsearch-mcp-cli"
    config_dir.mkdir()
    return config_dir


@pytest.fixture
def mock_cdp_connection():
    """Mock CDPConnection that returns canned CDP responses."""
    conn = AsyncMock()
    conn.send = AsyncMock(return_value={"result": {}})
    conn.close = AsyncMock()
    return conn


@pytest.fixture
def mock_tab():
    """Mock Tab with canned evaluate() results."""
    tab = AsyncMock()
    tab.navigate = AsyncMock()
    tab.evaluate = AsyncMock(return_value={})
    tab.wait_for_load = AsyncMock()
    tab.override_ua = AsyncMock()
    tab.get_cookies = AsyncMock(return_value=[])

    async def _aenter(self):
        return tab
    async def _aexit(self, *exc):
        pass
    tab.__aenter__ = _aenter.__get__(tab)
    tab.__aexit__ = _aexit.__get__(tab)
    return tab


@pytest.fixture
def mock_browser_cdp(mock_tab):
    """Mock BrowserCDP that returns mock tabs."""
    browser = AsyncMock()
    browser.create_tab = AsyncMock(return_value=mock_tab)
    browser.close = AsyncMock()
    return browser


@pytest.fixture
def sample_search_html():
    """Load sample Google search results HTML fixture."""
    # Returns dict mimicking tab.evaluate() output for search extraction
    return {
        "results": [
            {"position": 1, "title": "Test Result", "url": "https://example.com",
             "snippet": "A test snippet", "source": "example.com"}
        ],
        "ai_overview": None,
        "people_also_ask": ["What is test?"],
        "related_searches": ["test related"],
    }


@pytest.fixture
def sample_ai_html():
    """Load sample Google AI Mode HTML fixture."""
    return {
        "synthesis": "AI-generated synthesis text for testing.",
        "sections": [{"heading": "Overview", "content": "Content text", "items": ["item1", "item2"]}],
        "sources": [{"title": "Source", "url": "https://example.com", "domain": "example.com", "snippet": ""}],
    }
```

- [ ] **Step 6: Create minimal README.md**

```markdown
# gsearch-mcp-cli

Google Search CLI and MCP server for AI agents. No-account, pseudonymous search.

## Install

\`\`\`bash
pip install gsearch-mcp-cli
\`\`\`

## Quick Start

\`\`\`bash
gsearch "what is kubernetes"
gsearch ai "explain docker architecture"
gsearch fetch "https://kubernetes.io/docs/"
\`\`\`
```

- [ ] **Step 7: Install in dev mode and verify**

```bash
cd /Users/jbendavi/dev_projects/google-search-mcp-cli
uv venv && source .venv/bin/activate
uv pip install -e ".[dev]"
python -c "import gsearch_mcp_cli; print(gsearch_mcp_cli.__version__)"
```

Expected: `0.1.0`

- [ ] **Step 8: Commit**

```bash
git init
git add -A
git commit -m "feat: project scaffold with pyproject.toml, .gitignore, ruff, and test fixtures"
```

---

### Task 2: Constants, Exceptions, Models

**Files:**
- Create: `src/gsearch_mcp_cli/constants.py`
- Create: `src/gsearch_mcp_cli/exceptions.py`
- Create: `src/gsearch_mcp_cli/models.py`
- Create: `tests/test_models.py`

- [ ] **Step 1: Write test for models**

```python
# tests/test_models.py
from gsearch_mcp_cli.models import SearchResult, OrganicResult, Source, AIResult, Section, FetchResult

def test_search_result_serialization():
    result = SearchResult(
        query="test",
        results=[OrganicResult(position=1, title="Test", url="https://example.com", snippet="A test", source="example.com")],
    )
    data = result.model_dump()
    assert data["query"] == "test"
    assert len(data["results"]) == 1
    assert data["ai_overview"] is None

def test_ai_result_with_sections():
    result = AIResult(
        query="test",
        synthesis="Test synthesis",
        sections=[Section(heading="Intro", content="Content", items=["item1"])],
        sources=[Source(title="Src", url="https://example.com", domain="example.com")],
    )
    json_str = result.model_dump_json()
    assert "Test synthesis" in json_str

def test_fetch_result():
    result = FetchResult(url="https://example.com", title="Example", content="# Hello", word_count=1)
    assert result.metadata == {}
```

- [ ] **Step 2: Run test to verify it fails**

```bash
pytest tests/test_models.py -v
```

Expected: FAIL — modules don't exist yet

- [ ] **Step 3: Implement constants.py**

```python
# src/gsearch_mcp_cli/constants.py
from pathlib import Path
import platform

APP_NAME = "gsearch-mcp-cli"
CONFIG_DIR = Path.home() / f".{APP_NAME}"
CHROME_PROFILE_DIR = CONFIG_DIR / "chrome-profile"
CONFIG_FILE = CONFIG_DIR / "config.json"
PORT_MAP_FILE = CONFIG_DIR / "chrome-port-map.json"

GOOGLE_SEARCH_URL = "https://www.google.com/search"
GOOGLE_AI_MODE_PARAMS = {"udm": "50"}
GOOGLE_HOMEPAGE = "https://www.google.com/"
GOOGLE_ABOUT = "https://about.google/"
GOOGLE_PREFERENCES = "https://www.google.com/preferences"

REQUIRED_WARMUP_COOKIES = {"NID", "AEC"}

# Timeouts (seconds)
TIMEOUT_CHROME_LAUNCH = 15
TIMEOUT_PAGE_LOAD = 30
TIMEOUT_JS_EVAL = 10
TIMEOUT_SEARCH = 15
TIMEOUT_AI_MODE = 30
TIMEOUT_FETCH_BROWSER = 30
TIMEOUT_FETCH_HTTP = 15
TIMEOUT_WARMUP = 30
TIMEOUT_TAB_CLEANUP = 5  # max time for Tab.__aexit__ cleanup

# Rate limiting
DEFAULT_RATE_LIMIT = 2.0  # seconds between searches
DEFAULT_BURST = 3
DEFAULT_MAX_TABS = 5

# Browser candidates per platform
def _macos_browser_candidates() -> list[tuple[str, str]]:
    candidates = []
    entries = [
        ("Google Chrome", "Google Chrome.app/Contents/MacOS/Google Chrome"),
        ("Brave Browser", "Brave Browser.app/Contents/MacOS/Brave Browser"),
        ("Microsoft Edge", "Microsoft Edge.app/Contents/MacOS/Microsoft Edge"),
        ("Chromium", "Chromium.app/Contents/MacOS/Chromium"),
        ("Vivaldi", "Vivaldi.app/Contents/MacOS/Vivaldi"),
        ("Opera", "Opera.app/Contents/MacOS/Opera"),
        ("Opera GX", "Opera GX.app/Contents/MacOS/Opera GX"),
    ]
    for name, rel in entries:
        candidates.append((name, str(Path("/Applications") / rel)))
        candidates.append((name, str(Path.home() / "Applications" / rel)))
    return candidates

def _linux_browser_candidates() -> list[tuple[str, str]]:
    import shutil
    names = [
        ("Google Chrome", "google-chrome"),
        ("Google Chrome", "google-chrome-stable"),
        ("Chromium", "chromium"),
        ("Chromium", "chromium-browser"),
        ("Chromium (Snap)", "/snap/bin/chromium"),
        ("Brave Browser", "brave-browser"),
        ("Microsoft Edge", "microsoft-edge-stable"),
        ("Vivaldi", "vivaldi"),
    ]
    candidates = []
    for display, cmd in names:
        if cmd.startswith("/"):
            if Path(cmd).exists():
                candidates.append((display, cmd))
        else:
            path = shutil.which(cmd)
            if path:
                candidates.append((display, path))
    return candidates

def _windows_browser_candidates() -> list[tuple[str, str]]:
    import os
    program_files = os.environ.get("ProgramFiles", "C:\\Program Files")
    program_files_x86 = os.environ.get("ProgramFiles(x86)", "C:\\Program Files (x86)")
    local_app_data = os.environ.get("LOCALAPPDATA", "")
    entries = [
        ("Google Chrome", "Google\\Chrome\\Application\\chrome.exe"),
        ("Brave Browser", "BraveSoftware\\Brave-Browser\\Application\\brave.exe"),
        ("Microsoft Edge", "Microsoft\\Edge\\Application\\msedge.exe"),
        ("Vivaldi", "Vivaldi\\Application\\vivaldi.exe"),
    ]
    candidates = []
    for name, rel in entries:
        for base in [program_files, program_files_x86, local_app_data]:
            if not base:
                continue
            full = Path(base) / rel
            if full.exists():
                candidates.append((name, str(full)))
    return candidates

def get_browser_candidates() -> list[tuple[str, str]]:
    system = platform.system()
    if system == "Darwin":
        return _macos_browser_candidates()
    elif system == "Linux":
        return _linux_browser_candidates()
    elif system == "Windows":
        return _windows_browser_candidates()
    return []

# Chrome launch flags
CHROME_BASE_FLAGS = [
    "--headless=new",
    "--no-first-run",
    "--no-default-browser-check",
    "--disable-extensions",
    "--disable-sync",
    "--disable-background-networking",
    "--remote-allow-origins=http://localhost",
    "--window-size=1280,900",
    "--disable-blink-features=AutomationControlled",
    "--disable-features=TranslateUI",
]

CHROME_PLATFORM_FLAGS = {
    "Linux": ["--disable-gpu", "--disable-dev-shm-usage", "--no-sandbox"],
    "Darwin": ["--disable-gpu"],
    "Windows": ["--disable-gpu"],
}

# SSRF blocklist — IPv4
PRIVATE_IPV4_RANGES = [
    "10.", "172.16.", "172.17.", "172.18.", "172.19.",
    "172.20.", "172.21.", "172.22.", "172.23.", "172.24.",
    "172.25.", "172.26.", "172.27.", "172.28.", "172.29.",
    "172.30.", "172.31.", "192.168.", "127.", "0.",
    "169.254.",
]

# SSRF blocklist — IPv6 (normalized forms)
PRIVATE_IPV6_PREFIXES = [
    "::1",       # loopback
    "fe80:",     # link-local
    "fc00:",     # unique local
    "fd00:",     # unique local
    "::ffff:127.", "::ffff:10.", "::ffff:192.168.",  # IPv4-mapped
]

BLOCKED_HOSTS = {"metadata.google.internal", "169.254.169.254"}
ALLOWED_SCHEMES = {"http", "https"}
```

- [ ] **Step 4: Implement exceptions.py**

```python
# src/gsearch_mcp_cli/exceptions.py
class GSearchError(Exception):
    """Base exception for gsearch-mcp-cli."""

class BrowserNotFoundError(GSearchError):
    """No supported Chromium browser found."""

class CDPConnectionError(GSearchError):
    """Failed to connect to Chrome via CDP."""

class SearchBlockedError(GSearchError):
    """Google blocked the search (unusual traffic)."""

class ProfileWarmupError(GSearchError):
    """Profile warmup failed."""

class FetchError(GSearchError):
    """Failed to fetch URL."""

class SSRFBlockedError(FetchError):
    """URL blocked for security (SSRF prevention)."""

class GSearchTimeoutError(GSearchError):
    """Operation timed out."""

class RateLimitError(GSearchError):
    """Rate limit exceeded."""

class PortExhaustedError(GSearchError):
    """All CDP ports in range are occupied."""

class ExtractionError(GSearchError):
    """Failed to extract results from DOM (selectors may be outdated)."""

class ConfigError(GSearchError):
    """Invalid configuration."""
```

> **Note:** `GSearchTimeoutError` is used instead of `TimeoutError` to avoid shadowing Python's built-in `TimeoutError` and `asyncio.TimeoutError`.

- [ ] **Step 5: Implement models.py**

```python
# src/gsearch_mcp_cli/models.py
from pydantic import BaseModel

class Source(BaseModel):
    title: str
    url: str
    snippet: str = ""
    domain: str

class AIOverview(BaseModel):
    text: str
    sources: list[Source] = []

class OrganicResult(BaseModel):
    position: int
    title: str
    url: str
    snippet: str
    source: str

class SearchResult(BaseModel):
    query: str
    ai_overview: AIOverview | None = None
    results: list[OrganicResult] = []
    people_also_ask: list[str] = []
    related_searches: list[str] = []

class Section(BaseModel):
    heading: str
    content: str
    items: list[str] = []

class AIResult(BaseModel):
    query: str
    synthesis: str
    sections: list[Section] = []
    sources: list[Source] = []

class FetchResult(BaseModel):
    url: str
    title: str
    content: str
    word_count: int
    metadata: dict = {}
```

- [ ] **Step 6: Run tests**

```bash
pytest tests/test_models.py -v
```

Expected: 3 PASSED

- [ ] **Step 7: Commit**

```bash
git add -A
git commit -m "feat: add constants, exceptions, and Pydantic models"
```

---

### Task 3: Config Management

**Files:**
- Create: `src/gsearch_mcp_cli/config.py`
- Create: `tests/test_config.py`

- [ ] **Step 1: Write tests**

```python
# tests/test_config.py
import json
import pytest
from gsearch_mcp_cli.config import Config
from gsearch_mcp_cli.exceptions import ConfigError

def test_config_defaults(tmp_config_dir):
    cfg = Config(config_dir=tmp_config_dir)
    assert cfg.get("browser") == "auto"
    assert cfg.get("rate_limit") == 2.0
    assert cfg.get("max_tabs") == 5

def test_config_set_and_get(tmp_config_dir):
    cfg = Config(config_dir=tmp_config_dir)
    cfg.set("browser", "brave")
    assert cfg.get("browser") == "brave"
    # Verify persistence
    cfg2 = Config(config_dir=tmp_config_dir)
    assert cfg2.get("browser") == "brave"

def test_config_show(tmp_config_dir):
    cfg = Config(config_dir=tmp_config_dir)
    cfg.set("lang", "fr")
    all_cfg = cfg.show()
    assert all_cfg["lang"] == "fr"
    assert all_cfg["browser"] == "auto"

def test_config_invalid_key_raises(tmp_config_dir):
    cfg = Config(config_dir=tmp_config_dir)
    with pytest.raises(ConfigError):
        cfg.set("invalid_key", "value")

def test_config_invalid_type_raises(tmp_config_dir):
    cfg = Config(config_dir=tmp_config_dir)
    with pytest.raises(ConfigError):
        cfg.set("rate_limit", "not_a_number")

def test_config_corrupt_json_falls_back_to_defaults(tmp_config_dir):
    config_file = tmp_config_dir / "config.json"
    config_file.write_text("{corrupt json!!")
    cfg = Config(config_dir=tmp_config_dir)
    assert cfg.get("browser") == "auto"
```

- [ ] **Step 2: Run test to verify it fails**

```bash
pytest tests/test_config.py -v
```

- [ ] **Step 3: Implement config.py**

Config class that reads/writes `~/.gsearch-mcp-cli/config.json` with defaults for all keys (browser, lang, region, max_results, json_output, rate_limit, max_tabs, timeout). Validates keys against known set and values against expected types. Raises `ConfigError` for invalid keys or types. Falls back to defaults on corrupt JSON.

- [ ] **Step 4: Run tests, verify pass**

```bash
pytest tests/test_config.py -v
```

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat: add config management with validation and persistence"
```

---

### Task 4: Browser Detection and Launch

**Files:**
- Create: `src/gsearch_mcp_cli/core/browser.py`
- Create: `tests/test_browser.py`

- [ ] **Step 1: Write tests**

Test `find_browser()` returns a path on the current system (mock `Path.exists` / `shutil.which` for cross-platform unit tests). Test `build_chrome_args()` returns expected flags including platform-specific ones. Test `parse_port_from_stderr()` parses port correctly. Test `ensure_browser()` health check with mocked HTTP. Test port map read/write/prune with mocked PIDs. Test `shutdown_chrome()` calls `Browser.close` then falls back to terminate/kill.

```python
# tests/test_browser.py (key tests)
from gsearch_mcp_cli.core.browser import parse_port_from_stderr, build_chrome_args
from gsearch_mcp_cli.exceptions import BrowserNotFoundError
import platform

def test_parse_port_from_stderr():
    line = "DevTools listening on ws://127.0.0.1:9345/devtools/browser/abc-123"
    assert parse_port_from_stderr(line) == 9345

def test_parse_port_from_stderr_no_match():
    assert parse_port_from_stderr("some other output") is None

def test_build_chrome_args_includes_platform_flags():
    args = build_chrome_args("/usr/bin/chrome", "/tmp/profile")
    assert "--headless=new" in args
    assert "--remote-allow-origins=http://localhost" in args
    if platform.system() == "Linux":
        assert "--disable-dev-shm-usage" in args
        assert "--no-sandbox" in args

def test_build_chrome_args_uses_port_zero():
    args = build_chrome_args("/usr/bin/chrome", "/tmp/profile")
    assert "--remote-debugging-port=0" in args
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement browser.py**

Key functions:
- `find_browser(preference: str = "auto") -> str` — scan candidates, return first found or raise `BrowserNotFoundError` with install guidance (including Windows with a clear "Windows not yet supported" or actual paths)
- `build_chrome_args(browser_path, profile_dir, extra_flags) -> list[str]` — assemble launch flags with platform-specific additions
- `parse_port_from_stderr(line: str) -> int | None` — extract port from `DevTools listening on ws://127.0.0.1:<PORT>/...`
- `launch_chrome(profile_dir, browser_path) -> tuple[subprocess.Popen, int]` — launch with `--remote-debugging-port=0`, read port from stderr, wait for CDP ready
- `ensure_browser(port: int) -> bool` — GET `/json/version`, return True if responsive
- `shutdown_chrome(proc, port)` — send CDP `Browser.close` first, then `proc.terminate()`, then `proc.kill()` as fallback (sync-compatible for use in signal handlers)
- `read_port_map() -> dict` — read `chrome-port-map.json`
- `write_port_map(port, pid)` — write entry to port map
- `prune_port_map()` — remove stale entries by checking PID is alive AND is a Chrome process AND port is listening. Uses `os.kill(pid, 0)` + process name check (`ps -p <pid> -o comm=` on macOS/Linux)

**Signal handling:** Register `atexit` with sync `proc.terminate()` / `proc.kill()` for CLI. For MCP, use `loop.add_signal_handler()` (see Task 10). Set Chrome process group for OS-level cleanup on parent death.

- [ ] **Step 4: Run tests, verify pass**

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat: browser detection, launch with port=0, port map, health check"
```

---

### Task 5: Async CDP Connection and Tab Management

**Files:**
- Create: `src/gsearch_mcp_cli/core/cdp.py`
- Create: `tests/test_cdp.py`

- [ ] **Step 1: Write tests**

Test `CDPConnection.connect()` and `CDPConnection.send()` with a mock WebSocket (mock `websockets.connect()` to return a fake that queues responses). Test CDP message dispatcher routes responses by `id`. Test `Tab` context manager guarantees cleanup even when body raises exception. Test Tab cleanup with timeout (doesn't hang on dead WebSocket). Test UA override. Test `evaluate()` returns parsed result. Test max_tabs enforcement raises when limit reached.

```python
# tests/test_cdp.py (key tests)
import pytest

async def test_tab_cleanup_on_exception(mock_browser_cdp):
    """Tab.__aexit__ must call closeTarget even when body raises."""
    # Implementation: verify Target.closeTarget was called despite the exception

async def test_tab_cleanup_timeout_on_dead_connection():
    """Tab.__aexit__ should not hang if WebSocket is dead — timeout after 5s."""

async def test_max_tabs_enforcement():
    """BrowserCDP should reject tab creation beyond max_tabs limit."""

async def test_cdp_response_routing():
    """Responses with matching id should be routed to the correct pending future."""
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement cdp.py**

```python
import asyncio
import json
import websockets
from constants import TIMEOUT_TAB_CLEANUP

class CDPConnection:
    """Async CDP connection with message dispatcher."""

    def __init__(self):
        self._ws = None
        self._msg_id = 0
        self._pending: dict[int, asyncio.Future] = {}
        self._reader_task: asyncio.Task | None = None

    async def connect(self, ws_url: str) -> None:
        self._ws = await websockets.connect(ws_url, max_size=None)
        self._reader_task = asyncio.create_task(self._read_loop())

    async def _read_loop(self):
        """Route incoming messages: responses by id, events to handlers."""
        async for raw in self._ws:
            msg = json.loads(raw)
            if "id" in msg and msg["id"] in self._pending:
                self._pending[msg["id"]].set_result(msg)
            # Events (no id) can be handled by subclass or ignored

    async def send(self, method: str, params: dict | None = None) -> dict:
        self._msg_id += 1
        msg_id = self._msg_id
        future = asyncio.get_event_loop().create_future()
        self._pending[msg_id] = future
        payload = {"id": msg_id, "method": method}
        if params:
            payload["params"] = params
        await self._ws.send(json.dumps(payload))
        try:
            return await asyncio.wait_for(future, timeout=30)
        finally:
            self._pending.pop(msg_id, None)

    async def close(self) -> None:
        if self._reader_task:
            self._reader_task.cancel()
        if self._ws:
            await self._ws.close()


class BrowserCDP:
    """Manages the browser-level CDP connection and tab lifecycle."""

    def __init__(self, max_tabs: int = 5):
        self._conn = CDPConnection()
        self._active_tabs: set[str] = set()
        self._max_tabs = max_tabs
        self._port: int = 0

    async def connect(self, port: int) -> None:
        self._port = port
        # GET /json/version to get browser WebSocket URL
        # Connect CDPConnection to that URL

    async def create_tab(self, url: str = "about:blank") -> "Tab":
        if len(self._active_tabs) >= self._max_tabs:
            raise GSearchError(f"Max concurrent tabs ({self._max_tabs}) reached")
        # Target.createTarget → get targetId
        # GET /json/list to find the tab's webSocketDebuggerUrl
        # Create per-tab CDPConnection for the new tab
        return Tab(self, target_id, tab_conn)

    async def close(self) -> None:
        await self._conn.close()


class Tab:
    """Page-level CDP with context manager for guaranteed cleanup."""

    async def __aenter__(self) -> "Tab":
        await self._conn.connect(self._ws_url)
        await self.override_ua()
        return self

    async def __aexit__(self, *exc) -> None:
        """Always close tab — with timeout and CancelledError protection."""
        try:
            await asyncio.wait_for(
                asyncio.shield(self._close_tab()),
                timeout=TIMEOUT_TAB_CLEANUP,
            )
        except (Exception, asyncio.CancelledError):
            pass  # best-effort cleanup — tab may already be gone

    async def _close_tab(self) -> None:
        try:
            await self._conn.close()
        finally:
            # Use browser-level connection to close the target
            await self._browser._conn.send(
                "Target.closeTarget", {"targetId": self._target_id}
            )
            self._browser._active_tabs.discard(self._target_id)

    async def navigate(self, url: str, timeout: float = 30) -> None
    async def evaluate(self, expression: str, timeout: float = 10) -> Any
    async def wait_for_load(self, timeout: float = 30) -> None
    async def override_ua(self) -> None  # HeadlessChrome -> Chrome
    async def get_cookies(self) -> list[dict]
```

**Key design decisions:**
- **Per-tab WebSocket connections**: Each `Tab` gets its own WebSocket to its `webSocketDebuggerUrl`. Simpler than session multiplexing, acceptable overhead for max 5 tabs.
- **Message dispatcher**: `CDPConnection._read_loop()` routes responses by `id` to pending futures. Events are handled separately.
- **Tab cleanup**: `__aexit__` uses `asyncio.wait_for()` + `asyncio.shield()` to guarantee cleanup doesn't hang and survives `CancelledError`.

- [ ] **Step 4: Run tests, verify pass**

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat: async CDP connection with message dispatcher and safe tab cleanup"
```

---

### Task 6: Rate Limiter

**Files:**
- Create: `src/gsearch_mcp_cli/core/rate_limiter.py`
- Create: `tests/test_rate_limiter.py`

- [ ] **Step 1: Write tests**

Test token bucket allows burst of 3, then throttles. Test configurable rate. Test `async with limiter:` pattern. Test concurrent access with `asyncio.gather()` — verify no more than burst + refilled tokens are granted. Test that after exhaustion, waiting exactly `rate_limit` seconds yields one token. Test zero/negative rate values are rejected.

```python
# tests/test_rate_limiter.py (key tests)
import asyncio

async def test_burst_allows_3_immediate():
    """First 3 calls should not wait."""

async def test_throttle_after_burst():
    """4th call should wait approximately rate_limit seconds."""

async def test_concurrent_access_is_safe():
    """Multiple concurrent acquire() calls should not grant excess tokens."""
    limiter = RateLimiter(rate=2.0, burst=3)
    results = []
    async def worker():
        async with limiter:
            results.append(asyncio.get_event_loop().time())
    # Launch 6 workers concurrently
    await asyncio.gather(*[worker() for _ in range(6)])
    # First 3 should be near-instant, next 3 should be spaced ~2s apart
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement rate_limiter.py**

Async token bucket with `asyncio.Lock` to make token check-and-acquire atomic under concurrent access. `acquire()` awaits if no tokens available. Burst capacity = 3. Refill rate = 1 token per `rate_limit` seconds.

```python
class RateLimiter:
    def __init__(self, rate: float = 2.0, burst: int = 3):
        self._rate = rate
        self._burst = burst
        self._tokens = float(burst)
        self._last_refill = asyncio.get_event_loop().time()
        self._lock = asyncio.Lock()

    async def acquire(self):
        async with self._lock:
            now = asyncio.get_event_loop().time()
            elapsed = now - self._last_refill
            self._tokens = min(self._burst, self._tokens + elapsed / self._rate)
            self._last_refill = now
            if self._tokens >= 1:
                self._tokens -= 1
                return
        # If no token, wait and retry
        await asyncio.sleep(self._rate)
        await self.acquire()

    async def __aenter__(self):
        await self.acquire()
        return self

    async def __aexit__(self, *exc):
        pass
```

> **Note:** `self._lock` must be created within a running event loop. If the RateLimiter is instantiated before the loop starts (e.g., at module level), create the lock lazily on first `acquire()` call.

- [ ] **Step 4: Run tests, verify pass**

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat: async token bucket rate limiter with concurrency safety"
```

---

### Task 7: Profile Warmup

**Files:**
- Create: `src/gsearch_mcp_cli/core/warmup.py`
- Create: `tests/test_warmup.py`

- [ ] **Step 1: Write tests**

Test warmup visits 3 pages using `wait_for_load()` (not fixed delays). Test cookie verification passes when NID+AEC present. Test warmup fails with clear `ProfileWarmupError` when cookies missing. Test partial cookie acquisition (NID present but AEC missing) still raises. Test warmup on already-warmed profile (idempotency). Mock CDP for all unit tests.

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement warmup.py**

```python
async def warmup_profile(browser_cdp: BrowserCDP) -> None:
    """Visit Google pages to establish BotGuard trust cookies."""
    async with browser_cdp.create_tab() as tab:
        await tab.override_ua()
        await tab.navigate(GOOGLE_HOMEPAGE)
        await tab.wait_for_load()
        await tab.navigate(GOOGLE_ABOUT)
        await tab.wait_for_load()
        await tab.navigate(GOOGLE_PREFERENCES)
        await tab.wait_for_load()
        cookies = await tab.get_cookies()
        cookie_names = {c["name"] for c in cookies}
        missing = REQUIRED_WARMUP_COOKIES - cookie_names
        if missing:
            raise ProfileWarmupError(f"Missing cookies after warmup: {missing}")

async def needs_warmup(profile_dir: Path) -> bool:
    """Check if profile exists and has been warmed up."""
```

- [ ] **Step 4: Run tests, verify pass**

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat: profile warmup with event-based waits and cookie verification"
```

---

### Task 8: Search Result Extractor

**Files:**
- Create: `src/gsearch_mcp_cli/core/extractor.py`
- Create: `tests/test_extractor.py`

**IMPORTANT — validate DOM first:** Before writing extractors, manually inspect current Google Search DOM structure. Open a headless Chrome session, search for a test query, and save the HTML. AI Mode and regular search DOM structures change frequently. Saved HTML fixtures should be committed for regression testing.

- [ ] **Step 1: Capture current DOM fixtures**

Open a Chrome session and save HTML for: regular search results page, AI Mode results page, empty/no-results page, and Google's "unusual traffic" block page. Save to `tests/fixtures/`.

- [ ] **Step 2: Write tests**

Test `extract_search_results()` with mocked `tab.evaluate()` returning the fixture data — verify `SearchResult` has correct fields. Test `extract_ai_mode()` similarly. Test extraction of empty results returns empty lists (not crash). Test extraction of block page raises `SearchBlockedError`. Test extraction of page with AI Overview absent returns `ai_overview=None`.

- [ ] **Step 3: Run tests to verify they fail**

- [ ] **Step 4: Implement extractor.py**

Two main functions:
- `async def extract_search_results(tab: Tab) -> SearchResult` — executes JS to extract h3 titles, URLs, snippets, AI Overview text, People Also Ask, related searches. Raises `ExtractionError` if zero organic results found for a non-empty query. Raises `SearchBlockedError` if block page detected.
- `async def extract_ai_mode(tab: Tab) -> AIResult` — executes JS to extract AI synthesis text, h3 section headings, li bullet points, source links

Each function contains a single large `tab.evaluate()` call with a JS script that returns structured JSON from the DOM. The Python side deserializes into Pydantic models.

- [ ] **Step 5: Run tests, verify pass**

- [ ] **Step 6: Commit**

```bash
git add -A
git commit -m "feat: DOM extractors for search results and AI Mode with fixtures"
```

---

### Task 9: SSRF Validator and Page Fetcher

**Files:**
- Create: `src/gsearch_mcp_cli/core/fetcher.py`
- Create: `tests/test_fetcher.py`

- [ ] **Step 1: Write comprehensive SSRF tests**

```python
# tests/test_fetcher.py
import pytest
from gsearch_mcp_cli.core.fetcher import validate_url
from gsearch_mcp_cli.exceptions import SSRFBlockedError

# --- SSRF validation tests (parametrized for full coverage) ---

@pytest.mark.parametrize("url", [
    "https://example.com",
    "https://google.com/search?q=test",
    "http://93.184.216.34",  # example.com's public IP
])
def test_validate_url_accepts_valid(url):
    assert validate_url(url) == url  # or normalized form

@pytest.mark.parametrize("url,reason", [
    # Protocol attacks
    ("file:///etc/passwd", "blocked scheme"),
    ("ftp://example.com/file", "blocked scheme"),
    ("data:text/html,<h1>hi</h1>", "blocked scheme"),
    ("javascript:alert(1)", "blocked scheme"),
    # IPv4 private ranges
    ("http://10.0.0.1", "private IPv4"),
    ("http://172.16.0.1", "private IPv4"),
    ("http://192.168.1.1", "private IPv4"),
    ("http://127.0.0.1", "loopback"),
    ("http://169.254.169.254", "cloud metadata"),
    ("http://0.0.0.0", "unspecified"),
    # IPv6 attacks
    ("http://[::1]", "IPv6 loopback"),
    ("http://[fe80::1]", "IPv6 link-local"),
    ("http://[::ffff:127.0.0.1]", "IPv4-mapped IPv6"),
    # IP encoding bypasses
    ("http://0x7f000001", "hex-encoded loopback"),
    ("http://2130706433", "decimal-encoded loopback"),
    ("http://017700000001", "octal-encoded loopback"),
    # Special hostnames
    ("http://metadata.google.internal", "cloud metadata host"),
    ("http://localhost", "localhost"),
    ("http://localhost:9222/json", "CDP endpoint"),
])
def test_validate_url_rejects_ssrf(url, reason):
    with pytest.raises(SSRFBlockedError):
        validate_url(url)

# Fetch tests
async def test_fetch_page_returns_markdown():
    """Fetch https://example.com and verify Markdown output."""

async def test_fetch_with_browser_never_uses_httpx():
    """use_browser='never' should use httpx, not CDP."""
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement fetcher.py**

```python
import ipaddress
import socket
from urllib.parse import urlparse

def validate_url(url: str) -> str:
    """Validate URL for SSRF. Returns normalized URL or raises SSRFBlockedError."""
    parsed = urlparse(url)

    # 1. Scheme check
    if parsed.scheme not in ALLOWED_SCHEMES:
        raise SSRFBlockedError(f"Blocked scheme: {parsed.scheme}")

    # 2. Hostname extraction and DNS resolution
    hostname = parsed.hostname
    if not hostname:
        raise SSRFBlockedError("No hostname in URL")

    # Check blocked hostnames
    if hostname in BLOCKED_HOSTS or hostname == "localhost":
        raise SSRFBlockedError(f"Blocked host: {hostname}")

    # 3. DNS resolution — resolve BEFORE connecting to prevent DNS rebinding
    try:
        infos = socket.getaddrinfo(hostname, parsed.port or 443, proto=socket.IPPROTO_TCP)
    except socket.gaierror:
        raise SSRFBlockedError(f"DNS resolution failed: {hostname}")

    # 4. Check all resolved IPs against blocklists
    for family, _, _, _, sockaddr in infos:
        ip_str = sockaddr[0]
        try:
            ip = ipaddress.ip_address(ip_str)
        except ValueError:
            continue
        if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
            raise SSRFBlockedError(f"Resolved to blocked IP: {ip_str}")

    return url

async def fetch_page(url: str, format: str = "md", use_browser: str = "auto",
                     browser_cdp: BrowserCDP | None = None, timeout: float = 30) -> FetchResult:
    """Fetch URL and convert to markdown/text."""
```

> **Key:** Uses Python's `ipaddress` module for IP validation instead of string prefix matching. This correctly handles hex, decimal, octal, and IPv6 representations. DNS is resolved before the connection to prevent rebinding attacks.

Fetch flow:
1. `validate_url()` — scheme check, DNS resolve, IP blocklist
2. If `use_browser == "never"` or (`"auto"` and no browser available): httpx GET
3. Else: CDP tab navigate + extract innerHTML
4. trafilatura extract main content from HTML
5. html-to-markdown convert to Markdown
6. Return `FetchResult`

- [ ] **Step 4: Run tests, verify pass**

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat: SSRF-safe page fetcher with IPv4/IPv6 validation and MD conversion"
```

---

### Task 10: Shared Backend

**Files:**
- Create: `src/gsearch_mcp_cli/shared.py`
- Create: `tests/test_shared.py`

- [ ] **Step 1: Write unit tests with mocked browser**

```python
# tests/test_shared.py
import pytest
from unittest.mock import patch, AsyncMock
from gsearch_mcp_cli import shared
from gsearch_mcp_cli.models import SearchResult, AIResult, FetchResult

async def test_get_browser_uses_lock_for_init():
    """Two concurrent _get_browser() calls should only launch Chrome once."""
    launch_count = 0
    original_launch = shared._launch_browser
    async def counting_launch():
        nonlocal launch_count
        launch_count += 1
        return await original_launch()
    # Patch and call concurrently...
    # Assert launch_count == 1

async def test_search_returns_search_result(mock_browser_cdp, mock_tab):
    """search() should return SearchResult via mocked browser."""
    with patch.object(shared, '_get_browser', return_value=mock_browser_cdp):
        mock_tab.evaluate.return_value = {
            "results": [{"position": 1, "title": "T", "url": "https://x.com", "snippet": "S", "source": "x.com"}],
            "ai_overview": None, "people_also_ask": [], "related_searches": [],
        }
        result = await shared.search("test query")
        assert isinstance(result, SearchResult)

async def test_search_retries_on_blocked(mock_browser_cdp):
    """SearchBlockedError should trigger auto-warmup + one retry."""

@pytest.mark.integration
async def test_search_real():
    """Integration: real Google search."""
    result = await shared.search("what is python")
    assert len(result.results) > 0
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement shared.py**

```python
import asyncio
import signal
from gsearch_mcp_cli.core.browser import launch_chrome, ensure_browser, shutdown_chrome
from gsearch_mcp_cli.core.cdp import BrowserCDP
from gsearch_mcp_cli.core.rate_limiter import RateLimiter
from gsearch_mcp_cli.core.warmup import warmup_profile, needs_warmup

_browser_cdp: BrowserCDP | None = None
_rate_limiter: RateLimiter | None = None
_chrome_proc = None
_init_lock: asyncio.Lock | None = None  # created lazily in running loop

async def _get_init_lock() -> asyncio.Lock:
    """Get or create the init lock (must be in a running event loop)."""
    global _init_lock
    if _init_lock is None:
        _init_lock = asyncio.Lock()
    return _init_lock

async def _get_browser() -> BrowserCDP:
    """Lazy init with double-check locking to prevent concurrent launches."""
    global _browser_cdp, _chrome_proc, _rate_limiter
    if _browser_cdp is not None and await _browser_cdp.is_healthy():
        return _browser_cdp

    lock = await _get_init_lock()
    async with lock:
        # Double-check after acquiring lock
        if _browser_cdp is not None and await _browser_cdp.is_healthy():
            return _browser_cdp

        # Launch Chrome, connect CDP, warmup if needed
        proc, port = launch_chrome(...)
        _chrome_proc = proc
        browser = BrowserCDP(max_tabs=config.get("max_tabs"))
        await browser.connect(port)
        if await needs_warmup(CHROME_PROFILE_DIR):
            await warmup_profile(browser)
        _browser_cdp = browser

        if _rate_limiter is None:
            _rate_limiter = RateLimiter(rate=config.get("rate_limit"))

    return _browser_cdp

async def search(query: str, max_results: int = 10, include_ai: bool = True,
                 language: str = "en", region: str | None = None,
                 timeout: float = TIMEOUT_SEARCH) -> SearchResult:
    """Google Search: AI Overview + organic results."""

async def ai_search(query: str, language: str = "en",
                    timeout: float = TIMEOUT_AI_MODE) -> AIResult:
    """Google AI Mode search."""

async def fetch(url: str, format: str = "md", use_browser: str = "auto",
                timeout: float = TIMEOUT_FETCH_BROWSER) -> FetchResult:
    """Fetch and parse URL."""

async def shutdown() -> None:
    """Shutdown Chrome and cleanup."""
    global _browser_cdp, _chrome_proc, _rate_limiter
    if _browser_cdp:
        await _browser_cdp.close()
        _browser_cdp = None
    if _chrome_proc:
        shutdown_chrome(_chrome_proc, None)
        _chrome_proc = None
    _rate_limiter = None
```

Each function: acquire rate limiter → get browser → create tab → navigate → extract → return model.

If `SearchBlockedError` detected: auto-warmup + retry once.

**Cleanup strategy:**
- **CLI:** Each Click command calls `asyncio.run(main())` where `main()` uses `try/finally` to call `shutdown()`.
- **MCP:** FastMCP lifecycle hooks or `loop.add_signal_handler(signal.SIGTERM, ...)` for graceful shutdown.
- **Neither uses `atexit`** (which cannot call async functions).

- [ ] **Step 4: Run tests (unit + integration)**

```bash
pytest tests/test_shared.py -v -k "not integration"   # unit tests
pytest tests/test_shared.py -v -k "integration"        # integration tests
```

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat: shared backend with init lock, crash recovery, and cleanup"
```

---

### Task 11: CLI — Main Group and Search Command

**Files:**
- Create: `src/gsearch_mcp_cli/cli/main.py`
- Create: `src/gsearch_mcp_cli/cli/search_cmd.py`
- Create: `src/gsearch_mcp_cli/cli/ai_doc.py`
- Create: `tests/test_cli.py`

- [ ] **Step 1: Write tests**

Test CLI `--version` outputs version. Test `--docs` outputs AI doc and exits. Test `gsearch "query"` invokes search (default command). Test `--json` flag outputs valid JSON.

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement main.py**

Click group with `--version`, `--docs` (eager — renamed from `--ai` to avoid collision with `ai` subcommand), `--verbose`, `--timeout`. Default command = search. Register all subcommands.

**Sync-to-async bridge:** Each Click command wraps the async call:
```python
@cli.command()
@click.argument("query")
def search(query, **kwargs):
    async def _run():
        try:
            result = await shared.search(query, ...)
            # format and print
        finally:
            await shared.shutdown()
    asyncio.run(_run())
```

> **Note:** `asyncio.run()` is only called at the CLI boundary. It creates a fresh event loop, which is correct for CLI usage. This CANNOT be called from within an already-running event loop (MCP) — but MCP uses its own async entry point, so there's no conflict.

- [ ] **Step 4: Implement search_cmd.py**

`gsearch search "query"` (also default). Flags: `--max-results`, `--json`, `--no-ai`, `--lang`, `--region`, `--timeout`. Calls `shared.search()`, formats output with Rich tables or JSON.

- [ ] **Step 5: Implement ai_doc.py**

Plain text AI-optimized documentation string. Printed by `--docs` flag.

- [ ] **Step 6: Run tests and manual test**

```bash
pytest tests/test_cli.py -v
gsearch --version
gsearch --docs | head -20
gsearch "what is python" --json
```

- [ ] **Step 7: Commit**

```bash
git add -A
git commit -m "feat: CLI main group with search command and --docs flag"
```

---

### Task 12a: CLI — AI and Fetch Commands

**Files:**
- Create: `src/gsearch_mcp_cli/cli/ai_cmd.py`
- Create: `src/gsearch_mcp_cli/cli/fetch_cmd.py`
- Update: `tests/test_cli_commands.py`

- [ ] **Step 1: Write tests**

```python
# tests/test_cli_commands.py
from click.testing import CliRunner
from gsearch_mcp_cli.cli.main import cli

def test_ai_command_help():
    runner = CliRunner()
    result = runner.invoke(cli, ["ai", "--help"])
    assert result.exit_code == 0
    assert "--json" in result.output
    assert "--lang" in result.output

def test_fetch_command_help():
    runner = CliRunner()
    result = runner.invoke(cli, ["fetch", "--help"])
    assert result.exit_code == 0
    assert "--raw" in result.output
    assert "--browser" in result.output
    assert "--timeout" in result.output
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement ai_cmd.py**

`gsearch ai "query"` with `--json`, `--lang`, `--region`, `--timeout`. Calls `shared.ai_search()`.

- [ ] **Step 4: Implement fetch_cmd.py**

`gsearch fetch "url"` with `--raw`, `--json`, `--format`, `--browser` (accepts `auto`/`always`/`never`), `--timeout`. Calls `shared.fetch()`.

- [ ] **Step 5: Register commands in main.py**

- [ ] **Step 6: Run tests, verify pass**

- [ ] **Step 7: Commit**

```bash
git add -A
git commit -m "feat: CLI ai and fetch commands"
```

---

### Task 12b: CLI — Setup, Config, Doctor Commands

**Files:**
- Create: `src/gsearch_mcp_cli/cli/setup_cmd.py`
- Create: `src/gsearch_mcp_cli/cli/config_cmd.py`
- Create: `src/gsearch_mcp_cli/cli/doctor_cmd.py`
- Update: `tests/test_cli_commands.py`

- [ ] **Step 1: Write tests**

```python
def test_config_show_outputs_json():
    runner = CliRunner()
    result = runner.invoke(cli, ["config", "show"])
    assert result.exit_code == 0
    assert "browser" in result.output

def test_config_set_invalid_key():
    runner = CliRunner()
    result = runner.invoke(cli, ["config", "set", "bad_key", "value"])
    assert result.exit_code != 0

def test_doctor_runs():
    runner = CliRunner()
    result = runner.invoke(cli, ["doctor"])
    assert result.exit_code == 0
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement setup_cmd.py**

`gsearch setup` — runs warmup. `--reset` deletes profile directory first. `--check` verifies profile works by running a test search.

- [ ] **Step 4: Implement config_cmd.py**

`gsearch config show`, `gsearch config set <key> <value>`, `gsearch config get <key>`. Validates keys and provides "did you mean?" suggestions for typos.

- [ ] **Step 5: Implement doctor_cmd.py**

`gsearch doctor` — comprehensive checks with Rich pass/fail indicators:
- Browser found (name + path)
- Chrome version compatibility (minimum for `--headless=new`)
- Profile directory exists + permissions
- Profile warmed up (cookies present)
- CDP connectable (launch + health check)
- Google reachable (DNS + HTTPS)
- Port conflicts (scan 9222-9241)
- Config file valid
- Test search works

- [ ] **Step 6: Register commands in main.py**

- [ ] **Step 7: Run tests and manual test**

```bash
pytest tests/test_cli_commands.py -v
gsearch setup --check
gsearch config show
gsearch doctor
```

- [ ] **Step 8: Commit**

```bash
git add -A
git commit -m "feat: CLI setup, config, and doctor commands"
```

---

### Task 12c: CLI — Skill Command

**Files:**
- Create: `src/gsearch_mcp_cli/cli/skill_cmd.py`
- Update: `tests/test_cli_commands.py`

- [ ] **Step 1: Write tests**

```python
def test_skill_list():
    runner = CliRunner()
    result = runner.invoke(cli, ["skill", "list"])
    assert result.exit_code == 0
    assert "claude-code" in result.output

def test_skill_install_creates_files(tmp_path):
    """Verify skill install copies SKILL.md to target directory."""
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement skill_cmd.py**

`gsearch skill install <tool>`, `gsearch skill list`, `gsearch skill update`. Copies `data/SKILL.md` + `data/references/` to platform-specific paths.

- [ ] **Step 4: Register in main.py, run tests**

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat: CLI skill command for AI tool integration"
```

---

### Task 13: MCP Server

**Files:**
- Create: `src/gsearch_mcp_cli/mcp/server.py`
- Create: `src/gsearch_mcp_cli/mcp/__init__.py`
- Create: `src/gsearch_mcp_cli/mcp/__main__.py`
- Create: `tests/test_mcp.py`

- [ ] **Step 1: Write tests**

Test MCP tool registration (3 tools: gsearch_search, gsearch_ai, gsearch_fetch). Test tool schemas match Pydantic models. Test error sanitization — verify no file paths, PIDs, or port numbers in error responses.

```python
# tests/test_mcp.py
from gsearch_mcp_cli.mcp.server import sanitize_error

def test_sanitize_error_strips_paths():
    err = CDPConnectionError("Failed to connect at /Users/jacob/.gsearch-mcp-cli/chrome-profile on port 9222 (pid 12345)")
    sanitized = sanitize_error(err)
    assert "/Users/" not in sanitized
    assert "9222" not in sanitized
    assert "12345" not in sanitized
    assert "Failed to connect" in sanitized

def test_sanitize_error_preserves_user_facing_message():
    err = SearchBlockedError("Google blocked the search — unusual traffic detected")
    sanitized = sanitize_error(err)
    assert "blocked" in sanitized.lower()
```

- [ ] **Step 2: Run tests to verify they fail**

- [ ] **Step 3: Implement server.py**

```python
import re
from fastmcp import FastMCP
from gsearch_mcp_cli.exceptions import GSearchError
from gsearch_mcp_cli import shared

mcp = FastMCP("gsearch-mcp", instructions="...")

def sanitize_error(e: Exception) -> str:
    """Strip file paths, PIDs, port numbers from error messages for MCP clients."""
    msg = str(e)
    msg = re.sub(r'/[\w/.-]+', '<path>', msg)           # file paths
    msg = re.sub(r'\bpid\s*\d+\b', 'pid <hidden>', msg, flags=re.IGNORECASE)
    msg = re.sub(r'\bport\s*\d+\b', 'port <hidden>', msg, flags=re.IGNORECASE)
    return msg

@mcp.tool()
async def gsearch_search(query: str, max_results: int = 10,
                         include_ai: bool = True, language: str = "en",
                         region: str | None = None,
                         timeout_seconds: int = 30) -> dict:
    """Search Google: AI Overview + organic results."""
    try:
        result = await shared.search(query, max_results=max_results,
                                     include_ai=include_ai, language=language,
                                     region=region, timeout=timeout_seconds)
        return result.model_dump()
    except GSearchError as e:
        return {"error": sanitize_error(e)}

@mcp.tool()
async def gsearch_ai(query: str, language: str = "en",
                     timeout_seconds: int = 30) -> dict:
    """Google AI Mode search with synthesized response."""
    try:
        result = await shared.ai_search(query, language=language,
                                        timeout=timeout_seconds)
        return result.model_dump()
    except GSearchError as e:
        return {"error": sanitize_error(e)}

@mcp.tool()
async def gsearch_fetch(url: str, format: str = "md",
                        use_browser: str = "auto",
                        timeout_seconds: int = 30) -> dict:
    """Fetch and parse a URL to clean Markdown."""
    try:
        result = await shared.fetch(url, format=format, use_browser=use_browser,
                                    timeout=timeout_seconds)
        return result.model_dump()
    except GSearchError as e:
        return {"error": sanitize_error(e)}
```

- [ ] **Step 4: Implement __init__.py and __main__.py**

```python
# mcp/__init__.py
from .server import mcp
def run_server():
    mcp.run()

# mcp/__main__.py
from gsearch_mcp_cli.mcp import run_server
run_server()
```

- [ ] **Step 5: Run tests**

```bash
pytest tests/test_mcp.py -v
```

- [ ] **Step 6: Manual test MCP server**

```bash
gsearch-mcp  # should start server on stdio
```

- [ ] **Step 7: Commit**

```bash
git add -A
git commit -m "feat: MCP server with 3 tools and sanitized error responses"
```

---

### Task 14: Skill File and AI Docs

**Files:**
- Create: `src/gsearch_mcp_cli/data/SKILL.md`
- Create: `src/gsearch_mcp_cli/data/AGENTS_SECTION.md`
- Create: `src/gsearch_mcp_cli/data/references/mcp-tools.md`
- Create: `src/gsearch_mcp_cli/data/references/commands.md`

- [ ] **Step 1: Write SKILL.md**

Follow the pattern from Perplexity/Gemini skills. Include: quick reference (`gsearch --docs`), critical rules, workflow decision tree, CLI commands, MCP tools summary, error recovery.

- [ ] **Step 2: Write AGENTS_SECTION.md**

Section for inclusion in AGENTS.md / CLAUDE.md describing how to use gsearch tools.

- [ ] **Step 3: Write references/mcp-tools.md**

Document all 3 MCP tool signatures with parameters, types, defaults, and example usage.

- [ ] **Step 4: Write references/commands.md**

Document all CLI commands with flags and examples.

- [ ] **Step 5: Verify skill install works**

```bash
gsearch skill install cursor
ls ~/.cursor/skills/gsearch-mcp/
```

- [ ] **Step 6: Commit**

```bash
git add -A
git commit -m "feat: SKILL.md, AGENTS_SECTION.md, and reference docs"
```

---

### Task 15: Integration Tests and Polish

**Files:**
- Create: `tests/test_integration.py`
- Modify: various files for fixes

- [ ] **Step 1: Write integration test suite**

End-to-end tests that actually search Google, extract results, fetch a page. Marked with `@pytest.mark.integration` so they can be skipped in CI. Session-scoped browser fixture for shared Chrome instance across tests.

```python
# tests/test_integration.py
import pytest
from gsearch_mcp_cli import shared

@pytest.fixture(scope="module")
async def browser():
    """Shared browser for integration tests — launch once, cleanup after all."""
    await shared._get_browser()
    yield
    await shared.shutdown()

@pytest.mark.integration
async def test_full_search_flow(browser):
    result = await shared.search("what is python")
    assert len(result.results) > 0
    assert result.query == "what is python"

@pytest.mark.integration
async def test_full_ai_mode(browser):
    result = await shared.ai_search("what is docker")
    assert len(result.synthesis) > 50

@pytest.mark.integration
async def test_full_fetch(browser):
    result = await shared.fetch("https://example.com")
    assert "Example Domain" in result.content

@pytest.mark.integration
async def test_fetch_httpx_only(browser):
    result = await shared.fetch("https://example.com", use_browser="never")
    assert "Example Domain" in result.content

@pytest.mark.integration
async def test_concurrent_searches(browser):
    """Two concurrent searches should both succeed via rate limiter."""
    import asyncio
    results = await asyncio.gather(
        shared.search("python tutorial"),
        shared.search("javascript tutorial"),
    )
    assert all(len(r.results) > 0 for r in results)

@pytest.mark.integration
async def test_non_english_search(browser):
    result = await shared.search("qu'est-ce que python", language="fr")
    assert len(result.results) > 0
```

- [ ] **Step 2: Run integration tests**

```bash
pytest tests/test_integration.py -v -m integration
```

- [ ] **Step 3: Fix any issues found**

- [ ] **Step 4: Run full test suite**

```bash
pytest tests/ -v --tb=short --cov=gsearch_mcp_cli --cov-fail-under=70
```

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "test: integration tests for search, AI mode, fetch, and concurrency"
```

---

## Execution Notes

- Tasks 1–3 are pure Python, no browser needed — fast to implement and test
- Tasks 4–5 build the browser infrastructure — **highest risk**, front-load these
- Task 6 (Rate Limiter) has NO dependency on browser/CDP — can parallelize with Tasks 3-4
- Tasks 7–9 can run in parallel after Task 5
- Task 8 (Extractor): **capture real DOM fixtures FIRST** before writing extraction code
- Task 10 ties everything together — first time real end-to-end search works. Budget extra time.
- Tasks 11–14 can be parallelized across agents
- Task 12 is split into 12a/12b/12c with tests for each

## Key Architecture Decisions (from spec review)

| Decision | Choice | Rationale |
|----------|--------|-----------|
| CDP transport | `websockets` (async) | FastMCP is async; sync `websocket-client` would block the event loop |
| Data models | Pydantic `BaseModel` | Automatic JSON serialization, validation, and MCP schema generation |
| CDP connection model | Per-tab WebSocket | Simpler than session multiplexing; acceptable overhead for max 5 tabs |
| Browser lifecycle | Health check + auto-relaunch | Long-running MCP sessions need crash recovery |
| Timeout budget | Explicit constants per operation | AI agents need predictable response times |
| SSRF prevention | `ipaddress` module + DNS pre-resolution | Handles all IP encoding bypasses (hex, decimal, IPv6) correctly |
| Error sanitization | Regex-strip paths/PIDs/ports for MCP | Prevent information leakage to AI agents |
| Init guard | `asyncio.Lock` with double-check | Prevent concurrent Chrome launches from parallel MCP calls |
| Cleanup | `try/finally` + signal handlers (not `atexit`) | `atexit` cannot run async cleanup |
| TimeoutError naming | `GSearchTimeoutError` | Avoid shadowing Python's built-in `TimeoutError` |
