# Fix: 1st Degree Connection Search — Extract, Store, Search Locally

## Problem

LinkedIn search (via Unipile `search_people()`) does NOT reliably filter by connection degree. When a user asks "find logistics people in my 1st degree connections," the current flow calls `search_people()` or `network search`, both of which return a mix of 1st, 2nd, and 3rd degree results. Users verify on LinkedIn and find the results are 2nd degree — this is a **repeatedly confirmed bug**.

The `connections_only` filter (`network_distance: [1]`) in campaign refill is also unreliable because it depends on LinkedIn's search API honoring the filter, which it doesn't consistently.

## Root Cause

We're relying on LinkedIn's search API to filter by degree, but it doesn't. The only **reliable** way to guarantee 1st degree is to **extract all connections via Unipile's connections endpoint**, store them locally, and search our own database.

## Solution: 3-Phase Architecture

### Phase 1: Full Connection Sync via Unipile API

**File to modify:** `src/heylead/services/connection_sync.py`

The existing `connection_sync.py` already syncs connections for dedup purposes. Extend it to do a **full extraction** with profile data, not just IDs.

Use Unipile's `GET /api/v1/users/{account_id}/relations` endpoint (paginated) to pull ALL 1st degree connections. This endpoint returns the user's actual connection list — guaranteed 1st degree because it's reading the contact book, not searching.

**Implementation:**

```
async def full_connection_sync(account_id: str, force: bool = False) -> dict:
    """
    Extract ALL 1st-degree connections from LinkedIn via Unipile relations endpoint.
    Paginate through the full list. Store each connection in local DB.

    Returns: { total_synced: int, new: int, updated: int, duration_s: float }
    """
```

Key requirements:
- **Paginate fully** — Unipile paginates relations. Follow `cursor`/`next` until exhausted.
- **Rate limit aware** — Add small delays between pages (200-500ms) to avoid Unipile rate limits.
- **Store rich data** — For each connection, store: `linkedin_id`, `name`, `headline/title`, `company`, `location`, `profile_url` (the vanity URL like `/in/username`), `connection_degree` (always 1 since we're reading relations), `synced_at` timestamp.
- **Idempotent** — Use upsert by `linkedin_id`. Update existing records, insert new ones.
- **Background-friendly** — This may take 1-5 minutes for large networks (1000+ connections). Must not block MCP tool responses. Run as async background task, return progress.
- **Cache freshness** — Store `last_full_sync_at` timestamp. Skip re-sync if synced within last 24 hours (unless `force=True`).

### Phase 2: Local Storage in SQLite

**Files to modify:** `src/heylead/db/schema.py`, new file `src/heylead/db/connection_queries.py`

Add a new table `my_connections` (or extend `global_contacts` with a `is_1st_degree: bool` + `connection_synced_at` column — choose whichever is cleaner):

**Option A — New table (recommended, cleaner separation):**
```sql
CREATE TABLE IF NOT EXISTS my_connections (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    account_id TEXT NOT NULL,          -- which LinkedIn account this connection belongs to
    linkedin_id TEXT NOT NULL,          -- Unipile provider_id / LinkedIn member ID
    profile_url TEXT,                   -- vanity URL: /in/username (for clickable links!)
    name TEXT NOT NULL,
    headline TEXT,                      -- job title / headline
    company TEXT,
    location TEXT,
    industry TEXT,
    profile_json TEXT,                  -- full profile data as JSON (for future enrichment)
    synced_at TEXT NOT NULL,            -- when this record was last synced
    created_at TEXT DEFAULT (datetime('now')),
    UNIQUE(account_id, linkedin_id)     -- one record per connection per account
);

CREATE INDEX IF NOT EXISTS idx_connections_name ON my_connections(name COLLATE NOCASE);
CREATE INDEX IF NOT EXISTS idx_connections_headline ON my_connections(headline COLLATE NOCASE);
CREATE INDEX IF NOT EXISTS idx_connections_company ON my_connections(company COLLATE NOCASE);
CREATE INDEX IF NOT EXISTS idx_connections_location ON my_connections(location COLLATE NOCASE);
```

**CRITICAL: `profile_url` must be the vanity URL** (e.g., `https://www.linkedin.com/in/osavenkova`), NOT the internal LinkedIn member ID. The Unipile relations endpoint returns `public_identifier` or similar — use that to construct the URL. If not available from the relations endpoint, enrich lazily on first access.

**Query functions in `connection_queries.py`:**
```python
async def search_my_connections(
    query: str,                    # free-text search across name, headline, company, location
    account_id: str = None,        # filter by account (None = all accounts)
    limit: int = 25
) -> list[dict]:
    """
    Full-text search across my_connections table.
    Use LIKE '%query%' on name, headline, company, location columns.
    For better results, split query into terms and AND them.
    Return results sorted by relevance (name match first, then headline, etc.)
    """

async def upsert_connection(account_id: str, connection: dict) -> None:
    """Upsert a single connection record."""

async def bulk_upsert_connections(account_id: str, connections: list[dict]) -> int:
    """Bulk upsert connections. Return count of inserted/updated."""

async def get_sync_status(account_id: str) -> dict:
    """Return { total_connections: int, last_synced_at: str | None }"""
```

### Phase 3: New Tool Action — `contacts(action="my_connections")`

**File to modify:** `src/heylead/tools/contacts.py`

Add a new action `my_connections` (or `search_connections`) to the contacts tool:

```python
# In run_contacts() dispatcher:
elif action == "my_connections":
    return await _handle_my_connections(query=query, limit=limit)
```

**Implementation:**
```python
async def _handle_my_connections(query: str, limit: int = 25) -> str:
    """
    Search ONLY 1st-degree connections from locally synced data.

    1. Check if connections have been synced (my_connections table non-empty).
       If not synced, trigger sync first (or return message asking user to wait).
    2. Search locally using search_my_connections().
    3. Format results with WORKING LinkedIn profile URLs.
    4. Return formatted table.
    """
```

**Output format must include working LinkedIn URLs:**
```
1. Olena Savenkova
   Chief Procurement & Logistics Officer | Alamak DMCC
   Location: Ukraine
   https://www.linkedin.com/in/osavenkova    <-- MUST be vanity URL, not member ID
```

Also update the tool description/docstring in `server.py` to mention this new action:
```
"my_connections" — Search your 1st-degree LinkedIn connections (locally synced, guaranteed 1st degree)
```

### Phase 4: Auto-Sync on First Use + Periodic Refresh

**Trigger sync automatically:**
1. When `contacts(action="my_connections")` is called and no sync has happened → trigger sync, return "Syncing your connections... this takes 1-2 minutes for the first time. Please try again shortly."
2. When `network(action="sync")` is called → also trigger full connection sync (extend existing behavior).
3. Optionally: scheduler job to refresh connections weekly.

**In `network.py` `_handle_sync()`:**
After the existing backend sync call, also run the local connection sync:
```python
async def _handle_sync(self):
    # Existing backend sync...
    result = await self.client.network_pool_sync()

    # NEW: Also sync connections locally
    from ..services.connection_sync import full_connection_sync
    sync_result = await full_connection_sync(self.account_id)

    return f"{result}\n\nLocal connections: {sync_result['total_synced']} synced ({sync_result['new']} new)"
```

## Key Files to Touch

| File | Change |
|------|--------|
| `src/heylead/db/schema.py` | Add `my_connections` table + indexes |
| `src/heylead/db/connection_queries.py` | **New file** — CRUD + search queries |
| `src/heylead/services/connection_sync.py` | Add `full_connection_sync()` with pagination |
| `src/heylead/tools/contacts.py` | Add `my_connections` action |
| `src/heylead/tools/network.py` | Extend sync to trigger local connection sync |
| `src/heylead/server.py` | Update contacts tool docstring |
| `src/heylead/linkedin/unipile.py` | Add/verify `get_relations()` method for paginated connection fetch |
| `src/heylead/linkedin/backend_client.py` | Add relations endpoint if proxied through backend |

## Unipile API Reference

Check the Unipile docs for the exact endpoint. It's likely one of:
- `GET /api/v1/users/{account_id}/relations` — list all connections
- `GET /api/v1/users/me/relations` — list own connections
- May support `?cursor=` or `?offset=` for pagination
- Response includes: `provider_id` (LinkedIn member ID), `public_identifier` (vanity slug), `first_name`, `last_name`, `headline`, etc.

If Unipile doesn't expose a relations/connections list endpoint, use their **messaging contacts** endpoint as an alternative source, or use LinkedIn's own connections page scraping via the Unipile proxy.

## Testing

1. Run `contacts(action="my_connections", query="logistics")` — should return ONLY 1st degree connections
2. Verify every returned profile URL works when opened in browser
3. Verify results show "1st" degree badge on LinkedIn when checked manually
4. Test with empty query (list all connections)
5. Test sync with large network (500+ connections) — should complete without timeout
6. Test re-sync (should be fast, skip if recent)

## Success Criteria

- User asks "find logistics people in my connections" → gets ONLY 1st degree results with working LinkedIn URLs
- No more 2nd degree connections appearing in results
- Sync happens transparently (auto on first use, cached after)
- Profile URLs are vanity URLs (`/in/username`), not member IDs
