# Fix: Ghost Account in Per-Account Aggregation + StmtFunds Topic Name

## Context

After fixing the IBKR Flex Phase 2 polling (commit `f4dac23f`), realized performance for `source=ibkr_flex` regressed from **-11.37%** (baseline, close to -9.35% broker actual) to **+111.42%**. Two bugs identified:

1. **StmtFunds topic name wrong** — `"StmtFunds"` should be `"StatementOfFundsLine"` (the actual XML element tag in IBKR Flex reports). Already fixed locally, confirmed 564 raw rows → 105 normalized MTM events extracting correctly with -$42,945 total cash impact.

2. **Ghost account in per-account aggregation** — `_discover_account_ids()` discovers 2 accounts that are the same IBKR account:
   - `"Interactive Brokers (Henry Chien)"` from SnapTrade position data (`account_name`)
   - `"U2471778"` from Flex transaction data (`account_id`, since Flex `acctAlias` is empty)

   The ghost "Interactive Brokers (Henry Chien)" account matches **0 transactions** → synthetic-only analysis → March 2025 extreme return (+641%) → dominates aggregated headline.

## Root Cause

`_discover_account_ids()` at `core/realized_performance_analysis.py:5177`:
- Positions: prefers `account_name` over `account_id` → gets `"Interactive Brokers (Henry Chien)"`
- Transactions: prefers `account_name` (empty for Flex) → falls to `account_id` → gets `"U2471778"`
- SnapTrade's `account_id` on positions is an internal UUID, NOT the IBKR account number, so it can't reconcile
- `_match_account()` does exact match — "Interactive Brokers (Henry Chien)" ≠ "U2471778"

## Files to Modify

| File | Change |
|------|--------|
| `ibkr/flex.py:1103,1114` | `"StmtFunds"` → `"StatementOfFundsLine"` (already applied locally) |
| `core/realized_performance_analysis.py:5177` | Ghost account filtering in `_discover_account_ids()` |
| `tests/core/test_discover_account_ids.py` | New tests for ghost account fix |

## Implementation

### 1. StmtFunds topic fix (`ibkr/flex.py`) — already applied

Two lines in `fetch_ibkr_flex_payload()`:
- Line 1103: topic detection `"StmtFunds"` → `"StatementOfFundsLine"`
- Line 1114: row extraction `"StmtFunds"` → `"StatementOfFundsLine"`

### 2. Ghost account fix (`core/realized_performance_analysis.py`)

Replace `_discover_account_ids()` to track position-derived vs transaction-derived accounts separately, then filter out position-only accounts that are **display names** (not real account IDs) with zero matching transactions.

Key insight from Codex review: the original approach was too aggressive — it would remove legitimate dormant accounts (real accounts with positions but no in-period trades). The fix narrows the filter to only remove candidates that look like **brokerage display names** (e.g. "Interactive Brokers (Henry Chien)") rather than real account identifiers (e.g. "U2471778", "Individual ...XXXX").

We use the existing `_is_ibkr_identity_field()` helper (line 661) and a new `_looks_like_display_name()` check: a candidate is a display name if it contains the institution name (via `match_institution`) or doesn't match any known account ID pattern. Real account IDs (IBKR `U\d+`, Schwab masked numbers) are never filtered.

```python
def _looks_like_display_name(candidate: str, institution: str) -> bool:
    """Return True if candidate looks like a provider display name, not a real account ID.

    Display names like "Interactive Brokers (Henry Chien)" are generated by
    aggregators (SnapTrade) and can't be matched against transaction account_id
    fields from native data sources (e.g. IBKR Flex "U2471778").

    Real account IDs (IBKR U-numbers, Schwab masked numbers) are never treated
    as display names, so dormant accounts with real IDs are preserved.
    """
    normalized = candidate.strip().lower()
    if not normalized:
        return False
    # If it matches an IBKR account ID pattern (U1234567), it's a real ID
    if _IBKR_ACCOUNT_ID_RE.match(normalized.replace(" ", "")):
        return False
    # If it contains the institution keyword, it's a display name
    if match_institution(normalized, institution):
        return True
    # If it contains common display-name patterns (parentheses, long names)
    if "(" in candidate or len(candidate) > 30:
        return True
    return False


def _discover_account_ids(
    positions: "PositionResult",
    fifo_transactions: List[Dict[str, Any]],
    institution: str,
) -> List[str]:
    """Discover account IDs from positions and normalized transactions.

    Position-derived display names (e.g. "Interactive Brokers (Henry Chien)")
    that match zero transactions are removed when transaction-derived accounts
    exist.  Only display names are filtered — real account IDs (IBKR U-numbers,
    Schwab masked numbers) are always preserved even if dormant.
    """
    from_positions: set[str] = set()
    from_transactions: set[str] = set()

    for pos in list(getattr(getattr(positions, "data", None), "positions", []) or []):
        brokerage_name = str(pos.get("brokerage_name") or pos.get("institution") or "")
        if not match_institution(brokerage_name, institution):
            continue
        account_name = str(pos.get("account_name") or "").strip()
        if account_name:
            from_positions.add(account_name)
            continue
        account_id = str(pos.get("account_id") or "").strip()
        if account_id:
            from_positions.add(account_id)

    for txn in fifo_transactions or []:
        txn_institution = str(txn.get("_institution") or txn.get("institution") or "")
        if not match_institution(txn_institution, institution):
            continue
        account_name = str(txn.get("account_name") or "").strip()
        if account_name:
            from_transactions.add(account_name)
            continue
        account_id = str(txn.get("account_id") or "").strip()
        if account_id:
            from_transactions.add(account_id)

    seen = from_positions | from_transactions

    # Remove position-only DISPLAY NAMES with zero matching transactions.
    # Real account IDs (U2471778, etc.) are never removed — only names that
    # look like provider-generated display names are candidates.
    position_only = from_positions - from_transactions
    if position_only and from_transactions:
        institution_txns = [
            txn for txn in (fifo_transactions or [])
            if match_institution(
                str(txn.get("_institution") or txn.get("institution") or ""),
                institution,
            )
        ]
        for candidate in list(position_only):
            if not _looks_like_display_name(candidate, institution):
                continue  # Real account ID — keep even if dormant
            if not any(_match_account(txn, candidate) for txn in institution_txns):
                seen.discard(candidate)

    return sorted(seen)
```

**Design decisions:**
- **Only display names are filtered** — real account IDs (IBKR `U\d+`, Schwab masked numbers) are always preserved, even if dormant with no trades. This addresses Codex concern about dormant buy-and-hold accounts.
- **`_looks_like_display_name()`** — conservative check: must contain institution keyword OR have parentheses/long length. IBKR account ID regex match = definitely not a display name.
- Only filter when `from_transactions` is non-empty — preserves pure position-only scenarios
- Uses existing `_match_account()` — checks both `account_id` and `account_name` for exact case-insensitive match
- Pre-filters transactions by institution — avoids cross-institution false matches
- Schwab unaffected — Schwab positions and transactions share the same `account_name`, so `position_only` is empty

### 3. Tests (`tests/core/test_discover_account_ids.py`)

**Ghost account filtering:**
1. **`test_ghost_display_name_filtered`** — Position "Interactive Brokers (Henry)" + txn account_id "U123" → only "U123" returned
2. **`test_matching_accounts_kept`** — Position and txn share same account_name → kept (Schwab pattern)
3. **`test_position_only_preserved_when_no_transactions`** — Position "Acct A" + zero txns → "Acct A" preserved (buy-and-hold)
4. **`test_multiple_real_accounts`** — Multiple txn-derived accounts all preserved
5. **`test_mixed_match_and_ghost`** — Some position accounts match txns, some don't → only display-name ghosts removed

**Dormant account protection (Codex concern #1):**
6. **`test_dormant_real_account_id_preserved`** — Position "U9999999" (real IBKR ID) + txn "U123" → both preserved (dormant real account NOT filtered)
7. **`test_dormant_schwab_account_preserved`** — Position "Individual ...1234" + txn "Individual ...5678" → both preserved

**Display name detection:**
8. **`test_looks_like_display_name_ibkr_id`** — "U2471778" → False (real ID)
9. **`test_looks_like_display_name_brokerage_name`** — "Interactive Brokers (Henry)" → True
10. **`test_looks_like_display_name_short_unknown`** — "MyAcct" → False (conservative — short name without institution keyword)

**Edge cases (Codex concern #5):**
11. **`test_cross_institution_no_false_removal`** — IBKR position + Schwab txn → no cross-institution ghost filtering
12. **`test_missing_account_fields`** — Positions/txns with empty account_name and account_id → gracefully ignored

## Verification

1. `python3 -m pytest tests/ibkr/test_flex.py tests/core/test_discover_account_ids.py -v`
2. Live test after `/mcp` reconnect: `get_performance(mode="realized", source="ibkr_flex", use_cache=false)`
3. Confirm:
   - StmtFunds warning gone
   - Account aggregation shows 1 account (U2471778), not 2
   - Total return closer to -11% baseline (was +111%)
   - March extreme month reduced
