"""get_portfolio_data -- Bulk portfolio snapshot from the LMA.

Runs 2 SOQL queries in parallel:
  1. All active/trial production licenses linked to an Account
  2. Exception licenses (Company__c-only, from internal overrides)

Version currency is sourced from the local PE cache (.cache/),
populated by load_compass_data. If the cache is missing or expired,
the tool returns CACHE_REQUIRED and stops.

Groups by customer, computes deterministic health scores (renewal risk,
version currency, seat utilization, product breadth, composite, tier,
opportunity flags) using scoring-rules.md, and returns a ranked view.
Writes full data with scores to portfolio/portfolio-data.json.

The AI interprets and presents the pre-computed scores. The scoring
rules file (portfolio/scoring-rules.md) is auto-created with sensible
defaults on first run, auto-migrated when stale, and fully editable
by the user.
"""

import asyncio
import json
import logging
import re
from collections import defaultdict
from datetime import datetime, timezone
from pathlib import Path
from typing import Any

from ..utils import pe_helpers, sf_cli, state
from ..utils.pe_helpers import DEFAULT_NAMESPACE_MAPPING

logger = logging.getLogger(__name__)

DEFAULT_SCORING_RULES = """\
# Portfolio Health Scoring Rules

> Edit this file to customise thresholds, weights, and core packages.
> Changes take effect on the next `get_portfolio_data` call.
> You can also override any rule conversationally during analysis
> (e.g. "for this analysis, treat csbb as critical").

## Dimension Weights

| Dimension          | Weight |
|--------------------|--------|
| Renewal Risk       | 0.30   |
| Version Currency   | 0.30   |
| Seat Utilization   | 0.20   |
| Product Breadth    | 0.20   |

## Core Packages

These packages carry **2x weight** in seat utilization scoring.
Add or remove packages as needed.

- Orchestrator (CSPOFA)
- CloudSense CPQ (Configurator) (cscfga)
- CloudSense Order Management (csord)
- CloudSense Sales Console (BB) (csbb)
- CloudSense Solution Management (csslm)
- CloudSense Order Codebase (csordcb)
- CS Order Telco A (csordtelcoa)
- Product & Pricing Data Model (cspmb)

## 1. Renewal Risk

Renewal discussions typically take 2-3 months (legal, procurement,
sign-off). Thresholds reflect this lead time.

**IMPORTANT:** Use `upcoming_expiry_days` (not `nearest_expiry_days`)
for renewal scoring. `nearest_expiry_days` includes already-expired
packages and can be misleadingly negative. `upcoming_expiry_days` is
the nearest FUTURE expiry -- the next actual renewal deadline.

Per-package `expiry_days` is also available for detailed analysis
(e.g. which specific packages expire when).

| Window                     | Label    | Score |
|----------------------------|----------|-------|
| < 90 days                  | CRITICAL | 0     |
| 90 - 180 days              | URGENT   | 25    |
| 180 - 365 days             | WATCH    | 50    |
| 1 - 2 years                | HEALTHY  | 75    |
| > 2 years or non-expiring  | SAFE     | 100   |

## 2. Version Currency

Each package has a `releases_behind` count: the number of seasonal
releases (with actual package updates) between the customer's
installed version and the latest available version. This is sourced
from the PE cache and stays accurate as new releases appear.

Scoring uses the customer's **max_releases_behind** (worst gap
across all their packages):

| Gap (releases behind) | Label        | Score |
|-----------------------|--------------|-------|
| 0                     | CURRENT      | 100   |
| 1 - 2                 | NEAR CURRENT | 80    |
| 3 - 5                 | BEHIND       | 55    |
| 6 - 10                | FAR BEHIND   | 30    |
| > 10                  | LEGACY       | 0     |
| null / unknown        | UNKNOWN      | 50    |

When `max_releases_behind` is null, the package version could not be
resolved against the PE cache (e.g. package not in cache, or version
not found). Score 50 is a neutral default -- do not treat it as a
real assessment. Flag these customers as having incomplete version
data in any report.

## 3. Seat Utilization

Scoring is non-linear -- both extremes are signals.

| Utilization   | Label     | Score | Signal             |
|---------------|-----------|-------|--------------------|
| 50 - 80%      | HEALTHY   | 100   | sweet spot         |
| 80 - 90%      | GROWING   | 75    | potential upsell   |
| > 90%         | CAPACITY  | 50    | upsell urgently    |
| 30 - 50%      | LOW       | 50    | some concern       |
| < 30%         | UNDERUSED | 25    | churn risk         |
| < 10%         | DORMANT   | 0     | high churn risk    |
| Site License  | NEUTRAL   | 80    | no signal          |

For overall customer seat score: weighted average across packages,
with core packages at 2x weight (see Core Packages above).

## 4. Product Breadth

CloudSense has 50+ packages. More packages = deeper integration =
stickier customer.

| Count       | Label      | Score |
|-------------|------------|-------|
| > 20        | ENTERPRISE | 100   |
| 15 - 20     | DEEP       | 80    |
| 10 - 14     | MODERATE   | 60    |
| 5 - 9       | LIGHT      | 40    |
| 1 - 4       | MINIMAL    | 20    |

## Composite Health Score

```
overall = (renewal × 0.30) + (currency × 0.30) +
          (utilization × 0.20) + (breadth × 0.20)
```

| Score  | Tier      |
|--------|-----------|
| > 80   | HEALTHY   |
| 60 - 80| ATTENTION |
| 40 - 60| AT RISK   |
| < 40   | CRITICAL  |

## Opportunity Detection

Flag these automatically when presenting portfolio analysis:

| Signal     | Trigger                                                     |
|------------|-------------------------------------------------------------|
| UPSELL     | Seat utilization > 90% on any core package                  |
| CROSS-SELL | Missing companion packages (e.g. has cscfga but not CSPOFA) |
| UPGRADE    | max_releases_behind >= 3                                    |
| AT-RISK    | Utilization < 30% + expiry < 180 days                       |
| STRATEGIC  | Large customer (>10 packages) + composite < 60              |

## License Status Nuance

A customer may have both "Active" and "Trial" licenses across
different orgs. "Trial" does NOT always mean prospect -- a customer
may have converted to Active in one production org while a legacy
Trial license remains on another. Only flag as a conversion
opportunity if ALL their licenses are Trial with no Active licenses.

## License Expiry Interpretation

The LMA stores expiry dates per license. Many are stale or
misleading. Use these rules to avoid hallucinating renewal
opportunities or misclassifying customers:

| Condition | Interpretation | Action |
|-----------|---------------|--------|
| Active + expired > 2 years ago | Perpetual / non-expiring license (stale LMA date) | Classify as SAFE -- not a renewal concern |
| Active + expired < 1 year ago | Operating on lapsed agreement | Flag as RENEWAL OVERDUE -- genuine action needed |
| Active + expiring soon (positive days) | Active renewal pipeline | Score normally with renewal risk thresholds |
| Trial-only + expired > 1 year ago | Never converted prospect | Exclude from active health scoring. Flag as WIN-BACK or LOST PROSPECT |
| Trial-only + recently expired or active | Conversion opportunity | Flag as CONVERSION -- not a renewal |

When `nearest_expiry_days` is very negative (e.g. -3000+) but the
customer has Active status and healthy seat utilization, this is
almost always a perpetual license with a stale expiry date in the
LMA. Do NOT treat these as renewal risks or lost customers.

When multiple customers share the exact same `nearest_expiry_days`
(e.g. 5+ customers at -156), flag this pattern. It may indicate a
master agreement covering multiple customers or a bulk data update
in the LMA.

## Portfolio Segmentation

Use account context for grouping and filtering:

- **By Region**: EMEA, APAC, Americas -- useful for regional teams
- **By Industry**: Telco/Comms, Media, Utilities -- vertical insights
- **By Parent Company**: Group subsidiaries under parent for
  strategic account views and cross-sell detection
- **By Health Tier**: HEALTHY / ATTENTION / AT RISK / CRITICAL

## Report Personas

When the user asks for a report, tailor it to the audience.

### Executive (VP / C-level)
- 1 page max, no jargon
- Key numbers: total customers, revenue at risk, top 3 actions
- Health distribution: how many HEALTHY / ATTENTION / AT RISK / CRITICAL
- Strategic risks and opportunities (3-5 bullet points)

### Account Manager (Sales)
- Per-customer action items: upsell, cross-sell, renewal
- Seat utilization signals (who needs more licenses?)
- Corporate group opportunities
- Sort by revenue impact / urgency

### Technical Lead (Engineering)
- Version currency detail: which packages, how far behind
- Package complexity per customer
- Upgrade effort indicators
- Sandbox readiness for metadata retrieval

### Customer Success
- Health trends (improving / declining -- requires historical data)
- Engagement signals: utilization trends, package adoption
- Churn risk indicators with recommended actions
- Relationship timeline (tenure, recent changes)
"""

LICENSE_FIELDS = (
    "sfLma__Account__r.Name, "
    "sfLma__Account__r.Id, "
    "sfLma__Account__r.Region__c, "
    "sfLma__Account__r.Country__c, "
    "sfLma__Account__r.Industry, "
    "sfLma__Account__r.ParentId, "
    "sfLma__Account__r.Parent.Name, "
    "Package_Name__c, "
    "Package_Version_Number__c, "
    "sfLma__License_Status__c, "
    "sfLma__Licensed_Seats__c, "
    "sfLma__Used_Licenses__c, "
    "sfLma__Expiration_Date__c, "
    "Days_To_Expire__c, "
    "sfLma__Package_Version__r.sfLma__Package__c"
)

TOOL_DEFINITION = {
    "name": "get_portfolio_data",
    "description": (
        "Fetch a snapshot of ALL active/trial production licenses from the "
        "CloudSense License Management Portal, enriched with seat utilization "
        "and version currency data. Groups results by customer and returns a "
        "compact summary with health-scoring inputs. Writes full data to "
        "portfolio/portfolio-data.json. Use this for cross-customer analysis: "
        "renewal risks, upgrade opportunities, upsell signals, portfolio "
        "health scoring. Requires connect_lma first."
    ),
    "inputSchema": {
        "type": "object",
        "properties": {
            "working_directory": {
                "type": "string",
                "description": "Workspace directory. The AI passes this automatically.",
            },
        },
        "required": [],
    },
}



def _parse_seats(licensed: Any, used: Any) -> dict[str, Any]:
    """Parse seat utilization into a compact structure."""
    if licensed is None and used is None:
        return {"type": "unknown"}
    if isinstance(licensed, str) and licensed.lower() == "site license":
        return {"type": "site"}
    try:
        total = int(licensed) if licensed is not None else None
        active = int(used) if used is not None else 0
        if total and total > 0:
            pct = round((active / total) * 100)
            return {"type": "numeric", "licensed": total, "used": active, "pct": pct}
        return {"type": "numeric", "licensed": total, "used": active, "pct": 0}
    except (ValueError, TypeError):
        return {"type": "other", "raw_licensed": str(licensed)}


def _query_all_active_licenses(lma_alias: str) -> list[dict[str, Any]]:
    query = (
        f"SELECT {LICENSE_FIELDS} "
        f"FROM sfLma__License__c "
        f"WHERE sfLma__Is_Sandbox__c = false "
        f"AND sfLma__License_Status__c IN ('Active', 'Trial') "
        f"AND sfLma__Account__r.Name != null "
        f"ORDER BY sfLma__Account__r.Name"
    )
    return sf_cli.lma_query(query, lma_alias)



def _group_by_customer(
    licenses: list[dict[str, Any]],
) -> dict[str, dict[str, Any]]:
    """Group Account-linked licenses by customer. Returns raw (mutable) dicts.

    The returned dicts use sets for statuses and defaultdicts for packages,
    allowing further merging before finalisation.
    """
    customers: dict[str, dict[str, Any]] = {}

    for lic in licenses:
        acct_ref = lic.get("sfLma__Account__r")
        if not isinstance(acct_ref, dict):
            continue
        acct_name = acct_ref.get("Name", "Unknown")
        acct_id = acct_ref.get("Id", "")

        region = acct_ref.get("Region__c") or ""
        country = acct_ref.get("Country__c") or ""
        industry = acct_ref.get("Industry") or ""
        parent_id = acct_ref.get("ParentId") or ""
        parent_ref = acct_ref.get("Parent")
        parent_name = parent_ref.get("Name") if isinstance(parent_ref, dict) else ""

        if acct_name not in customers:
            customers[acct_name] = {
                "account_id": acct_id,
                "region": region,
                "country": country,
                "industry": industry,
                "parent_id": parent_id,
                "parent_name": parent_name or "",
                "packages": defaultdict(lambda: {
                    "versions": [], "seats": [], "expiry_days": [],
                    "package_id": None,
                }),
                "statuses": set(),
                "nearest_expiry_days": None,
                "license_count": 0,
            }

        cust = customers[acct_name]
        cust["license_count"] += 1

        status = lic.get("sfLma__License_Status__c") or "Unknown"
        cust["statuses"].add(status)

        pkg_name = lic.get("Package_Name__c") or "Unknown"
        pkg_data = cust["packages"][pkg_name]
        pkg_data["versions"].append(
            lic.get("Package_Version_Number__c") or "?"
        )

        pkg_ver_ref = lic.get("sfLma__Package_Version__r")
        if isinstance(pkg_ver_ref, dict):
            pid = pkg_ver_ref.get("sfLma__Package__c")
            if pid:
                pkg_data["package_id"] = pid

        seat_info = _parse_seats(
            lic.get("sfLma__Licensed_Seats__c"),
            lic.get("sfLma__Used_Licenses__c"),
        )
        pkg_data["seats"].append(seat_info)

        days = lic.get("Days_To_Expire__c")
        try:
            d = float(days) if days is not None else None
        except (ValueError, TypeError):
            d = None
        pkg_data["expiry_days"].append(d)

        if d is not None:
            if cust["nearest_expiry_days"] is None or d < cust["nearest_expiry_days"]:
                cust["nearest_expiry_days"] = d

    return customers


def _finalise_customers(
    customers: dict[str, dict[str, Any]],
    cached_data: dict[str, Any],
) -> dict[str, dict[str, Any]]:
    """Convert raw customer dicts into serialisable summaries."""
    result = {}
    for name, cust in customers.items():
        pkg_summaries = {}
        max_gap: int | None = None

        for pkg_name, pkg_data in cust["packages"].items():
            versions = sorted(set(pkg_data["versions"]))

            sr_info: dict[str, Any] = {}
            ns = pe_helpers.resolve_lma_name_to_namespace(pkg_name, cached_data)
            if ns:
                valid_versions = [v for v in versions if v != "?"]
                best_installed = valid_versions[-1] if valid_versions else ""
                sr_info = pe_helpers.get_installed_and_latest_sr(
                    best_installed, ns, cached_data,
                )

            display_name = sr_info.get("pe_name") or pkg_name
            seat_summary = _summarise_seats(pkg_data["seats"])
            pkg_expiry = _summarise_expiry(pkg_data.get("expiry_days", []))

            pkg_summaries[display_name] = {
                "lma_name": pkg_name,
                "versions": versions,
                "installed_sr": sr_info.get("installed_sr"),
                "latest_sr": sr_info.get("latest_sr"),
                "latest_version": sr_info.get("latest_version"),
                "on_latest": sr_info.get("on_latest"),
                "releases_behind": sr_info.get("releases_behind"),
                "seats": seat_summary,
                "expiry_days": pkg_expiry,
            }

            gap = sr_info.get("releases_behind")
            if gap is not None:
                if max_gap is None or gap > max_gap:
                    max_gap = gap

        all_pkg_expiries = [
            pkg["expiry_days"] for pkg in pkg_summaries.values()
            if pkg["expiry_days"] is not None
        ]
        upcoming = [d for d in all_pkg_expiries if d > 0]

        result[name] = {
            "account_id": cust["account_id"],
            "region": cust["region"],
            "country": cust["country"],
            "industry": cust["industry"],
            "parent_id": cust["parent_id"],
            "parent_name": cust["parent_name"],
            "license_count": cust["license_count"],
            "package_count": len(cust["packages"]),
            "statuses": sorted(cust["statuses"]),
            "packages": pkg_summaries,
            "nearest_expiry_days": cust["nearest_expiry_days"],
            "upcoming_expiry_days": min(upcoming) if upcoming else None,
            "max_releases_behind": max_gap,
        }
    return result


def _summarise_expiry(expiry_entries: list) -> float | None:
    """Return the most actionable expiry across licenses for the same package.

    Prefers upcoming (positive) expiry — a real renewal deadline — over
    already-expired (negative) values from stale trials or old orgs.
    """
    valid = []
    for d in expiry_entries:
        if d is not None:
            try:
                valid.append(float(d))
            except (ValueError, TypeError):
                pass
    if not valid:
        return None
    upcoming = [d for d in valid if d > 0]
    if upcoming:
        return min(upcoming)
    return max(valid)


def _summarise_seats(seat_entries: list[dict[str, Any]]) -> dict[str, Any]:
    """Aggregate seat data across multiple licenses for the same package."""
    numeric = [s for s in seat_entries if s["type"] == "numeric"]
    if numeric:
        total_licensed = sum(s.get("licensed") or 0 for s in numeric)
        total_used = sum(s.get("used") or 0 for s in numeric)
        pct = round((total_used / total_licensed) * 100) if total_licensed > 0 else 0
        return {"type": "numeric", "licensed": total_licensed, "used": total_used, "pct": pct}
    site = [s for s in seat_entries if s["type"] == "site"]
    if site:
        return {"type": "site"}
    return {"type": "unknown"}


def _group_by_parent(
    customers: dict[str, dict[str, Any]],
) -> dict[str, dict[str, Any]]:
    """Group customers by parent company. Returns {parent_name: {subsidiaries, ...}}."""
    groups: dict[str, dict[str, Any]] = {}
    for cust_name, cust in customers.items():
        pid = cust.get("parent_id")
        if not pid:
            continue
        pname = cust.get("parent_name") or pid
        if pname not in groups:
            groups[pname] = {
                "parent_id": pid,
                "subsidiaries": [],
                "total_licenses": 0,
                "total_packages": 0,
                "max_releases_behind": None,
                "nearest_expiry_days": None,
                "upcoming_expiry_days": None,
            }
        grp = groups[pname]
        grp["subsidiaries"].append(cust_name)
        grp["total_licenses"] += cust["license_count"]
        grp["total_packages"] += cust["package_count"]

        cust_gap = cust.get("max_releases_behind")
        if cust_gap is not None:
            if grp["max_releases_behind"] is None or cust_gap > grp["max_releases_behind"]:
                grp["max_releases_behind"] = cust_gap

        cust_exp = cust.get("nearest_expiry_days")
        if cust_exp is not None:
            if grp["nearest_expiry_days"] is None or cust_exp < grp["nearest_expiry_days"]:
                grp["nearest_expiry_days"] = cust_exp

        cust_upcoming = cust.get("upcoming_expiry_days")
        if cust_upcoming is not None:
            if grp["upcoming_expiry_days"] is None or cust_upcoming < grp["upcoming_expiry_days"]:
                grp["upcoming_expiry_days"] = cust_upcoming

    return {k: v for k, v in groups.items() if len(v["subsidiaries"]) > 0}


def _write_history_snapshot(working_dir: Path, data: dict[str, Any]) -> Path | None:
    """Write a timestamped snapshot to portfolio/history/. Returns the path."""
    history_dir = working_dir / "portfolio" / "history"
    history_dir.mkdir(parents=True, exist_ok=True)
    today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
    snapshot_path = history_dir / f"portfolio-data-{today}.json"
    if snapshot_path.exists():
        return None
    snapshot_path.write_text(json.dumps(data, indent=2, default=str))
    return snapshot_path


def _detect_changes(
    working_dir: Path,
    current_customers: dict[str, dict[str, Any]],
) -> list[str]:
    """Compare current snapshot with the most recent previous one."""
    history_dir = working_dir / "portfolio" / "history"
    if not history_dir.exists():
        return []

    snapshots = sorted(history_dir.glob("portfolio-data-*.json"), reverse=True)
    previous = [s for s in snapshots if s != max(snapshots)] if len(snapshots) > 1 else []
    if not previous:
        if snapshots:
            try:
                prev_data = json.loads(snapshots[0].read_text())
                prev_customers = prev_data.get("customers", {})
                if prev_customers == current_customers:
                    return []
            except (json.JSONDecodeError, OSError):
                pass
        return []

    prev_path = previous[0]
    try:
        prev_data = json.loads(prev_path.read_text())
    except (json.JSONDecodeError, OSError):
        return []

    prev_customers = prev_data.get("customers", {})
    prev_date = prev_path.stem.replace("portfolio-data-", "")
    changes: list[str] = []

    curr_names = set(current_customers.keys())
    prev_names = set(prev_customers.keys())

    new_custs = curr_names - prev_names
    if new_custs:
        for n in sorted(new_custs):
            lc = current_customers[n]["license_count"]
            changes.append(f"NEW: {n} ({lc} licenses)")

    lost_custs = prev_names - curr_names
    if lost_custs:
        for n in sorted(lost_custs):
            changes.append(f"REMOVED: {n} (no longer has active licenses)")

    for name in sorted(curr_names & prev_names):
        curr = current_customers[name]
        prev = prev_customers.get(name, {})

        curr_gap = curr.get("max_releases_behind")
        prev_gap = prev.get("max_releases_behind")
        if curr_gap is not None and prev_gap is not None and curr_gap != prev_gap:
            direction = "reduced" if curr_gap < prev_gap else "increased"
            changes.append(f"VERSION: {name} gap {direction} {prev_gap} -> {curr_gap}")

        curr_pkgs = curr.get("package_count", 0)
        prev_pkgs = prev.get("package_count", 0)
        diff = curr_pkgs - prev_pkgs
        if diff > 0:
            changes.append(f"GROWTH: {name} added {diff} package(s)")
        elif diff < 0:
            changes.append(f"SHRINK: {name} dropped {abs(diff)} package(s)")

        curr_statuses = set(curr.get("statuses", []))
        prev_statuses = set(prev.get("statuses", []))
        if curr_statuses != prev_statuses:
            changes.append(
                f"STATUS: {name} changed {', '.join(prev_statuses)} -> {', '.join(curr_statuses)}"
            )

    if changes:
        changes.insert(0, f"_Compared with snapshot from {prev_date}_\n")

    return changes


_NEW_CORE_SECTION = """\
## Core Packages

These packages carry **2x weight** in seat utilization scoring.
Add or remove packages as needed.

- Orchestrator (CSPOFA)
- CloudSense CPQ (Configurator) (cscfga)
- CloudSense Order Management (csord)
- CloudSense Sales Console (BB) (csbb)
- CloudSense Solution Management (csslm)
- CloudSense Order Codebase (csordcb)
- CS Order Telco A (csordtelcoa)
- Product & Pricing Data Model (cspmb)

"""


def _migrate_scoring_rules(rules_path: Path) -> bool:
    """Migrate stale scoring-rules.md sections.

    Handles two migrations:
    1. Version currency: R-number tables -> releases_behind
    2. Core packages: namespace prefixes -> PE display names

    Returns True if the file was updated.
    """
    try:
        content = rules_path.read_text()
    except OSError:
        return False

    changed = False

    if "oldest release" in content:
        new_vc_section = (
            "## 2. Version Currency\n"
            "\n"
            "Each package has a `releases_behind` count: the number of seasonal\n"
            "releases (with actual package updates) between the customer's\n"
            "installed version and the latest available version. This is sourced\n"
            "from the PE cache and stays accurate as new releases appear.\n"
            "\n"
            "Scoring uses the customer's **max_releases_behind** (worst gap\n"
            "across all their packages):\n"
            "\n"
            "| Gap (releases behind) | Label        | Score |\n"
            "|-----------------------|--------------|-------|\n"
            "| 0                     | CURRENT      | 100   |\n"
            "| 1 - 2                 | NEAR CURRENT | 80    |\n"
            "| 3 - 5                 | BEHIND       | 55    |\n"
            "| 6 - 10                | FAR BEHIND   | 30    |\n"
            "| > 10                  | LEGACY       | 0     |\n"
            "\n"
        )
        start_idx = content.find("## 2. Version Currency")
        end_idx = content.find("## 3. Seat Utilization")
        if start_idx >= 0 and end_idx > start_idx:
            content = content[:start_idx] + new_vc_section + content[end_idx:]
            changed = True

        content = content.replace(
            "Oldest release <= R35 (two or more behind)",
            "max_releases_behind >= 3",
        )

    if "- CSPOFA" in content:
        core_start = content.find("## Core Packages")
        core_end = content.find("## 1. Renewal Risk")
        if core_start >= 0 and core_end > core_start:
            content = content[:core_start] + _NEW_CORE_SECTION + content[core_end:]
            changed = True

    if changed:
        rules_path.write_text(content)
        logger.warning(
            "Migrated scoring rules at %s to current format",
            rules_path,
        )

    return changed


def _ensure_scoring_rules(working_dir: Path) -> Path:
    """Create scoring-rules.md with defaults if it doesn't exist. Returns the path."""
    portfolio_dir = working_dir / "portfolio"
    portfolio_dir.mkdir(parents=True, exist_ok=True)
    rules_path = portfolio_dir / "scoring-rules.md"
    if not rules_path.exists():
        rules_path.write_text(DEFAULT_SCORING_RULES)
        logger.info("Created default scoring rules at %s", rules_path)
    else:
        _migrate_scoring_rules(rules_path)
    return rules_path


def _read_scoring_rules(rules_path: Path) -> str:
    """Read the scoring rules file content."""
    try:
        return rules_path.read_text()
    except OSError:
        logger.warning("Could not read scoring rules at %s, using defaults", rules_path)
        return DEFAULT_SCORING_RULES


# ---------------------------------------------------------------------------
# Deterministic scoring
# ---------------------------------------------------------------------------

_DEFAULT_WEIGHTS = {
    "renewal": 0.30, "currency": 0.30, "seats": 0.20, "breadth": 0.20,
}
_DEFAULT_CORE = [
    "Orchestrator",
    "CloudSense CPQ (Configurator)",
    "CloudSense Order Management",
    "CloudSense Sales Console (BB)",
    "CloudSense Solution Management",
    "CloudSense Order Codebase",
    "CS Order Telco A",
    "Product & Pricing Data Model",
]

_DEFAULT_RENEWAL_TIERS = [
    (90, "CRITICAL", 0), (180, "URGENT", 25), (365, "WATCH", 50),
    (730, "HEALTHY", 75), (float("inf"), "SAFE", 100),
]
_DEFAULT_CURRENCY_TIERS = [
    (0, "CURRENT", 100), (2, "NEAR CURRENT", 80), (5, "BEHIND", 55),
    (10, "FAR BEHIND", 30), (float("inf"), "LEGACY", 0),
]
_DEFAULT_SEAT_TIERS = [
    (10, "DORMANT", 0), (30, "UNDERUSED", 25), (50, "LOW", 50),
    (80, "HEALTHY", 100), (90, "GROWING", 75), (float("inf"), "CAPACITY", 50),
]
_DEFAULT_BREADTH_TIERS = [
    (4, "MINIMAL", 20), (9, "LIGHT", 40), (14, "MODERATE", 60),
    (20, "DEEP", 80), (float("inf"), "ENTERPRISE", 100),
]
_DEFAULT_HEALTH_TIERS = [
    (40, "CRITICAL"), (60, "AT RISK"), (80, "ATTENTION"), (float("inf"), "HEALTHY"),
]


def _parse_weights(text: str) -> dict[str, float]:
    """Extract dimension weights from the Dimension Weights table."""
    weights: dict[str, float] = {}
    for m in re.finditer(
        r"\|\s*(Renewal Risk|Version Currency|Seat Utilization|Product Breadth)"
        r"\s*\|\s*([\d.]+)\s*\|",
        text, re.IGNORECASE,
    ):
        key_map = {
            "renewal risk": "renewal", "version currency": "currency",
            "seat utilization": "seats", "product breadth": "breadth",
        }
        key = key_map.get(m.group(1).lower())
        if key:
            weights[key] = float(m.group(2))
    return weights if len(weights) == 4 else _DEFAULT_WEIGHTS


def _parse_core_packages(text: str) -> list[str]:
    """Extract core package PE display names from the bullet list.

    Expects lines like: ``- Orchestrator (CSPOFA)``
    Captures everything before the trailing ``(namespace)`` hint.
    """
    cores: list[str] = []
    in_core = False
    for line in text.splitlines():
        if "core packages" in line.lower() and line.startswith("#"):
            in_core = True
            continue
        if in_core:
            if line.startswith("#"):
                break
            m = re.match(r"^-\s+(.+?)(?:\s+\([a-zA-Z]+\)\s*$|$)", line)
            if m and m.group(1).strip():
                cores.append(m.group(1).strip())
    return cores if cores else _DEFAULT_CORE


def _parse_scoring_rules(rules_text: str) -> dict[str, Any]:
    """Parse scoring-rules.md into structured thresholds for scoring."""
    try:
        weights = _parse_weights(rules_text)
        core = _parse_core_packages(rules_text)
    except Exception:
        logger.warning("Failed to parse scoring rules, using defaults")
        weights = _DEFAULT_WEIGHTS
        core = _DEFAULT_CORE

    return {
        "weights": weights,
        "core_packages": core,
        "renewal_tiers": _DEFAULT_RENEWAL_TIERS,
        "currency_tiers": _DEFAULT_CURRENCY_TIERS,
        "seat_tiers": _DEFAULT_SEAT_TIERS,
        "breadth_tiers": _DEFAULT_BREADTH_TIERS,
        "health_tiers": _DEFAULT_HEALTH_TIERS,
    }


def _build_core_set(core_list: list[str]) -> set[str]:
    """Build a comprehensive lowercase set of core package identifiers.

    Expands the parsed core list to include PE names, LMA names, and
    namespace prefixes so matching works regardless of which name form
    appears in the data.
    """
    result = {c.lower() for c in core_list}
    for entry in DEFAULT_NAMESPACE_MAPPING:
        if (entry["pe_name"].lower() in result
                or entry["namespace"].lower() in result
                or entry["lma_name"].lower() in result):
            result.add(entry["pe_name"].lower())
            result.add(entry["lma_name"].lower())
            result.add(entry["namespace"].lower())
    return result


def _score_renewal(
    upcoming_days: float | None,
    nearest_days: float | None,
    statuses: list[str],
    tiers: list[tuple[float, str, int]],
) -> dict[str, Any]:
    """Score renewal risk based on upcoming expiry days."""
    if upcoming_days is None:
        is_active = "Active" in statuses
        if is_active and nearest_days is not None and nearest_days < -730:
            return {"score": 100, "label": "SAFE"}
        return {"score": 100, "label": "SAFE"}

    for threshold, label, score in tiers:
        if upcoming_days < threshold:
            return {"score": score, "label": label}
    return {"score": 100, "label": "SAFE"}


def _score_currency(
    max_gap: int | None,
    tiers: list[tuple[float, str, int]],
) -> dict[str, Any]:
    """Score version currency based on max_releases_behind."""
    if max_gap is None:
        return {"score": 50, "label": "UNKNOWN"}

    for threshold, label, score in tiers:
        if max_gap <= threshold:
            return {"score": score, "label": label}
    return {"score": 0, "label": "LEGACY"}


def _score_seats(
    packages: dict[str, dict[str, Any]],
    core_set: set[str],
    tiers: list[tuple[float, str, int]],
) -> dict[str, Any]:
    """Score seat utilization as weighted average across packages.

    The label is derived from the weighted average *utilization %*
    (not the score), because the score scale is non-monotonic while
    the tier thresholds are utilization percentages.
    """
    score_sum = 0.0
    pct_sum = 0.0
    total_weight = 0.0
    pct_weight = 0.0

    for pkg_name, pkg in packages.items():
        seats = pkg.get("seats", {})
        seat_type = seats.get("type", "unknown")
        if seat_type == "unknown":
            continue

        is_core = (pkg_name.lower() in core_set
                   or pkg.get("lma_name", "").lower() in core_set)
        w = 2.0 if is_core else 1.0

        if seat_type == "site":
            pkg_score = 80
        else:
            pct = seats.get("pct", 0)
            pkg_score = 50
            for threshold, _, score in tiers:
                if pct < threshold:
                    pkg_score = score
                    break
            pct_sum += pct * w
            pct_weight += w

        score_sum += pkg_score * w
        total_weight += w

    if total_weight == 0:
        return {"score": 50, "label": "UNKNOWN"}

    avg_score = round(score_sum / total_weight)
    avg_pct = pct_sum / pct_weight if pct_weight > 0 else 50.0

    label = "HEALTHY"
    for threshold, lbl, _ in tiers:
        if avg_pct < threshold:
            label = lbl
            break

    return {"score": avg_score, "label": label}


def _score_breadth(
    package_count: int,
    tiers: list[tuple[float, str, int]],
) -> dict[str, Any]:
    """Score product breadth based on package count."""
    for threshold, label, score in tiers:
        if package_count <= threshold:
            return {"score": score, "label": label}
    return {"score": 100, "label": "ENTERPRISE"}


def _detect_opportunities(
    cust: dict[str, Any],
    composite: int,
    core_set: set[str],
) -> list[str]:
    """Detect opportunity flags for a customer."""
    opps: list[str] = []
    core_lower = core_set
    packages = cust.get("packages", {})

    for pkg_name, pkg in packages.items():
        seats = pkg.get("seats", {})
        if seats.get("type") == "numeric" and seats.get("pct", 0) > 90:
            is_core = (
                pkg_name.lower() in core_lower
                or pkg.get("lma_name", "").lower() in core_lower
            )
            if is_core:
                opps.append("UPSELL")
                break

    gap = cust.get("max_releases_behind")
    if gap is not None and gap >= 3:
        opps.append("UPGRADE")

    upcoming = cust.get("upcoming_expiry_days")
    if upcoming is not None and upcoming < 180:
        for pkg in packages.values():
            seats = pkg.get("seats", {})
            if seats.get("type") == "numeric" and seats.get("pct", 0) < 30:
                if seats.get("licensed", 0) > 0:
                    opps.append("AT-RISK")
                    break

    matched_core = 0
    for p in packages:
        if (p.lower() in core_lower
                or packages[p].get("lma_name", "").lower() in core_lower):
            matched_core += 1
    if 0 < matched_core < len(_DEFAULT_CORE):
        opps.append("CROSS-SELL")

    if cust.get("package_count", 0) > 10 and composite < 60:
        opps.append("STRATEGIC")

    return opps


def _score_customer(
    cust: dict[str, Any],
    rules: dict[str, Any],
    core_set: set[str],
) -> dict[str, Any]:
    """Compute health scores for a single customer."""
    weights = rules["weights"]

    renewal = _score_renewal(
        cust.get("upcoming_expiry_days"),
        cust.get("nearest_expiry_days"),
        cust.get("statuses", []),
        rules["renewal_tiers"],
    )
    currency = _score_currency(
        cust.get("max_releases_behind"),
        rules["currency_tiers"],
    )
    seats = _score_seats(
        cust.get("packages", {}),
        core_set,
        rules["seat_tiers"],
    )
    breadth = _score_breadth(
        cust.get("package_count", 0),
        rules["breadth_tiers"],
    )

    composite = round(
        renewal["score"] * weights["renewal"]
        + currency["score"] * weights["currency"]
        + seats["score"] * weights["seats"]
        + breadth["score"] * weights["breadth"]
    )

    tier = "HEALTHY"
    for threshold, label in rules["health_tiers"]:
        if composite < threshold:
            tier = label
            break

    opportunities = _detect_opportunities(cust, composite, core_set)

    return {
        "renewal": renewal,
        "currency": currency,
        "seats": seats,
        "breadth": breadth,
        "composite": composite,
        "tier": tier,
        "opportunities": opportunities,
    }


def _score_all_customers(
    customers: dict[str, dict[str, Any]],
    rules: dict[str, Any],
) -> None:
    """Score all customers in-place, adding a 'health' key to each."""
    core_set = _build_core_set(rules["core_packages"])
    for name, cust in customers.items():
        cust["health"] = _score_customer(cust, rules, core_set)


_DEFAULT_OVERRIDES: dict[str, Any] = {
    "_version": 1,
    "unlinked_customers": [
        {
            "name": "Warner Bros. Discovery",
            "company_values": ["Warner Bros. Discovery"],
        },
    ],
}

_OVERRIDES_FILENAME = ".portfolio-overrides.json"


def _ensure_overrides(working_dir: Path) -> Path:
    """Create the internal overrides file with defaults if absent."""
    portfolio_dir = working_dir / "portfolio"
    portfolio_dir.mkdir(parents=True, exist_ok=True)
    path = portfolio_dir / _OVERRIDES_FILENAME
    if not path.exists():
        path.write_text(json.dumps(_DEFAULT_OVERRIDES, indent=2))
        logger.info("Created portfolio overrides at %s", path)
    return path


def _read_overrides(path: Path) -> dict[str, Any]:
    """Read and parse the overrides file. Returns defaults on error."""
    try:
        data = json.loads(path.read_text())
        return data if isinstance(data, dict) else _DEFAULT_OVERRIDES
    except (json.JSONDecodeError, OSError):
        logger.warning("Could not read overrides at %s, using defaults", path)
        return _DEFAULT_OVERRIDES


def _get_unlinked_company_values(
    overrides: dict[str, Any],
) -> tuple[list[str], dict[str, str]]:
    """Extract Company__c values and build a reverse map to canonical names.

    Returns (flat_values, {company_value: canonical_name}).
    """
    entries = overrides.get("unlinked_customers", [])
    all_values: list[str] = []
    reverse_map: dict[str, str] = {}
    for entry in entries:
        canonical = entry.get("name", "")
        for cv in entry.get("company_values", []):
            all_values.append(cv)
            reverse_map[cv] = canonical
    return all_values, reverse_map


_EXCEPTION_LICENSE_FIELDS = (
    "Company__c, "
    "Package_Name__c, "
    "Package_Version_Number__c, "
    "sfLma__License_Status__c, "
    "sfLma__Licensed_Seats__c, "
    "sfLma__Used_Licenses__c, "
    "sfLma__Expiration_Date__c, "
    "Days_To_Expire__c, "
    "sfLma__Package_Version__r.sfLma__Package__c"
)


def _query_unlinked_licenses(
    company_values: list[str], lma_alias: str,
) -> list[dict[str, Any]]:
    """Query production licenses for Company__c-only customers."""
    if not company_values:
        return []
    in_clause = ", ".join(
        f"'{v.replace(chr(39), chr(92) + chr(39))}'" for v in company_values
    )
    query = (
        f"SELECT {_EXCEPTION_LICENSE_FIELDS} "
        f"FROM sfLma__License__c "
        f"WHERE Company__c IN ({in_clause}) "
        f"AND sfLma__Is_Sandbox__c = false "
        f"AND sfLma__Account__c = null "
        f"AND sfLma__License_Status__c IN ('Active', 'Trial') "
        f"ORDER BY Company__c"
    )
    try:
        return sf_cli.lma_query(query, lma_alias)
    except sf_cli.SfCliError as e:
        logger.warning("Unlinked customer license query failed: %s", e)
        return []


def _merge_unlinked_customers(
    customers: dict[str, dict[str, Any]],
    unlinked_licenses: list[dict[str, Any]],
    reverse_map: dict[str, str],
) -> int:
    """Merge unlinked (Company__c-only) licenses into the customer dict.

    Returns the number of licenses merged.
    """
    merged = 0
    for lic in unlinked_licenses:
        raw_company = lic.get("Company__c") or ""
        canonical = reverse_map.get(raw_company, raw_company)
        if not canonical:
            continue

        if canonical not in customers:
            customers[canonical] = {
                "account_id": "",
                "region": "",
                "country": "",
                "industry": "",
                "parent_id": "",
                "parent_name": "",
                "packages": defaultdict(lambda: {
                    "versions": [], "seats": [], "expiry_days": [],
                    "package_id": None,
                }),
                "statuses": set(),
                "nearest_expiry_days": None,
                "license_count": 0,
            }

        cust = customers[canonical]
        cust["license_count"] += 1
        merged += 1

        status = lic.get("sfLma__License_Status__c") or "Unknown"
        cust["statuses"].add(status)

        pkg_name = lic.get("Package_Name__c") or "Unknown"
        pkg_data = cust["packages"][pkg_name]
        pkg_data["versions"].append(
            lic.get("Package_Version_Number__c") or "?"
        )

        pkg_ver_ref = lic.get("sfLma__Package_Version__r")
        if isinstance(pkg_ver_ref, dict):
            pid = pkg_ver_ref.get("sfLma__Package__c")
            if pid:
                pkg_data["package_id"] = pid

        seat_info = _parse_seats(
            lic.get("sfLma__Licensed_Seats__c"),
            lic.get("sfLma__Used_Licenses__c"),
        )
        pkg_data["seats"].append(seat_info)

        days = lic.get("Days_To_Expire__c")
        try:
            d = float(days) if days is not None else None
        except (ValueError, TypeError):
            d = None
        pkg_data["expiry_days"].append(d)

        if d is not None:
            if cust["nearest_expiry_days"] is None or d < cust["nearest_expiry_days"]:
                cust["nearest_expiry_days"] = d

    return merged


def _write_portfolio_json(
    working_dir: Path,
    customers: dict[str, dict[str, Any]],
    total_licenses: int,
    parent_groups: dict[str, dict[str, Any]],
) -> Path:
    portfolio_dir = working_dir / "portfolio"
    portfolio_dir.mkdir(parents=True, exist_ok=True)
    out_path = portfolio_dir / "portfolio-data.json"
    out_path.write_text(json.dumps({
        "_warning": (
            "Raw data only. For portfolio analysis, call get_portfolio_data "
            "via MCP -- the tool response includes scoring rules and analysis "
            "guidance that this file does not contain."
        ),
        "generated_at": datetime.now(timezone.utc).isoformat(),
        "total_licenses": total_licenses,
        "customer_count": len(customers),
        "customers": customers,
        "parent_groups": parent_groups,
    }, indent=2, default=str))
    return out_path


async def run(arguments: dict[str, Any]) -> str:
    """Execute the get_portfolio_data tool."""
    state.set_working_dir(arguments.get("working_directory"))

    cache_result = pe_helpers.validate_and_load_cache(state.get_working_dir())
    if cache_result["status"] != "VALID":
        return (
            "STATUS: CACHE_REQUIRED\n\n"
            f"{cache_result.get('reason', 'Cache is not available.')}\n"
            "Call load_compass_data to refresh, then retry this tool."
        )
    cached_data = cache_result["data"]

    current_state = state.load()
    lma_alias = current_state.get("lma_org_alias") or "cs-lma"

    working_dir = state.get_working_dir()
    rules_path = _ensure_scoring_rules(working_dir)
    scoring_rules = _read_scoring_rules(rules_path)

    overrides_path = _ensure_overrides(working_dir)
    overrides = _read_overrides(overrides_path)
    company_values, reverse_map = _get_unlinked_company_values(overrides)

    loop = asyncio.get_running_loop()
    licenses, unlinked = await asyncio.gather(
        loop.run_in_executor(None, _query_all_active_licenses, lma_alias),
        loop.run_in_executor(None, _query_unlinked_licenses, company_values, lma_alias),
    )

    if not licenses and not unlinked:
        return (
            "STATUS: NO_DATA\n\n"
            "No active production licenses found in the LMA. "
            "Verify connect_lma was called and the LMA org has license data."
        )

    raw_customers = _group_by_customer(licenses)
    unlinked_count = _merge_unlinked_customers(
        raw_customers, unlinked, reverse_map,
    )

    total_licenses = len(licenses) + unlinked_count
    customers = _finalise_customers(raw_customers, cached_data)

    parsed_rules = _parse_scoring_rules(scoring_rules)
    _score_all_customers(customers, parsed_rules)

    parent_groups = _group_by_parent(customers)

    portfolio_data = {
        "generated_at": datetime.now(timezone.utc).isoformat(),
        "total_licenses": total_licenses,
        "customer_count": len(customers),
        "customers": customers,
        "parent_groups": parent_groups,
    }

    artifact_path = _write_portfolio_json(
        working_dir, customers, total_licenses, parent_groups,
    )
    _write_history_snapshot(working_dir, portfolio_data)

    change_lines = _detect_changes(working_dir, customers)

    sorted_customers = sorted(
        customers.items(),
        key=lambda x: x[1].get("health", {}).get("composite", 999),
    )

    distinct_packages = set()
    for cust in customers.values():
        distinct_packages.update(cust.get("packages", {}).keys())

    tier_counts: dict[str, int] = {}
    for cust in customers.values():
        t = cust.get("health", {}).get("tier", "UNKNOWN")
        tier_counts[t] = tier_counts.get(t, 0) + 1
    tier_summary = ", ".join(f"{v} {k}" for k, v in sorted(tier_counts.items()))

    lines = [
        "STATUS: READY\n",
        f"## Portfolio Snapshot\n",
        f"- **Customers:** {len(customers)}",
        f"- **Total active production licenses:** {total_licenses}",
        f"- **Distinct packages:** {len(distinct_packages)}",
        f"- **Corporate groups:** {len(parent_groups)}",
        f"- **Health distribution:** {tier_summary}",
        f"- **Data:** `{artifact_path}`\n",
    ]

    if change_lines:
        lines.append("### Changes Since Last Snapshot\n")
        lines.extend(change_lines)
        lines.append("")

    lines.append("### All Customers (ranked by health score, worst first)\n")
    lines.append(
        "| Customer | Score | Tier | Pkgs | Region | "
        "Max Gap | Upcoming Exp | Opportunities |"
    )
    lines.append(
        "|----------|-------|------|------|--------|"
        "---------|-------------|---------------|"
    )
    for name, cust in sorted_customers:
        health = cust.get("health", {})
        score = health.get("composite", "?")
        tier = health.get("tier", "?")
        upcoming = cust.get("upcoming_expiry_days")
        upcoming_str = str(int(upcoming)) if upcoming is not None else "—"
        gap = cust.get("max_releases_behind")
        gap_str = str(gap) if gap is not None else "?"
        region = cust.get("region") or "—"
        opps = ", ".join(health.get("opportunities", [])) or "—"

        lines.append(
            f"| {name} | {score} | {tier} | {cust['package_count']} | "
            f"{region} | {gap_str} | {upcoming_str} | {opps} |"
        )

    if parent_groups:
        lines.append("\n### Corporate Groups\n")
        lines.append(
            "| Parent Company | Subsidiaries | Total Licenses | "
            "Max Gap | Upcoming Exp | Overdue |"
        )
        lines.append(
            "|----------------|--------------|----------------|"
            "---------|-------------|---------|"
        )
        for pname, grp in sorted(parent_groups.items()):
            sub_count = len(grp["subsidiaries"])
            gap = grp.get("max_releases_behind")
            gap_str = str(gap) if gap is not None else "?"
            upcoming = grp.get("upcoming_expiry_days")
            upcoming_str = str(int(upcoming)) if upcoming is not None else "—"
            nearest = grp.get("nearest_expiry_days")
            overdue_str = str(int(nearest)) if nearest is not None and nearest < 0 else "—"
            lines.append(
                f"| {pname} | {sub_count} | {grp['total_licenses']} | "
                f"{gap_str} | {upcoming_str} | {overdue_str} |"
            )

    lines.append(
        f"\n\n**Scoring rules:** `{rules_path}`\n"
        "Health scores above are pre-computed using the scoring rules below. "
        "Present the ranked portfolio view using these scores. "
        "For what-if scenarios, re-apply the modified rule to the raw "
        "per-package data in portfolio-data.json to adjust the affected "
        "dimension only. Never generate external scripts for scoring. "
        "The user can edit the scoring rules file to change thresholds. "
        "Offer to:\n"
        "- Dive deeper into any specific customer (use find_customer)\n"
        "- Save findings as a shareable report (use save_report)\n"
        "- Generate a visual report (use generate_visual_report)\n"
    )

    lines.append("\n---\n\n<SCORING_RULES>\n")
    lines.append(scoring_rules)
    lines.append("</SCORING_RULES>")

    return "\n".join(lines)


def _seat_flags(packages: dict[str, dict[str, Any]]) -> str:
    """Generate compact seat utilization flags for a customer."""
    flags = []
    for pkg_name, pkg in packages.items():
        seats = pkg.get("seats", {})
        if seats.get("type") == "numeric":
            pct = seats.get("pct", 0)
            if pct >= 90:
                flags.append(f"{pkg_name[:12]}:{pct}%!")
            elif pct < 30 and seats.get("licensed", 0) > 0:
                flags.append(f"{pkg_name[:12]}:{pct}%?")
    if not flags:
        return "-"
    return "; ".join(flags[:3])
