"""
Neptune v2 REST API – educational fetcher + endpoint runner.

This module implements a pragmatic, well-instrumented read path over Neptune.ai v2’s
public REST APIs. It is intended to be both:
  (a) educational documentation for how Neptune’s v2 APIs actually behave, and
  (b) a reliable reference implementation for a future TypeScript client.

Architecture: Backend API vs Leaderboard API
--------------------------------------------
Neptune v2 exposes multiple API “surfaces”. For core data fetching we rely on:

1) Backend API (/api/backend/v1/*)
   - Authentication token exchange:
       GET /api/backend/v1/authorization/oauth-token
       Header: X-Neptune-Api-Token: <base64 token>
       Response: { "accessToken": "<...>" }
   - Project metadata:
       GET /api/backend/v1/projects/get?projectIdentifier=workspace/project

2) Leaderboard API (/api/leaderboard/v1/*)
   - The same API surface the Neptune UI uses for the run “table/leaderboard” view.
   - Primary endpoint for run snapshots (values + metadata):
       POST /api/leaderboard/v1/leaderboard/entries/search
   - Attribute definitions:
       POST /api/leaderboard/v1/leaderboard/attributes/definitions/query
   - Series point retrieval:
       GET /api/leaderboard/v1/attributes/series/float
       GET /api/leaderboard/v1/attributes/series/string
   - Batch export for FloatSeries:
       POST /api/leaderboard/v1/attributes/floatSeries/csv/multiple
       -> returns DownloadPrepareRequest(downloadUrl, id)
       -> follow downloadUrl (typically returns ZIP containing CSV files)

Identifiers
-----------
The stable identifier used throughout the API (including all series endpoints) is:
  - experimentId (UUID)

Human-readable naming should be derived from attributes, typically:
  - sys/name

Attributes and data types
-------------------------
Leaderboard returns an “entry snapshot” containing a list of typed attributes. Each
attribute object has exactly one "*Properties" payload, e.g.:
  - stringProperties.value, intProperties.value, floatProperties.value, boolProperties.value
  - floatSeriesProperties (rich series metadata: last, lastStep, min, max, average, variance, config)
  - stringSeriesProperties (last, lastStep)
  - fileProperties (name, ext, size), gitRefProperties, complexProperties, etc.

Goodseed normalizes these into stable internal models:
  - NormalizedAttribute: (name, kind, value, meta)
  - NeptuneRun: run-level structured representation (experiment_id, sys fields, scalars, series metadata)
  - NeptuneSeriesPoint: typed time-series point (step, value, timestampMillis)

Time series: metadata vs points
-------------------------------
A key performance concept is that the Leaderboard snapshot already contains summary
statistics for float series (min/max/average/variance/last/lastStep). Full curve
downloads are only required when you need all points.

Batch export behavior
---------------------
The “multiple CSV” endpoint does not return CSV inline. It returns a downloadUrl,
and the downloaded payload is typically a ZIP archive even for small exports.

Operational notes (real-world quirks)
-------------------------------------
- Invalid NQL query syntax can yield 400 “Syntax error”.
- In some cases, combining invalid query + sorting triggers a 500 instead of 400
  (server-side bug). The CLI prints request bodies and response text to aid debugging.
- Calling the wrong series endpoint for an attribute type (e.g. string endpoint for
  a float series attribute) may yield a 500 instead of a clean 4xx. Prefer dispatch
  based on attribute metadata.

Scope (current)
---------------
Implemented:
  - Auth + project info (backend)
  - Run listing and run snapshot parsing (leaderboard entries/search)
  - Attribute definitions query
  - FloatSeries and StringSeries point fetching (paged)
  - Batch FloatSeries export (downloadUrl -> bytes/ZIP -> extraction)
  - Swagger helpers for introspecting backend.swagger.json and leaderboard.swagger.json

Intentionally not implemented yet:
  - Image series fetching
  - Trash/restore/delete endpoints
  - Clipboard/selection endpoints

Typical usage
-------------
  export NEPTUNE_API_TOKEN="..."          # base64-encoded Neptune token
  export NEPTUNE_PROJECT="workspace/proj"

  # Project metadata
  python src/goodseed/neptune_fetch_new.py project-info

  # Search runs (prints experimentId, supports NQL filtering)
  python src/goodseed/neptune_fetch_new.py search-runs --limit 20
  python src/goodseed/neptune_fetch_new.py search-runs --query "my-experiment"
  python src/goodseed/neptune_fetch_new.py search-runs --trashed

  # Describe a run (attributes + series metadata; optional point stats)
  python src/goodseed/neptune_fetch_new.py describe-run --run <experimentId>
  python src/goodseed/neptune_fetch_new.py describe-run --run <experimentId> --series-stats

  # Fetch series points explicitly
  python src/goodseed/neptune_fetch_new.py fetch-float-series --experiment-id <uuid> --attr metrics/loss
  python src/goodseed/neptune_fetch_new.py fetch-string-series --experiment-id <uuid> --attr monitoring/<id>/stderr

  # Batch export float series (ZIP of CSVs)
  python src/goodseed/neptune_fetch_new.py batch-floatseries-csv \
    --runs <uuid> --attrs metrics/loss metrics/entropy --zip-path export.zip --out-dir export_dir

  # Swagger introspection
  python src/goodseed/neptune_fetch_new.py swagger-find --leaderboard --pattern entries/search
  python src/goodseed/neptune_fetch_new.py swagger-op --leaderboard --opid searchLeaderboardEntries
"""

from __future__ import annotations

import argparse
import base64
import dataclasses
import json
import os
import sys
import textwrap
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any, Dict, Iterable, List, Optional, Tuple, Union
from pathlib import Path
from collections import Counter
import zipfile

import httpx


Json = Union[Dict[str, Any], List[Any]]

def resolve_spec_path(filename: str) -> str:
    """
    Resolve swagger JSON path robustly:
    - first try CWD
    - then try repo root (walk up from this file)
    """
    cwd = Path.cwd() / filename
    if cwd.exists():
        return str(cwd)

    here = Path(__file__).resolve()
    # Walk up a few levels looking for the file
    for parent in [here.parent, *here.parents]:
        candidate = parent / filename
        if candidate.exists():
            return str(candidate)
    return filename  # fall back; will raise FileNotFoundError


# -----------------------------
# Token parsing and auth
# -----------------------------

def parse_api_token(token: str) -> Tuple[str, str]:
    """
    Neptune tokens are base64-encoded JSON objects containing:
      - api_address: base URL (e.g. https://app.neptune.ai or your self-hosted domain)
      - api_key: UUID string

    Returns (api_url, api_key).
    """
    decoded = base64.b64decode(token).decode("utf-8")
    data = json.loads(decoded)
    api_url = data.get("api_address") or data.get("api_url")
    api_key = data.get("api_key")
    if not api_url or not api_key:
        raise ValueError("Token missing api_address/api_key")
    return api_url.rstrip("/"), api_key


def iso(dt: Optional[datetime]) -> str:
    if dt is None:
        return "(none)"
    return dt.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")


def _try_parse_datetime_ms_or_iso(x: Any) -> Optional[datetime]:
    # Neptune sometimes returns ms timestamps or ISO strings depending on field.
    try:
        if isinstance(x, (int, float)):
            return datetime.fromtimestamp(float(x) / 1000.0, tz=timezone.utc)
        if isinstance(x, str):
            s = x.replace("Z", "+00:00")
            return datetime.fromisoformat(s)
    except Exception:
        return None
    return None


# -----------------------------
# Swagger/OpenAPI helpers
# -----------------------------

@dataclass(frozen=True)
class SwaggerOperation:
    method: str
    path: str
    operation_id: str
    parameters: List[Dict[str, Any]]
    responses: Dict[str, Any]


class SwaggerSpec:
    """
    Lightweight swagger.json loader + operation lookup by operationId.
    """
    def __init__(self, spec_path: str):
        with open(spec_path, "r", encoding="utf-8") as f:
            self.spec = json.load(f)

        self.paths: Dict[str, Any] = self.spec.get("paths", {})
        self.defs: Dict[str, Any] = self.spec.get("definitions", {})

    def find_operation(self, operation_id: str) -> SwaggerOperation:
        for path, methods in self.paths.items():
            if not isinstance(methods, dict):
                continue
            for m, op in methods.items():
                if not isinstance(op, dict):
                    continue
                if op.get("operationId") == operation_id:
                    return SwaggerOperation(
                        method=m.upper(),
                        path=path,
                        operation_id=operation_id,
                        parameters=op.get("parameters", []) or [],
                        responses=op.get("responses", {}) or {},
                    )
        raise KeyError(f"operationId not found: {operation_id}")

    def describe_operation(self, operation_id: str) -> str:
        op = self.find_operation(operation_id)

        lines: List[str] = []
        lines.append(f"{op.method} {op.path}")
        lines.append(f"operationId: {op.operation_id}")

        # Parameters
        lines.append("parameters:")
        if op.parameters:
            for p in op.parameters:
                name = p.get("name")
                where = p.get("in")
                required = p.get("required", False)
                if "schema" in p:
                    typ = self._schema_str(p["schema"])
                else:
                    typ = p.get("type")
                lines.append(f"  - {name} ({where}) required={required} type={typ}")
        else:
            lines.append("  (none)")

        # Request body DTOs (expanded)
        body_models: List[str] = []
        for p in op.parameters or []:
            if p.get("in") == "body" and isinstance(p.get("schema"), dict):
                body_models.extend(self._root_refs_from_schema(p["schema"]))
        body_models = sorted(set(body_models))

        if body_models:
            lines.append("request body models and properties:")
            for model in body_models:
                props_map = self.walk_model_properties(model)
                # Print root first, then nested
                root_props = props_map.get(model, [])
                lines.append(f"  {model}: {root_props}")
                for m in sorted(props_map.keys()):
                    if m == model:
                        continue
                    lines.append(f"    {m}: {props_map[m]}")

        # Responses
        lines.append("responses:")
        for code, r in op.responses.items():
            schema = r.get("schema")
            produces = r.get("produces")  # often absent in swagger v2
            lines.append(f"  - {code}: schema={schema} produces={produces}")

        return "\n".join(lines)

    @classmethod
    def _schema_str(cls, schema: dict) -> str:
        if not isinstance(schema, dict):
            return str(schema)
        if "$ref" in schema:
            return "$ref:" + schema["$ref"]
        t = schema.get("type")
        if t == "array":
            items = schema.get("items", {})
            return f"array[{cls._schema_str(items)}]"
        if t == "object":
            # Swagger v2 inline objects usually have properties; keep it brief
            props = schema.get("properties")
            if isinstance(props, dict):
                return "object{" + ",".join(list(props.keys())[:10]) + ("..." if len(props) > 10 else "") + "}"
            return "object"
        return t or "schema"

    @classmethod
    def _root_refs_from_schema(cls, schema: dict) -> List[str]:
        if not isinstance(schema, dict):
            return []
        if "$ref" in schema and schema["$ref"].startswith("#/definitions/"):
            return [schema["$ref"].split("/")[-1]]
        if schema.get("type") == "array":
            return cls._root_refs_from_schema(schema.get("items", {}))
        return []

    def walk_model_properties(self, model_name: str) -> Dict[str, List[str]]:
        """
        Returns a mapping: model_name -> list(properties)
        for model_name and any nested $ref models reachable from it.
        """
        if model_name not in self.defs:
            raise KeyError(f"Definition not found: {model_name}")

        seen: set[str] = set()
        out: Dict[str, List[str]] = {}
        queue: List[str] = [model_name]

        def _enqueue_ref(ref: str):
            if ref.startswith("#/definitions/"):
                nm = ref.split("/")[-1]
                if nm not in seen:
                    queue.append(nm)

        while queue:
            nm = queue.pop(0)
            if nm in seen:
                continue
            seen.add(nm)

            sch = self.defs.get(nm, {})
            props = sch.get("properties", {}) if isinstance(sch, dict) else {}
            out[nm] = sorted(list(props.keys()))

            for psch in props.values():
                if isinstance(psch, dict):
                    if "$ref" in psch:
                        _enqueue_ref(psch["$ref"])
                    if psch.get("type") == "array" and isinstance(psch.get("items"), dict):
                        it = psch["items"]
                        if "$ref" in it:
                            _enqueue_ref(it["$ref"])
        return out


# -----------------------------
# Core data models
# -----------------------------

@dataclass
class NeptuneRun:
    """Structured representation of a Neptune run with all its metadata."""
    experiment_id: str
    name: Optional[str] = None
    created_time: Optional[datetime] = None
    state: Optional[str] = None  # running, idle
    owner: Optional[str] = None

    # All scalar attributes, normalized
    scalars: Dict[str, Any] = field(default_factory=dict)

    # Series metadata from leaderboard (no point fetching needed)
    float_series: Dict[str, Dict[str, Any]] = field(default_factory=dict)   # meta includes last/min/max/avg/...
    string_series: Dict[str, Dict[str, Any]] = field(default_factory=dict)  # meta includes last/lastStep

    # Optional: keep raw attributes for debugging
    raw_entry: Dict[str, Any] = field(default_factory=dict)


@dataclass
class NeptuneSeriesPoint:
    """A single point from a Neptune time series."""
    path: str
    step: float
    value: Any  # float or str
    timestamp_ms: Optional[int] = None

    @classmethod
    def from_raw(cls, path: str, raw: Dict[str, Any]) -> "NeptuneSeriesPoint":
        """Create from raw API response point."""
        return cls(
            path=path,
            step=float(raw.get("step", 0)),
            value=raw.get("value"),
            timestamp_ms=raw.get("timestampMillis"),
        )


def parse_run_entry(entry: Dict[str, Any]) -> NeptuneRun:
    """
    Parse a raw Neptune run entry into a structured NeptuneRun object.

    Extracts:
    - Core identifiers (experiment_id, name, state, owner, created_time)
    - Scalar attributes into scalars dict
    - Float series metadata into float_series dict
    - String series metadata into string_series dict
    """
    attrs_raw = entry.get("attributes", []) or []
    attrs = [normalize_attribute(a) for a in attrs_raw]

    # Extract sys attributes
    name: Optional[str] = None
    state: Optional[str] = None
    owner: Optional[str] = None
    created_time: Optional[datetime] = None

    scalars: Dict[str, Any] = {}
    float_series: Dict[str, Dict[str, Any]] = {}
    string_series: Dict[str, Dict[str, Any]] = {}

    for a in attrs:
        if a.name == "sys/name" and a.value is not None:
            name = str(a.value)
        elif a.name == "sys/state" and a.value is not None:
            state = str(a.value)
        elif a.name == "sys/owner" and a.value is not None:
            owner = str(a.value)
        elif a.name == "sys/creation_time" and a.value is not None:
            created_time = _try_parse_datetime_ms_or_iso(a.value)

        # Categorize by kind
        if a.kind == "floatSeries":
            float_series[a.name] = a.meta
        elif a.kind == "stringSeries":
            string_series[a.name] = a.meta
        elif a.value is not None:
            # Scalar attribute
            scalars[a.name] = a.value

    return NeptuneRun(
        experiment_id=entry.get("experimentId", ""),
        name=name,
        created_time=created_time,
        state=state,
        owner=owner,
        scalars=scalars,
        float_series=float_series,
        string_series=string_series,
        raw_entry=entry,
    )


# -----------------------------
# Normalized attribute model
# -----------------------------

@dataclass
class NormalizedAttribute:
    name: str
    kind: str  # string|int|float|bool|datetime|floatSeries|stringSeries|...
    value: Any = None          # scalar value
    meta: Dict[str, Any] = field(default_factory=dict)  # series/file metadata
    raw: Optional[Dict[str, Any]] = None


def normalize_attribute(attr: Dict[str, Any]) -> NormalizedAttribute:
    """
    Normalize Neptune attribute using swagger-derived DTO fields.

    Extracts appropriate fields based on attribute type:
    - Scalars: value field
    - floatSeries: summary fields (last, lastStep, min, max, average, variance, config)
    - stringSeries: last fields (last, lastStep)
    - file: size/name/ext
    - fileSet: size
    - complex: content/typeDefinition
    - gitRef: all properties
    """
    name = attr.get("name", "")
    typ = attr.get("type")  # AttributeTypeDTO enum-ish; may be present

    # Find the one *Properties key
    prop_key = next((k for k in attr.keys() if k.endswith("Properties")), None)
    if not prop_key:
        return NormalizedAttribute(name=name, kind=typ or "unknown", raw=attr)

    props = attr.get(prop_key) or {}
    kind = prop_key[:-len("Properties")]  # e.g. floatSeries

    # Scalars: string, int, float, bool, datetime, experimentState
    if kind in {"string", "int", "float", "bool", "datetime", "experimentState"}:
        return NormalizedAttribute(
            name=name,
            kind=kind,
            value=props.get("value"),
            meta={},
            raw=attr,
        )

    # Float series: keep summary fields (FloatSeriesAttributeDTO)
    if kind == "floatSeries":
        meta = {
            k: props.get(k)
            for k in ("last", "lastStep", "min", "max", "average", "variance", "config")
            if props.get(k) is not None
        }
        return NormalizedAttribute(name=name, kind=kind, value=None, meta=meta, raw=attr)

    # String series: keep last fields (StringSeriesAttributeDTO)
    if kind == "stringSeries":
        meta = {
            k: props.get(k)
            for k in ("last", "lastStep")
            if props.get(k) is not None
        }
        return NormalizedAttribute(name=name, kind=kind, value=None, meta=meta, raw=attr)

    # Image series: keep lastStep if present
    if kind == "imageSeries":
        meta = {
            k: props.get(k)
            for k in ("lastStep",)
            if props.get(k) is not None
        }
        return NormalizedAttribute(name=name, kind=kind, value=None, meta=meta, raw=attr)

    # File: keep name, ext, size (FileAttributeDTO)
    if kind == "file":
        meta = {
            k: props.get(k)
            for k in ("name", "ext", "size")
            if props.get(k) is not None
        }
        return NormalizedAttribute(name=name, kind=kind, value=None, meta=meta, raw=attr)

    # FileSet: keep size (FileSetAttributeDTO)
    if kind == "fileSet":
        meta = {
            k: props.get(k)
            for k in ("size",)
            if props.get(k) is not None
        }
        return NormalizedAttribute(name=name, kind=kind, value=None, meta=meta, raw=attr)

    # Git ref: keep all properties (GitInfoDTO)
    if kind == "gitRef":
        return NormalizedAttribute(name=name, kind=kind, value=None, meta=dict(props), raw=attr)

    # Complex: keep content and typeDefinition (optionally truncated)
    if kind == "complex":
        meta = {
            k: props.get(k)
            for k in ("content", "typeDefinition")
            if props.get(k) is not None
        }
        return NormalizedAttribute(name=name, kind=kind, value=None, meta=meta, raw=attr)

    # String set: keep values
    if kind == "stringSet":
        return NormalizedAttribute(
            name=name,
            kind=kind,
            value=props.get("values"),
            meta={},
            raw=attr,
        )

    # Notebook ref, artifact, etc: keep value if present
    if kind in {"notebookRef", "artifact"}:
        return NormalizedAttribute(
            name=name,
            kind=kind,
            value=props.get("value"),
            meta={k: v for k, v in props.items() if k != "value"},
            raw=attr,
        )

    # Everything else: keep meta verbatim
    return NormalizedAttribute(
        name=name,
        kind=kind,
        value=props.get("value"),
        meta=dict(props),
        raw=attr,
    )


# -----------------------------
# Neptune REST client (backend + leaderboard)
# -----------------------------

def _bucket_attribute(attr_name: str) -> str:
    """
    Categorize attribute by name prefix for robust config inference.

    Returns:
        - "sys": system attributes (sys/*)
        - "series_or_metrics": metrics and monitoring data
        - "config": configuration and parameters (default bucket)
    """
    if attr_name.startswith("sys/"):
        return "sys"
    if attr_name.startswith("metrics/") or attr_name.startswith("monitoring/"):
        return "series_or_metrics"
    if attr_name.startswith("config/") or attr_name.startswith("parameters/"):
        return "config"
    # Pragmatic default: treat everything else as config-ish
    # This handles Hydra-like attribute names and custom user attributes
    return "config"


@dataclass
class NeptuneRunSummary:
    experiment_id: Optional[str]
    attrs: List[NormalizedAttribute]
    sys_attrs: Dict[str, Any]
    config_attrs: Dict[str, Any]
    other_attrs: Dict[str, Any]


class NeptuneAPI:
    """
    Minimal, explicit, educational Neptune REST client.

    - Uses backend oauth token exchange.
    - Uses leaderboard APIs to fetch run entries and series values.
    """

    def __init__(self, api_token: str, project: str, timeout_s: float = 30.0):
        self.raw_token = api_token
        self.project = project
        self.base_url, self.api_key = parse_api_token(api_token)

        if "/" not in project:
            raise ValueError("project must be 'workspace/project-name'")

        self.timeout_s = timeout_s
        self.access_token = self._exchange_token()

        self.client = httpx.Client(
            base_url=self.base_url,
            headers={
                "Authorization": f"Bearer {self.access_token}",
                "Content-Type": "application/json",
            },
            timeout=self.timeout_s,
        )

        self._project_info: Optional[Dict[str, Any]] = None

    def close(self):
        self.client.close()

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc, tb):
        self.close()

    def _exchange_token(self) -> str:
        with httpx.Client(
            base_url=self.base_url,
            headers={
                "X-Neptune-Api-Token": self.raw_token,
                "Content-Type": "application/json",
            },
            timeout=self.timeout_s,
        ) as c:
            r = c.get("/api/backend/v1/authorization/oauth-token")
            r.raise_for_status()
            j = r.json()
            # Your deployment confirmed accessToken
            return j["accessToken"]

    # -------- Backend --------

    def get_project_info(self) -> Dict[str, Any]:
        if self._project_info is not None:
            return self._project_info
        r = self.client.get("/api/backend/v1/projects/get", params={"projectIdentifier": self.project})
        r.raise_for_status()
        self._project_info = r.json()
        return self._project_info

    def list_organizations(self) -> List[Dict[str, Any]]:
        """
        GET /api/backend/v1/myOrganizations

        Returns list of organizations the user belongs to.
        Each entry has: id, name, organizationType, userRole, etc.
        """
        r = self.client.get("/api/backend/v1/myOrganizations")
        r.raise_for_status()
        return r.json()

    def list_projects_for_org(self, organization: str) -> Dict[str, Any]:
        """
        GET /api/backend/v1/projects/listForOrganization

        Args:
            organization: Organization name (e.g. "gqsoqa")

        Returns dict with:
            entries: list of project dicts (id, name, organizationName, lastActivity, ...)
            matchingItemCount: total matching projects
            totalItemCount: total projects in org

        Note: sortBy/sortDirection are array-typed per swagger; omitted to use defaults.
        """
        r = self.client.get(
            "/api/backend/v1/projects/listForOrganization",
            params={"organizationIdentifier": organization},
        )
        r.raise_for_status()
        return r.json()

    # -------- Leaderboard: runs and attributes --------

    def trash_runs(self, experiment_ids: List[str]) -> Dict[str, Any]:
        """
        POST /api/leaderboard/v1/experiments/trash
        Move experiments to trash.

        Args:
            experiment_ids: List of experimentId UUIDs to trash.

        Returns:
            Response JSON with updatedExperimentIdentifiers and errors.
        """
        r = self.client.post(
            "/api/leaderboard/v1/experiments/trash",
            params={"projectIdentifier": self.project},
            json=experiment_ids,
        )
        r.raise_for_status()
        return r.json()

    def entries_search_runs(self, limit: int = 100, offset: int = 0) -> Dict[str, Any]:
        """
        POST /api/leaderboard/v1/leaderboard/entries/search
        Returns a payload that includes:
          - entries: list of runs with attributes
          - pagination metadata (deployment-dependent)
        """
        body = {"pagination": {"limit": limit, "offset": offset}}
        r = self.client.post(
            "/api/leaderboard/v1/leaderboard/entries/search",
            params={"projectIdentifier": self.project, "type": ["run"]},
            json=body,
        )
        r.raise_for_status()
        return r.json()

    def entries_search(
        self,
        *,
        limit: int = 100,
        offset: int = 0,
        nql: str = "",
        truncate_string_to: int = 0,
        types: Optional[List[str]] = None,
    ) -> Dict[str, Any]:
        """
        POST /api/leaderboard/v1/leaderboard/entries/search with full options.

        Implements searchLeaderboardEntries with NQL filtering, sorting, and string truncation.

        Args:
            limit: Maximum number of entries to return
            offset: Pagination offset
            nql: Neptune Query Language filter string (e.g., '`sys/state`:running')
            truncate_string_to: Truncate string attribute values to this length (0 = no truncation)
            types: Entry types to search (default: ["run"])

        Returns:
            Response JSON with entries and pagination metadata.

        Note:
            sortBy schema is QueryLeaderboardParamsFieldDTO with fields:
            - name: attribute name
            - type: "ATTRIBUTE" for regular attributes
            - aggregationMode: optional aggregation mode for series
        """
        body: Dict[str, Any] = {
            "pagination": {"limit": limit, "offset": offset},
        }

        if nql:
            body["query"] = {"query": nql}

        if truncate_string_to > 0:
            body["truncateStringTo"] = truncate_string_to

        r = self.client.post(
            "/api/leaderboard/v1/leaderboard/entries/search",
            params={"projectIdentifier": self.project, "type": (types or ["run"])},
            json=body,
        )
        try:
            r.raise_for_status()
        except httpx.HTTPStatusError as e:
            print("status:", r.status_code, file=sys.stderr)
            print("response text:", r.text[:5000], file=sys.stderr)  # may be empty on 500
            raise

        r.raise_for_status()
        return r.json()

    def iter_all_runs(self, page_limit: int = 200) -> Iterable[Dict[str, Any]]:
        offset = 0
        while True:
            data = self.entries_search_runs(limit=page_limit, offset=offset)
            entries = data.get("entries", []) or []
            for e in entries:
                yield e
            if len(entries) < page_limit:
                break
            offset += len(entries)

    def query_attribute_definitions(self, experiment_identifiers: List[str]) -> Dict[str, Any]:
        """
        POST /api/leaderboard/v1/leaderboard/attributes/definitions/query
        This returns attribute definition entries + nextPage token.
        """
        body = {"experimentIdentifiers": experiment_identifiers}
        r = self.client.post(
            "/api/leaderboard/v1/leaderboard/attributes/definitions/query",
            params={"projectIdentifier": self.project},
            json=body,
        )
        r.raise_for_status()
        return r.json()

    def get_run_by_identifier(self, ident: str) -> Dict[str, Any]:
        """Get raw run entry by experimentId."""
        for e in self.iter_all_runs(page_limit=200):
            # Common candidates
            if e.get("experimentId") == ident:
                return e
            if e.get("id") == ident:
                return e
            if e.get("sysId") == ident:
                return e
        raise KeyError(f"Run not found by identifier: {ident}")

    def get_run(self, ident: str) -> NeptuneRun:
        """Get a structured NeptuneRun by experimentId."""
        entry = self.get_run_by_identifier(ident)
        return parse_run_entry(entry)

    def list_runs(self, limit: int = 100, offset: int = 0) -> List[NeptuneRun]:
        """List runs as structured NeptuneRun objects."""
        data = self.entries_search_runs(limit=limit, offset=offset)
        entries = data.get("entries", []) or []
        return [parse_run_entry(e) for e in entries]

    def summarize_run_entry(self, entry: Dict[str, Any]) -> NeptuneRunSummary:
        """
        Summarize a run entry with robust config inference.

        Uses _bucket_attribute to categorize attributes into:
        - sys_attrs: system attributes (sys/*)
        - config_attrs: config/parameters and other config-ish attributes
        - other_attrs: metrics and monitoring series data
        """
        attrs_raw = entry.get("attributes", []) or []
        attrs = [normalize_attribute(a) for a in attrs_raw]

        sys_attrs: Dict[str, Any] = {}
        config_attrs: Dict[str, Any] = {}
        other_attrs: Dict[str, Any] = {}

        for a in attrs:
            bucket = _bucket_attribute(a.name)
            # Use value if available, otherwise use meta (for series types)
            attr_value = a.value if a.value is not None else a.meta

            if bucket == "sys":
                sys_attrs[a.name] = attr_value
            elif bucket == "config":
                # Strip prefix for cleaner keys
                key = a.name
                for prefix in ("config/", "parameters/"):
                    if key.startswith(prefix):
                        key = key[len(prefix):]
                        break
                config_attrs[key] = attr_value
            else:  # series_or_metrics
                other_attrs[a.name] = attr_value

        return NeptuneRunSummary(
            experiment_id=entry.get("experimentId"),
            attrs=attrs,
            sys_attrs=sys_attrs,
            config_attrs=config_attrs,
            other_attrs=other_attrs,
        )

    # -------- Leaderboard: series values --------

    def get_float_series_via_channels(
        self,
        *,
        experiment_id: str,
        attribute: str,
        point_count: int = 10000,
        holder_type: str = "experiment",
    ) -> List[Dict[str, Any]]:
        """
        Fetch float series points via the channels/view endpoint (POST).

        This is the preferred endpoint for fetching series data - it's simpler
        and supports downsampling via pointCount.

        POST /api/leaderboard/v1/channels/view?pointCount=N&projectIdentifier=...
        Body: [{"attributePath": "...", "holderIdentifier": "...", "holderType": "..."}]

        Response structure:
            {
                "elements": [{
                    "attributePath": "...",
                    "holderIdentifier": "...",
                    "values": [
                        {"x": step, "y": value, "timestamp": "...", ...},
                        ...
                    ]
                }]
            }

        Returns list of points as dicts with keys: step, value, timestampMillis.
        """
        body = [
            {
                "attributePath": attribute,
                "holderIdentifier": experiment_id,
                "holderType": holder_type,
            }
        ]
        r = self.client.post(
            "/api/leaderboard/v1/channels/view",
            params={
                "pointCount": point_count,
                "projectIdentifier": self.project,
            },
            json=body,
        )
        r.raise_for_status()
        data = r.json()
        return _parse_channels_view_response(data)

    def get_float_series_via_channels_batch(
        self,
        *,
        requests: List[Dict[str, str]],
        point_count: int = 10000,
    ) -> List[List[Dict[str, Any]]]:
        """
        Fetch multiple float series in a single request via channels/view.

        Args:
            requests: List of dicts with keys:
                - attributePath: attribute name (e.g., "metrics/loss")
                - holderIdentifier: experiment UUID
                - holderType: typically "experiment"
            point_count: Maximum points per series (downsampling)

        Returns:
            List of point lists, one per request item.
        """
        r = self.client.post(
            "/api/leaderboard/v1/channels/view",
            params={
                "pointCount": point_count,
                "projectIdentifier": self.project,
            },
            json=requests,
        )
        r.raise_for_status()
        data = r.json()

        # Response: {"elements": [{...values...}, {...values...}, ...]}
        # Each element corresponds to a request item
        elements = data.get("elements", [])
        results = []
        for element in elements:
            raw_values = element.get("values", [])
            points = []
            for v in raw_values:
                point: Dict[str, Any] = {
                    "step": v.get("x"),
                    "value": v.get("y"),
                }
                ts = v.get("timestamp")
                if ts:
                    dt = _try_parse_datetime_ms_or_iso(ts)
                    if dt:
                        point["timestampMillis"] = int(dt.timestamp() * 1000)
                points.append(point)
            results.append(points)
        return results

    def fetch_float_series_points(
        self,
        *,
        experiment_id: str,
        attribute: str,
        point_count: int = 10000,
        holder_type: str = "experiment",
    ) -> List[NeptuneSeriesPoint]:
        """
        Fetch float series points as typed NeptuneSeriesPoint objects.

        Uses the channels/view endpoint (preferred over the legacy GET endpoint).
        """
        raw_points = self.get_float_series_via_channels(
            experiment_id=experiment_id,
            attribute=attribute,
            point_count=point_count,
            holder_type=holder_type,
        )
        return [NeptuneSeriesPoint.from_raw(attribute, p) for p in raw_points]

    # -------- Leaderboard: series values (legacy GET endpoint) --------
    # Kept for documentation - the channels/view POST endpoint above is preferred.

    def get_float_series_values_page(
        self,
        *,
        experiment_id: str,
        attribute: str,
        limit: int = 10000,
        offset: int = 0,
        holder_type: str = "experiment",
        holder_identifier: Optional[str] = None,
    ) -> Dict[str, Any]:
        params: Dict[str, Any] = {
            "attribute": attribute,
            "experimentId": experiment_id,
            "holderType": holder_type,
            "limit": limit,
            "offset": offset,
        }
        if holder_identifier is not None:
            params["holderIdentifier"] = holder_identifier
        r = self.client.get("/api/leaderboard/v1/attributes/series/float", params=params)
        r.raise_for_status()
        return r.json()

    def iter_float_series_points_raw(
        self,
        *,
        experiment_id: str,
        attribute: str,
        page_limit: int = 10000,
        holder_type: str = "experiment",
        holder_identifier: Optional[str] = None,
        max_points: Optional[int] = None,
    ) -> Iterable[Dict[str, Any]]:
        """Iterate raw float series points as dicts."""
        offset = 0
        yielded = 0
        while True:
            page = self.get_float_series_values_page(
                experiment_id=experiment_id,
                attribute=attribute,
                limit=page_limit,
                offset=offset,
                holder_type=holder_type,
                holder_identifier=holder_identifier,
            )
            points = extract_points_list(page)
            for p in points:
                yield p
                yielded += 1
                if max_points is not None and yielded >= max_points:
                    return
            if len(points) < page_limit:
                break
            offset += len(points)

    def iter_float_series_points(
        self,
        *,
        experiment_id: str,
        attribute: str,
        page_limit: int = 10000,
        holder_type: str = "experiment",
        holder_identifier: Optional[str] = None,
        max_points: Optional[int] = None,
    ) -> Iterable[NeptuneSeriesPoint]:
        """Iterate float series points as typed NeptuneSeriesPoint objects."""
        for raw in self.iter_float_series_points_raw(
            experiment_id=experiment_id,
            attribute=attribute,
            page_limit=page_limit,
            holder_type=holder_type,
            holder_identifier=holder_identifier,
            max_points=max_points,
        ):
            yield NeptuneSeriesPoint.from_raw(attribute, raw)

    def get_string_series_values_page(
        self,
        *,
        experiment_id: str,
        attribute: str,
        limit: int = 10000,
        offset: int = 0,
        holder_type: str = "experiment",
        holder_identifier: Optional[str] = None,
    ) -> Dict[str, Any]:
        params: Dict[str, Any] = {
            "attribute": attribute,
            "experimentId": experiment_id,
            "holderType": holder_type,
            "limit": limit,
            "offset": offset,
        }
        if holder_identifier is not None:
            params["holderIdentifier"] = holder_identifier
        r = self.client.get("/api/leaderboard/v1/attributes/series/string", params=params)
        r.raise_for_status()
        return r.json()

    def iter_string_series_points_raw(
        self,
        *,
        experiment_id: str,
        attribute: str,
        page_limit: int = 10000,
        holder_type: str = "experiment",
        holder_identifier: Optional[str] = None,
        max_points: Optional[int] = None,
    ) -> Iterable[Dict[str, Any]]:
        """Iterate raw string series points as dicts."""
        offset = 0
        yielded = 0
        while True:
            page = self.get_string_series_values_page(
                experiment_id=experiment_id,
                attribute=attribute,
                limit=page_limit,
                offset=offset,
                holder_type=holder_type,
                holder_identifier=holder_identifier,
            )
            points = extract_points_list(page)
            for p in points:
                yield p
                yielded += 1
                if max_points is not None and yielded >= max_points:
                    return
            if len(points) < page_limit:
                break
            offset += len(points)

    def iter_string_series_points(
        self,
        *,
        experiment_id: str,
        attribute: str,
        page_limit: int = 10000,
        holder_type: str = "experiment",
        holder_identifier: Optional[str] = None,
        max_points: Optional[int] = None,
    ) -> Iterable[NeptuneSeriesPoint]:
        """Iterate string series points as typed NeptuneSeriesPoint objects."""
        for raw in self.iter_string_series_points_raw(
            experiment_id=experiment_id,
            attribute=attribute,
            page_limit=page_limit,
            holder_type=holder_type,
            holder_identifier=holder_identifier,
            max_points=max_points,
        ):
            yield NeptuneSeriesPoint.from_raw(attribute, raw)

    # -------- Leaderboard: batch float series to CSV --------

    def download_prepare_and_fetch(self, prepare: Dict[str, Any]) -> bytes:
        download_url = prepare.get("downloadUrl")
        if not download_url:
            raise ValueError(f"Missing downloadUrl in DownloadPrepareRequest keys={list(prepare.keys())}")
        print(f"Download URL: {download_url}")
        r = self.client.get(download_url)
        r.raise_for_status()
        return r.content

    def get_multiple_float_series_values_csv(
        self,
        *,
        run_ids: List[str],
        attributes: List[str],
        holder_type: str = "experiment",
    ) -> bytes:
        body = {
            "requests": [
                {
                    "holderType": holder_type,
                    "holderIdentifier": rid,
                    "attributes": attributes,
                }
                for rid in run_ids
            ]
        }
        r = self.client.post("/api/leaderboard/v1/attributes/floatSeries/csv/multiple", json=body)
        r.raise_for_status()
        prepare = r.json()
        return self.download_prepare_and_fetch(prepare)


    # -------- Generic call utility --------

    def call(self, method: str, path: str, *, query: Optional[Dict[str, Any]] = None, json_body: Optional[Any] = None) -> httpx.Response:
        r = self.client.request(method.upper(), path, params=query, json=json_body)
        r.raise_for_status()
        return r


# -----------------------------
# Payload normalization helpers
# -----------------------------

def _parse_channels_view_response(data: Dict[str, Any]) -> List[Dict[str, Any]]:
    """
    Parse the channels/view endpoint response into normalized points.

    Response structure:
        {
            "elements": [{
                "attributePath": "...",
                "values": [{"x": step, "y": value, "timestamp": "...", ...}, ...]
            }]
        }

    Returns list of points with keys: step, value, timestampMillis.
    """
    elements = data.get("elements", [])
    if not elements:
        return []

    # Take the first element (single attribute request)
    element = elements[0]
    raw_values = element.get("values", [])

    points = []
    for v in raw_values:
        # Map x -> step, y -> value
        point: Dict[str, Any] = {
            "step": v.get("x"),
            "value": v.get("y"),
        }
        # Convert ISO timestamp to milliseconds if present
        ts = v.get("timestamp")
        if ts:
            dt = _try_parse_datetime_ms_or_iso(ts)
            if dt:
                point["timestampMillis"] = int(dt.timestamp() * 1000)
        points.append(point)

    return points


def extract_points_list(payload: Any) -> List[Dict[str, Any]]:
    """
    Neptune series endpoints typically wrap points in {entries|values|items|points|elements},
    sometimes returning a direct list.

    This keeps your tooling robust while you learn exact schemas.
    """
    if isinstance(payload, list):
        return [p for p in payload if isinstance(p, dict)]
    if not isinstance(payload, dict):
        raise ValueError(f"Unexpected payload type: {type(payload)}")
    for key in ("entries", "values", "items", "points", "series", "elements"):
        v = payload.get(key)
        if isinstance(v, list):
            return [p for p in v if isinstance(p, dict)]
    # nested envelope
    for ck in ("data", "result", "payload"):
        c = payload.get(ck)
        if isinstance(c, dict):
            for key in ("entries", "values", "items", "points", "series", "elements"):
                v = c.get(key)
                if isinstance(v, list):
                    return [p for p in v if isinstance(p, dict)]
    raise ValueError(f"Could not find points list in payload keys={list(payload.keys())}")


@dataclass
class FloatSeriesStats:
    count: int
    step_min: Optional[float]
    step_max: Optional[float]
    value_min: Optional[float]
    value_max: Optional[float]
    value_mean: Optional[float]


def compute_float_series_stats(points: Iterable[NeptuneSeriesPoint]) -> FloatSeriesStats:
    """
    Computes basic stats without loading everything into memory.
    Works with NeptuneSeriesPoint objects.
    """
    n = 0
    step_min = step_max = None
    val_min = val_max = None
    val_sum = 0.0

    for p in points:
        if p.value is None:
            continue
        try:
            step_f = float(p.step)
            val_f = float(p.value)
        except Exception:
            continue

        n += 1
        step_min = step_f if step_min is None else min(step_min, step_f)
        step_max = step_f if step_max is None else max(step_max, step_f)
        val_min = val_f if val_min is None else min(val_min, val_f)
        val_max = val_f if val_max is None else max(val_max, val_f)
        val_sum += val_f

    mean = (val_sum / n) if n > 0 else None
    return FloatSeriesStats(
        count=n,
        step_min=step_min,
        step_max=step_max,
        value_min=val_min,
        value_max=val_max,
        value_mean=mean,
    )


def format_float_series(meta: Dict[str, Any]) -> str:
    """
    Format float series summary stats as a compact line.

    Uses metadata from FloatSeriesAttributeDTO: last, lastStep, min, max, average, variance.
    """
    parts = []
    for k in ("last", "lastStep", "min", "max", "average"):
        val = meta.get(k)
        if val is not None:
            # Format floats nicely
            if isinstance(val, float):
                parts.append(f"{k}={val:.6g}")
            else:
                parts.append(f"{k}={val}")
    if meta.get("variance") is not None:
        var = meta["variance"]
        if isinstance(var, float):
            parts.append(f"var={var:.4g}")
        else:
            parts.append(f"var={var}")
    return ", ".join(parts) if parts else "(no meta)"


def format_string_series(meta: Dict[str, Any]) -> str:
    """
    Format string series summary as a compact line.

    Uses metadata from StringSeriesAttributeDTO: last, lastStep.
    """
    parts = []
    for k in ("last", "lastStep"):
        val = meta.get(k)
        if val is not None:
            # Truncate long strings for display
            if isinstance(val, str) and len(val) > 50:
                val = val[:47] + "..."
            parts.append(f"{k}={val}")
    return ", ".join(parts) if parts else "(no meta)"


# -----------------------------
# CLI
# -----------------------------

def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(
        prog="neptune_fetch_full",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        description=textwrap.dedent(__doc__ or ""),
    )

    p.add_argument("--token", default=os.environ.get("NEPTUNE_API_TOKEN"), help="Neptune base64 token (env: NEPTUNE_API_TOKEN)")
    p.add_argument("--project", default=os.environ.get("NEPTUNE_PROJECT"), help="workspace/project (env: NEPTUNE_PROJECT)")
    p.add_argument("--timeout", type=float, default=30.0)

    sub = p.add_subparsers(dest="cmd", required=True)

    # Swagger exploration
    sp = sub.add_parser("swagger-op", help="Describe a swagger operation by operationId")
    sp.add_argument("--leaderboard", action="store_true", help="Use leaderboard.swagger.json")
    sp.add_argument("--backend", action="store_true", help="Use backend.swagger.json")
    sp.add_argument("--spec", default=None, help="Path to swagger.json (overrides --leaderboard/--backend)")
    sp.add_argument("--opid", required=True, help="operationId to describe")

    sp = sub.add_parser("swagger-find", help="Find swagger operations by substring/regex over path and operationId")
    sp.add_argument("--leaderboard", action="store_true")
    sp.add_argument("--backend", action="store_true")
    sp.add_argument("--spec", default=None)
    sp.add_argument("--pattern", required=True, help="Substring (case-insensitive) to match path or operationId")

    # Project info
    sub.add_parser("project-info", help="Fetch backend project info")

    # List organizations
    sub.add_parser("list-orgs", help="List organizations the user belongs to")

    # List projects for an organization
    sp = sub.add_parser("list-projects", help="List projects for an organization")
    sp.add_argument("--org", required=True, help="Organization name (e.g. 'gqsoqa')")

    # Trash runs
    sp = sub.add_parser("trash-runs", help="Move runs to trash by experimentId")
    sp.add_argument("runs", nargs="+", help="One or more experimentId UUIDs to trash")

    # Search runs with name query
    sp = sub.add_parser("search-runs", help="Search runs by name (sys/name CONTAINS query)")
    sp.add_argument("--limit", type=int, default=100)
    sp.add_argument("--offset", type=int, default=0)
    sp.add_argument("--query", default="", help="Search string to match in sys/name (uses CONTAINS)")
    sp.add_argument("--trashed", action="store_true", help="Search for trashed runs instead of active ones")
    sp.add_argument("--truncate", type=int, default=0, help="Truncate string values to N chars (0=no truncation)")
    sp.add_argument("--types", nargs="+", default=["run"], help="Entry types to search (default: run)")

    # Describe run
    sp = sub.add_parser("describe-run", help="Print everything about a run (configs, scalars, series metadata, optional stats)")
    sp.add_argument("--run", required=True, help="Run identifier: experimentId UUID")
    sp.add_argument("--series-stats", action="store_true", help="Fetch float series points and compute statistics")
    sp.add_argument("--series-max", type=int, default=0, help="Optional cap on number of series expanded (0=all)")
    sp.add_argument("--series-page-limit", type=int, default=10000)

    # Fetch float series
    sp = sub.add_parser("fetch-float-series", help="Fetch float series points for (experiment_id, attr)")
    sp.add_argument("--experiment-id", required=True)
    sp.add_argument("--attr", required=True)
    sp.add_argument("--limit", type=int, default=10000)
    sp.add_argument("--max-points", type=int, default=0)

    # Fetch string series
    sp = sub.add_parser("fetch-string-series", help="Fetch string series points for (experiment_id, attr)")
    sp.add_argument("--experiment-id", required=True)
    sp.add_argument("--attr", required=True)
    sp.add_argument("--limit", type=int, default=10000)
    sp.add_argument("--max-points", type=int, default=0)

    sp = sub.add_parser("batch-floatseries-csv", help="Batch export float series to CSV via DownloadPrepareRequest")
    sp.add_argument("--runs", nargs="+", required=True, help="Run identifiers (experimentId UUIDs in your deployment)")
    sp.add_argument("--attrs", nargs="+", required=True, help="Float series attribute paths, e.g. metrics/loss")
    sp.add_argument("--zip-path", default="batch.zip", help="Output zip path.")
    sp.add_argument("--out-dir", default="batch", help="Output directory.")


    # Generic endpoint call
    sp = sub.add_parser("call", help="Call any endpoint by method/path with optional query/body")
    sp.add_argument("--method", required=True)
    sp.add_argument("--path", required=True)
    sp.add_argument("--query", default="", help="Query string like 'a=1&b=2'")
    sp.add_argument("--json", default="", help="JSON body as string")

    return p


def parse_query_string(q: str) -> Dict[str, Any]:
    out: Dict[str, Any] = {}
    q = q.strip()
    if not q:
        return out
    for part in q.split("&"):
        if not part:
            continue
        if "=" not in part:
            out[part] = ""
        else:
            k, v = part.split("=", 1)
            out[k] = v
    return out


def main(argv: Optional[List[str]] = None) -> int:
    args = build_parser().parse_args(argv)

    if args.cmd == "swagger-op":
        # Determine spec path
        if args.spec:
            spec_path = args.spec
        else:
            if args.leaderboard == args.backend:
                raise SystemExit("Specify exactly one of --leaderboard or --backend (or use --spec).")
            spec_path = "leaderboard.swagger.json" if args.leaderboard else "backend.swagger.json"

        sw = SwaggerSpec(resolve_spec_path(spec_path))
        print(sw.describe_operation(args.opid))
        # Additionally, if op references a schema, show model props for convenience
        try:
            op = sw.find_operation(args.opid)
            for code, resp in op.responses.items():
                sch = resp.get("schema") if isinstance(resp, dict) else None
                if isinstance(sch, dict) and "$ref" in sch and sch["$ref"].startswith("#/definitions/"):
                    model = sch["$ref"].split("/")[-1]
                    props = sw.walk_model_properties(model)
                    print("\nReferenced models and properties:")
                    for m, keys in props.items():
                        print(f"  {m}: {keys}")
            for p in op.parameters:
                if p.get("in") == "body" and isinstance(p.get("schema"), dict) and "$ref" in p["schema"]:
                    ref = p["schema"]["$ref"]
                    if ref.startswith("#/definitions/"):
                        model = ref.split("/")[-1]
                        props = sw.walk_model_properties(model)
                        print("\nRequest body models and properties:")
                        for m, keys in props.items():
                            print(f"  {m}: {keys}")
        except Exception:
            pass
        return 0
    if args.cmd == "swagger-find":
        if args.spec:
            spec_path = args.spec
        else:
            if args.leaderboard == args.backend:
                raise SystemExit("Specify exactly one of --leaderboard or --backend (or use --spec).")
            spec_path = resolve_spec_path("leaderboard.swagger.json" if args.leaderboard else "backend.swagger.json")

        sw = SwaggerSpec(spec_path)
        pat = args.pattern.lower()
        for path, methods in sw.paths.items():
            if not isinstance(methods, dict):
                continue
            for m, op in methods.items():
                if not isinstance(op, dict):
                    continue
                opid = (op.get("operationId") or "")
                hay = f"{m.upper()} {path} {opid}".lower()
                if pat in hay:
                    print(m.upper(), path, "-", opid)
        return 0

    token = args.token
    project = args.project
    if not token:
        raise SystemExit("Missing token. Provide --token or set NEPTUNE_API_TOKEN.")

    # Commands that only need a token (no project required)
    if args.cmd in ("list-orgs", "list-projects"):
        # Use a dummy project to satisfy NeptuneAPI constructor
        with NeptuneAPI(api_token=token, project=project or "unused/placeholder", timeout_s=float(args.timeout)) as api:
            if args.cmd == "list-orgs":
                orgs = api.list_organizations()
                for org in orgs:
                    role = org.get("userRole", "?")
                    name = org.get("name", "?")
                    org_type = org.get("organizationType", "?")
                    plan = org.get("pricingPlan", "?")
                    print(f"  {name}\trole={role}\ttype={org_type}\tplan={plan}")
                return 0

            if args.cmd == "list-projects":
                data = api.list_projects_for_org(args.org)
                entries = data.get("entries", []) or []
                total = data.get("matchingItemCount", len(entries))
                print(f"Found {total} projects in '{args.org}':")
                for e in entries:
                    name = e.get("name", "?")
                    org_name = e.get("organizationName", "?")
                    last_activity = e.get("lastActivity", "?")
                    role = e.get("userRoleInProject", "?")
                    print(f"  {org_name}/{name}\trole={role}\tlastActivity={last_activity}")
                return 0

    if not project:
        raise SystemExit("Missing project. Provide --project or set NEPTUNE_PROJECT.")

    with NeptuneAPI(api_token=token, project=project, timeout_s=float(args.timeout)) as api:
        if args.cmd == "project-info":
            j = api.get_project_info()
            print(json.dumps(j, indent=2, default=str))
            return 0

        if args.cmd == "trash-runs":
            j = api.trash_runs(args.runs)
            updated = j.get("updatedExperimentIdentifiers", [])
            errors = j.get("errors", [])
            print(f"Trashed {len(updated)} run(s): {updated}")
            if errors:
                print(f"Errors: {errors}")
                return 1
            return 0

        if args.cmd == "search-runs":
            # Build NQL from query and flags
            nql_parts = []
            if args.query:
                escaped_query = args.query.replace('"', '\\"')
                nql_parts.append(f'`sys/name`:string CONTAINS "{escaped_query}"')
            if args.trashed:
                nql_parts.append("(`sys/trashed`:bool = true)")
            nql = " AND ".join(nql_parts)

            j = api.entries_search(
                limit=args.limit,
                offset=args.offset,
                nql=nql,
                truncate_string_to=args.truncate,
                types=args.types,
            )
            entries = j.get("entries", []) or []
            # Print summary with optional pagination info
            total = j.get("matchingItemCount", len(entries))
            print(f"Found {total} matching entries (showing {len(entries)})")
            print("-" * 80)
            for e in entries:
                eid = e.get("experimentId", "(none)")
                # Extract sys/name and sys/id if present
                name = "(none)"
                sys_id = "(none)"
                for attr in (e.get("attributes") or []):
                    if attr.get("name") == "sys/name":
                        props = attr.get("stringProperties") or {}
                        name = props.get("value", "")
                    elif attr.get("name") == "sys/id":
                        props = attr.get("stringProperties") or {}
                        sys_id = props.get("value", "")
                print(f"  {eid=}\t{sys_id=}\t{name}")
            return 0

        if args.cmd == "describe-run":
            entry = api.get_run_by_identifier(args.run)
            s = api.summarize_run_entry(entry)

            # Print basic sys info
            print("=" * 80)
            print(f"experimentId: {s.experiment_id}")
            print("Identifiers present:", {k: entry.get(k) for k in ("experimentId","id","sysId") if entry.get(k) is not None})
            print("=" * 80)

            kinds = Counter(a.kind for a in s.attrs)
            print("\nAttribute kinds:")
            for k, c in sorted(kinds.items(), key=lambda kv: (-kv[1], kv[0])):
                print(f"  {k}: {c}")

            # sys attrs (best-effort)
            creation = s.sys_attrs.get("sys/creation_time")
            dt = _try_parse_datetime_ms_or_iso(creation)
            if dt:
                print(f"Created: {iso(dt)}")
            if "sys/name" in s.sys_attrs:
                print(f"Name: {s.sys_attrs.get('sys/name')}")
            if "sys/state" in s.sys_attrs:
                print(f"State: {s.sys_attrs.get('sys/state')}")
            if "sys/failed" in s.sys_attrs:
                print(f"Failed: {s.sys_attrs.get('sys/failed')}")
            if "sys/owner" in s.sys_attrs:
                print(f"Owner: {s.sys_attrs.get('sys/owner')}")
            if "sys/tags" in s.sys_attrs:
                print(f"Tags: {s.sys_attrs.get('sys/tags')}")
            if "sys/trashed" in s.sys_attrs:
                print(f"Trashed: {s.sys_attrs.get('sys/trashed')}")
            if "sys/id" in s.sys_attrs:
                print(f"ID: {s.sys_attrs.get('sys/id')}")

            # configs
            print(f"\nConfigs (sample of out {len(s.config_attrs)} keys):")
            for k in sorted(s.config_attrs.keys())[:20]:
                v = s.config_attrs[k]
                if isinstance(v, dict) and "last" in v:
                    continue
                print(f"  {k}: {v}")
            if len(s.config_attrs) > 20:
                print(f"  ... and {len(s.config_attrs) - 20} more")

            # Series metadata with enhanced summary stats
            float_series = [a for a in s.attrs if a.kind == "floatSeries"]
            string_series = [a for a in s.attrs if a.kind == "stringSeries"]

            print("\nMetrics (float series):")
            print(f"  Count: {len(float_series)}")
            shown = 0
            for a in sorted(float_series, key=lambda x: x.name):
                stats_line = format_float_series(a.meta)
                print(f"    {a.name}: {stats_line}")
                shown += 1
                if shown >= 200:
                    break
            if len(float_series) > 200:
                print(f"    ... and {len(float_series) - 200} more")

            if string_series:
                print(f"\nString series:")
                print(f"  Count: {len(string_series)}")
                shown = 0
                for a in sorted(string_series, key=lambda x: x.name):
                    stats_line = format_string_series(a.meta)
                    print(f"    {a.name}: {stats_line}")
                    shown += 1
                    if shown >= 50:
                        break
                if len(string_series) > 50:
                    print(f"    ... and {len(string_series) - 50} more")

            # Optional: compute real stats from points (float series only)
            if args.series_stats:
                if not s.experiment_id:
                    print("\nCannot fetch series stats: missing experimentId")
                    return 0
                print("\nFloat series statistics (from actual points):")
                max_series = args.series_max if args.series_max > 0 else len(float_series)
                for a in sorted(float_series, key=lambda x: x.name)[:max_series]:
                    # Skip empty series quickly
                    if a.meta.get("last") is None:
                        print(f"  {a.name}: empty (last=None)")
                        continue
                    pts = api.fetch_float_series_points(
                        experiment_id=s.experiment_id,
                        attribute=a.name,
                        point_count=args.series_page_limit,
                    )
                    st = compute_float_series_stats(pts)
                    if st.count == 0:
                        print(f"  {a.name}: empty (0 points)")
                        continue
                    print(
                        f"  {a.name}: "
                        f"{st.count} pts, "
                        f"step[{st.step_min}, {st.step_max}], "
                        f"val[min={st.value_min}, max={st.value_max}, mean={st.value_mean}]"
                    )

            return 0

        if args.cmd == "fetch-float-series":
            point_count = args.max_points if args.max_points > 0 else args.limit
            pts = api.fetch_float_series_points(
                experiment_id=args.experiment_id,
                attribute=args.attr,
                point_count=point_count,
            )
            # Serialize NeptuneSeriesPoint objects
            pts_dicts = [dataclasses.asdict(p) for p in pts]
            print(json.dumps(pts_dicts[:50], indent=2))
            if len(pts) > 50:
                print(f"... total points fetched: {len(pts)}")
            return 0

        if args.cmd == "fetch-string-series":
            max_points = args.max_points if args.max_points > 0 else None
            pts = list(api.iter_string_series_points(
                experiment_id=args.experiment_id,
                attribute=args.attr,
                page_limit=args.limit,
                max_points=max_points,
            ))
            # Serialize NeptuneSeriesPoint objects
            pts_dicts = [dataclasses.asdict(p) for p in pts]
            print(json.dumps(pts_dicts[:50], indent=2))
            if len(pts) > 50:
                print(f"... total points fetched: {len(pts)}")
            return 0

        if args.cmd == "batch-floatseries-csv":
            data = api.get_multiple_float_series_values_csv(run_ids=args.runs, attributes=args.attrs)
            with open(args.zip_path, "wb") as f:
                f.write(data)
            print(f"Wrote: {args.zip_path} ({len(data)} bytes)")

            try:
                with zipfile.ZipFile(args.zip_path, "r") as zf:
                    zf.extractall(args.out_dir)
                print(f"Extracted all files to: {args.out_dir}")
            except Exception as e:
                print(f"[Error extracting files from zip] {e}")
            return 0


        if args.cmd == "call":
            q = parse_query_string(args.query)
            jb = json.loads(args.json) if args.json.strip() else None
            r = api.call(args.method, args.path, query=q, json_body=jb)
            # Print as JSON if possible, otherwise raw text
            ct = r.headers.get("content-type", "")
            if "application/json" in ct:
                print(json.dumps(r.json(), indent=2, default=str))
            else:
                print(r.text)
            return 0

    return 0


if __name__ == "__main__":
    raise SystemExit(main())
