"""
llm-guard-kit v0.33.0: Real-time reliability monitoring, failure diagnosis,
cross-domain scoring, A2A trust management, and NVIDIA NIM backend for LLM agents.
New in v0.33.0: ToolCallExtractor, GuardPipeline, ModelTransferCalibrator,
ActiveSamplingStrategy, ChatResult.metadata, ChatGuard unfitted guard.

Validated performance (held-out evaluation only — no in-sample figures):
  Within-domain AUROC (MiniJudge v2, $0):        0.816  (14 feat, exp_minijudge_v2, HP 200 5-fold CV)
  Within-domain AUROC (MiniJudge v1, $0):        0.802  (11 feat SC_OLD, exp_minijudge_v2)
  Within-domain AUROC (+ Sonnet judge):         ~0.78   (exp89)
  Within-domain AUROC (+ local verifier):       ~0.80   (exp111, 200 labels, $0 inference)
  Cross-domain AUROC (behavioral L1, $0):       ~0.77   (exp153, HP→TriviaQA real chains)
  Cross-domain AUROC (behavioral+P(True) 50/50):~0.78   (exp120, best cross-domain signal)
  P(True) AUROC n=200:                           0.767  CI[0.697, 0.829] (exp_ptrue_expanded, CONFIRMED)
  Alert precision at FPR ≤ 10%:                 0.896  CI[0.826, 0.961] (exp_publishability, n=200)
  Non-ReAct (function-calling): use score_with_ptrue() — auto-boosts ptrue_weight to 0.95
  Code domain P(True):                           AUROC 0.833  (exp_code_math_farl, n=100 chains)
  Math domain:                                   MathVerifier TPR=100% FPR=15% (OUT OF SCOPE for SC_OLD)

v0.31.0 new:
  AdversarialChainDetector — Detect adversarially-crafted "confident wrong" chains via 16 behavioral
    features ($0 inference). CV AUROC 0.9960 ± 0.0025 CI [0.9906, 0.9996]; Precision@FPR10=0.955.
    Top signals: frac_search (adversarial chains under-search), avg_obs_len (longer retrieval),
    n_unique_queries (repeat queries), question WH-position. Pre-trained via load_default().
    Use: det = AdversarialChainDetector.load_default(); r = det.score_chain(steps, question)

v0.30.0 new:
  FederatedCalibrator — Privacy-preserving shared isotonic calibration across deployments.
    Sites export breakpoints (no raw chains). Optional Laplace DP noise (ε=1.0 default).
    Federated merge ≈ centralised calibration within ±5pp MAE (3-site DP-noised: <8pp).
    Use: cal.fit(scores, labels); params = cal.export_params(epsilon=1.0)
         merged = FederatedCalibrator.merge([params_a, params_b, params_c])
         risk   = merged.predict_one(raw_score)

v0.29.0 new:
  ConformalRiskBudget — Per-step risk budget allocator for multi-step agent chains.
    Three methods: bonferroni (budget/n), product (1-(1-B)^(1/n)), adaptive (redistribute remaining).
    Use: budget = ConformalRiskBudget(0.20); budget.allocate(5); abort = budget.observe(i, risk)
  QuestionComplexityScorer — Cross-domain novelty detector via structural question features.
    Separates multi-hop (HotpotQA) from single-hop (TriviaQA) WITHOUT any LLM or embedder.
    AUROC 0.794 unsupervised, 0.848 supervised (HP vs TV, exp_question_complexity). Gate PASSED.
    Use: scorer = QuestionComplexityScorer(); scorer.fit(hp_chains); result = scorer.score(chain)
  DebateVerifier — Multi-agent structured debate triage via Haiku mediator.
    Precision 85.4% when mediator overrides Agent 0 (OR=6.01, Fisher p<0.0001).
    Mediator calibration Spearman=-0.278, p=0.0001 (high confidence → correct).
    AUROC gate FAIL (J_debate 0.763): use as precision triage tool, not chain ranker.
    Use: dv = DebateVerifier(api_key="..."); result = dv.verify(q, [ans_a, ans_b, ans_c])
  GuardPhaseController + PhaseStatus — judge→guard phase controller.
    Phases: bootstrap → calibrating → self_sustaining. Starvation injection at <5% wrong chains.
    Use: ctrl = GuardPhaseController(); ctrl.observe(label); ctrl.should_use_judge

v0.28.0 new:
  ComprehensiveMathVerifier — Programmatic math verification combining SymPy symbolic solving,
    Python exec sandbox, step-pattern analysis (equation density, units, variable tracking), and
    answer format validation. No LLM required. Returns risk in [0, 1] (0.5 = unverifiable).
    Use: from llm_guard import ComprehensiveMathVerifier; v = ComprehensiveMathVerifier(); risk = v.score(q, steps, fa)

v0.27.0 new:
  ChatGuard.auto_fit() — Zero-label bootstrapping via Haiku P(True) pseudo-labeler. Achieves
    AUROC 0.8948 on HaluEval (600 pseudo-labels, no ground truth). Gate PASSED.
    Use: guard.auto_fit(unlabeled_pairs, n_pseudo=50)
  ChatGuard.fit(cov_features_list=...) — 16-feature CoV-extended LogReg (12 native + 4 CoV).
    AUROC 0.9930 on HaluEval (400 test pairs). Gate PASSED.
    Use: guard.fit(labeled_pairs, cov_features_list=[cov_vec.to_dict(), ...])
  ChatCoVFeatureExtractor — Extract 4 risk features from Chain-of-Verification text.
    contradiction_score, final_confidence, claim_count_norm, evidence_gap_ratio.
    Use: from llm_guard import ChatCoVFeatureExtractor; vec = ChatCoVFeatureExtractor().extract(cov_text)
  COV_FEATURE_KEYS — Canonical key order for CoV feature dicts (used in fit() and score_chat()).
  MathVerifier+SC (Component C): gate FAILED (class imbalance in GSM8K chains — not shipped).
    Root cause: Haiku achieves 95-96% correct on GSM8K, leaving <5 wrong chains in natural/temperature
    approaches; perturbation gives 0 correct chains. Next step: use harder math benchmarks.

v0.26.0 new:
  ChatGuard — Reliability scoring for chatbot (non-ReAct) responses; native ($0) and
    synthetic_trace (~$0.001/call) modes. Validated: native AUROC 0.852 (synthetic data),
    fitted ChatMiniJudge AUROC 0.931 (synthetic) / 0.991 (HaluEval real data).
    fit() REQUIRED for reliable production use (rule engine inverts on confident hallucinations).
    Use: from llm_guard import ChatGuard; guard = ChatGuard(); guard.fit(labeled_pairs); guard.score_chat(q, r)
  ChatFeatureExtractor — 12 zero-cost features from (query, response) pairs; <5ms, no API calls.
    Use: from llm_guard import ChatFeatureExtractor; vec = ChatFeatureExtractor().extract(q, r)

v0.25.0 new:
  StepTransformerVerifier — 2-layer Transformer over MiniLM step embeddings; best with
    HP+TV joint training (TV AUROC 0.705, HP 0.563); still below LSTM (0.744) on cross-domain
    Use: from llm_guard import StepTransformerVerifier; v.fit(hp_chains+tv_chains); v.score(...)
  ZeroShotCalibrator — conformal calibration with 0 human labels via P(True) pseudo-labels
    Precision 0.708 [0.574, 0.835] with 0 labels vs QuickCalibrator 0.850 with 20 labels
    Use: from llm_guard import ZeroShotCalibrator; cal.fit(unlabeled_chains, n_pseudo=50)
  MathVerifier — SymPy + Python exec sandbox for math chain risk scoring (TPR=100%, FPR=15%)
    Integrated into score_chain() via is_math_chain() routing; behavioral_components["math_verified_risk"]
    Use: score_chain() auto-routes; or from llm_guard import MathVerifier; mv.score(q, steps, fa)
  NLIGroundingVerifier — NLI entailment signal (NEGATIVE RESULT: HP AUROC 0.44 — DO NOT USE)
    TV AUROC 0.58 but CIs overlap with SC_OLD; not recommended for production
  score_with_selfconsistency() — pairwise token-F1 agreement across k candidates ($0)
  score_with_selfcritique() — Haiku self-critique with quality rating extraction (~$0.0001)
  2WikiMultiHop CORRECTION: SC_OLD AUROC corrected to 0.521 CI[0.440,0.600] (was 0.703 — invalid)

v0.24.0 new:
  confident_wrong detection fix — FARL-calibrated risk_score > 0.37 gate prevents FP on easy chains
    (FARL cycle_1 n=14 confirmed chains: mean_risk=0.382, p25=0.37; exp_farl_phase2 novelty=0.15)
  Non-ReAct auto-boost — score_chain() detects low thought coverage (<30%) and stores
    thought_coverage in behavioral_components; score_with_ptrue() auto-raises ptrue_weight
    to 0.95 when thoughts are absent, routing accuracy to P(True) for function-calling agents
  P(True) n=200 validated — AUROC 0.767 CI[0.697, 0.829] (exp_ptrue_expanded, CONFIRMED)

v0.23.0 new:
  MiniJudge v2 — 14 features (SC_OLD + 3 embedding); AUROC 0.816 +0.014 vs v1 (exp_minijudge_v2)
  Non-ReAct validation — SC_OLD transfers to function-calling format (AUROC 0.714 vs 0.708 baseline)
  Code+Math domain results — code P(True) AUROC 0.833; math AUROC 0.50 (scope limitation confirmed)
  docs/scope_limitations.md — confirmed validated/out-of-scope domains

v0.22.0 new:
  Beta status (4 - Beta) — validated across 5 benchmarks, production-ready
  Domain-invariant feature analysis — Tier-1 features documented (collect_domain_data.py)
  scripts/collect_domain_data.py — pipeline to collect + label domain chains for QuickCalibrator

v0.21.0 new:
  StreamGuardResult.skip_rewrite — fast-path flag: 59% chains skip QueryRewriter (save 400-800ms)
  AgentGuard.score_chain_guarded() — latency-optimized scoring with step-2 pre-check
  Cross-domain few-shot calibration — exp_crossdomain_fewshot: N=20 labels → +1.4pp AUROC (TriviaQA)

v0.17.0 new:
  MiniJudge                  — $0 local judge distilled from Sonnet (AUROC 0.747, exp159)
  Cross-domain TV validated  — AUROC 0.660 [CI 0.614–0.705] n=1000 chains (exp156)
  NQ cross-domain            — AUROC 0.524 (open-domain factoid, harder domain)
  QuickCalibrator            — domain-adaptive isotonic calibration (20-chain setup)
  Platform integrations      — Langfuse, LangSmith, Prometheus/Grafana, Datadog

v0.16.1 new:
  Fix FastAPI server dep to >=0.115 (starlette 0.52 compat — middleware 3-tuple fix)
  exp153 domain-invariant selection: L1+selected L2-L4 = 0.778 cross-domain (+0.8pp vs L1 alone)

v0.16.0 new:
  POST /v2/calibrate/fit   — hosted calibration endpoint: fit DeepLocalVerifier on labeled chains
  exp153 multilevel feats  — L1+L2+L3+L4 30 features; L1 baseline dominates cross-domain (0.770)
  exp155 semantic encoder  — L5 (sentence-transformer cosine) + Transformer encoder + energy baseline
  exp150-152 domain valid  — code interpreter, Chinook SQL, customer service domains validated
  exp154 Mistral-7B probe  — pending model download (expected ≥0.65 based on Qwen2.5 exp145/146)
  Latency SLA benchmark    — behavioral p50=19ms, local verifier p50=19ms

v0.15.0 new:
  probe_ensemble_blend()     — blend WhiteBoxProbe + P(True) at alpha=0.25 (+1.6pp AUROC, exp148)
  Corrected LSTM AUROC       — LSTMRiskAccumulator cross-domain 0.545 not 0.8280 (exp143 3-domain)
  Real probe validation      — exp145/146 Qwen2.5 results documented (mean=0.633/0.585)

v0.14.0 new:
  ProcessReliabilityMonitor  — generic domain-agnostic monitor (plug in any StepExtractor)
  StepExtractor ABC          — define extract(step) to add new process domains
  LLMReActExtractor          — ReAct-specific extractor (default for AgentGuard)
  confident_wrong adapter    — detects short confident chains; routes to verify_claim
  Per-domain threshold       — DriftMonitor auto-calibrates alert threshold after ALARM
  Domain-invariant LSTM      — retrieval_conf + semantic_gap replace content-heavy features

v0.10.0 new:
  AgentGuard(nim_api_key="nvapi-...")  — NVIDIA NIM backend (llama-3.1-70b judge, 8b P(True))
  compare_backends MCP tool            — per-backend AUROC A/B tracking over time
  Automatic Anthropic↔NIM fallback     — no single point of failure

v0.9.0 new:
  guard.score_with_ptrue()           — P(True) ensemble (exp120, ~$0.0003/call)
  guard.fit_structural_verifier()    — structural-only LogReg for cross-domain (exp119/124)
  guard.fit_structural_verifier(..., target_unlabeled_runs=...) — +2pp with unlabeled target data

Quick start
-----------
    from llm_guard import AgentGuard, LLMGuard, SmartRouter

    # --- Chain scoring (behavioral, $0) ---
    guard  = AgentGuard()
    result = guard.score_chain(question, steps, final_answer)
    trust  = guard.generate_trust_object(question, steps, final_answer)

    # --- Cross-domain scoring with P(True) via Anthropic (~$0.0003/call) ---
    guard  = AgentGuard(api_key="sk-ant-...")
    result = guard.score_with_ptrue(question, steps, final_answer)

    # --- Cross-domain scoring with P(True) via NVIDIA NIM (llama-3.1-8b) ---
    guard  = AgentGuard(nim_api_key="nvapi-...")
    result = guard.score_with_ptrue(question, steps, final_answer)
    # result.risk_score is behavioral+P(True) ensemble (AUROC ~0.78 cross-domain)

    # --- Cross-domain structural LogReg ($0 inference, 200 labeled source chains) ---
    guard = AgentGuard()
    guard.fit_structural_verifier(labeled_source_runs)
    # Optional: pass unlabeled target chains for feature normalization (+2pp)
    guard.fit_structural_verifier(labeled_source_runs, target_unlabeled_runs=target_runs)
    result = guard.score_chain(question, steps, final_answer)

    # --- Local verifier (within-domain, $0 inference after 200 labels) ---
    guard = AgentGuard(use_local_verifier=True)
    guard.fit_verifier(labeled_runs)
    result = guard.score_chain(question, steps, final_answer)

    # --- Zero-install nano scorer (no package needed) ---
    from llm_guard import QPPGNano
    nano = QPPGNano()
    result = nano.score_chain(question, steps, final_answer)

    # --- Self-consistency (format-agnostic, $0 — caller provides candidate answers) ---
    guard  = AgentGuard()
    result = guard.score_with_selfconsistency(
        question, steps, answer, candidate_answers=[ans2, ans3, ans4]
    )
    # result.behavioral_components["self_consistency"] = mean pairwise token-F1

    # --- Self-critique (~$0.0001/call, Haiku) ---
    guard  = AgentGuard(api_key="sk-ant-...")
    result = guard.score_with_selfcritique(question, steps, final_answer)
    # result.behavioral_components["critique_score"]  = 1-5 quality rating
    # result.behavioral_components["critique_issues"] = number of issues identified

    # --- Cost-optimal model routing ---
    router = SmartRouter(LLMGuard(api_key="sk-ant-..."))
    result = router.route("What is 15% of 240?")
    print(result.model_used)  # Haiku for easy, Sonnet/Opus for hard
"""

from qppg.guard import QPPGLLMGuard as LLMGuard
from qppg.guard import GuardResult
from llm_guard.client import GuardClient, ScoreResult, MonitorResult as _ClientMonitorResult
from llm_guard.step_extractor import StepExtractor, LLMReActExtractor
from llm_guard.process_monitor import ProcessReliabilityMonitor, MonitorResult
from llm_guard.router import SmartRouter, RouterResult
from llm_guard.agent_guard import AgentGuard, AgentStepResult, ChainTrustResult, PtrueWeightBandit, GuardPhaseController, PhaseStatus
from llm_guard.novelty_detector import LLMNoveltyDetector, NoveltyResult, QuestionComplexityScorer, ComplexityResult, _complexity_features
from llm_guard.debate_verifier import DebateVerifier, DebateResult
from llm_guard.trust_object import A2ATrustObject, TrustHop, MeshResult, StreamGuardResult, TemporalValidity
from llm_guard.query_rewriter import QueryRewriter, RewriteResult
from llm_guard.nano import QPPGNano
from llm_guard.local_verifier import LocalVerifier, FEATURE_NAMES, extract_features
from llm_guard.adapter_registry import AdapterRegistry, AdapterConfig, AdapterResult
from llm_guard.white_box_probe import WhiteBoxProbe, ProbeResult, probe_ensemble_blend
from llm_guard.step_normalizer import normalize_steps, validate_step_coverage
from llm_guard.adaptive_cisc import AdaptiveCISC, AdaptiveCISCRegistry
from llm_guard.drift_detector import DriftDetector, DriftMonitor, DriftEvent
from llm_guard.deep_verifier import DeepLocalVerifier, LSTMRiskAccumulator
from llm_guard.step_transformer import StepTransformerVerifier, encode_step
from llm_guard.quick_calibration import QuickCalibrator
from llm_guard.zero_shot_calibrator import ZeroShotCalibrator
from llm_guard.nli_grounding import NLIGroundingVerifier
from llm_guard.mini_judge import MiniJudge
from llm_guard.math_verifier import MathVerifier
from llm_guard.chat_guard import ChatGuard, ChatResult, COV_FEATURE_KEYS
from llm_guard.chat_features import ChatFeatureExtractor, ChatFeatureVector
from llm_guard.chat_cov_features import ChatCoVFeatureExtractor, CoVFeatureVector
from llm_guard.comprehensive_math_verifier import ComprehensiveMathVerifier
from llm_guard.conformal_risk_budget import ConformalRiskBudget, BudgetState
from llm_guard.federated_calibrator import FederatedCalibrator, CalibParams
from llm_guard.adversarial_detector import (
    AdversarialChainDetector, AdversarialResult, AdversarialParams,
    extract_chain_features, ADVERSARIAL_FEATURE_NAMES,
)
from llm_guard.domain_adaptive_calibrator import DomainAdaptiveCalibrator
from llm_guard.tool_call_extractor import ToolCallExtractor
from llm_guard.guard_pipeline import GuardPipeline, PipelineResult
from llm_guard.active_sampling import ActiveSamplingStrategy
from llm_guard.model_transfer_calibrator import ModelTransferCalibrator
from llm_guard.direct_generation_guard import DirectGenerationGuard, DirectGenResult
from llm_guard.function_call_guard import FunctionCallGuard
from llm_guard.multi_turn_guard import MultiTurnGuard, MultiTurnResult
from llm_guard.integrations.langchain import AgentGuardCallback, langchain_trace_to_steps
from llm_guard.integrations.llamaindex import AgentGuardEventHandler
from llm_guard.integrations.crewai import AgentGuardCrewCallback
from llm_guard.selective_router import SelectiveVerificationRouter
from llm_guard.adaptive_sampler import FailureRateAdaptiveSampler
from llm_guard.trust_scorer import TrustScorer, TrustResult
from llm_guard.schemas import (
    ChainTrustSchema, DirectGenSchema, TrustResultSchema, MultiTurnResultSchema
)

__version__ = "0.42.0"
__all__ = [
    # One-line SDK client (SaaS + local modes)
    "GuardClient",
    "ScoreResult",
    # Core guard (KNN + repair)
    "LLMGuard",
    "GuardResult",
    # Cost-optimal routing
    "SmartRouter",
    "RouterResult",
    # Agent monitoring (SC_OLD + Sonnet judge / local verifier)
    "AgentGuard",
    "AgentStepResult",
    "ChainTrustResult",
    "PtrueWeightBandit",
    # Phase controller: judge→guard transition with starvation injection (v0.29.0)
    "GuardPhaseController",
    "PhaseStatus",
    # A2A trust object + stream guard (exp113) + mesh routing (exp115) + multi-hop chain (v0.4)
    "A2ATrustObject",
    "TrustHop",
    "StreamGuardResult",
    "MeshResult",
    "TemporalValidity",
    # Query diversification
    "QueryRewriter",
    "RewriteResult",
    # Zero-install nano scorer
    "QPPGNano",
    # Local verifier (LogReg on 12 SC features, $0 inference)
    "LocalVerifier",
    "FEATURE_NAMES",
    "extract_features",
    # Adaptive adapter selection (exp116)
    "AdapterRegistry",
    "AdapterConfig",
    "AdapterResult",
    # White-box hidden-state probe (exp117) + ensemble blend (exp148)
    "WhiteBoxProbe",
    "ProbeResult",
    "probe_ensemble_blend",
    # Framework integrations (2-line drop-in callbacks)
    "AgentGuardCallback",
    "AgentGuardEventHandler",
    "AgentGuardCrewCallback",
    # Multi-format step normalization
    "normalize_steps",
    "validate_step_coverage",
    # Adaptive CISC threshold bandit
    "AdaptiveCISC",
    "AdaptiveCISCRegistry",
    # Drift detection (exp118: CUSUM + PSI, validated)
    "DriftDetector",
    "DriftMonitor",
    "DriftEvent",
    # Deep verifiers (exp138: bootstrap MLP + LSTM risk accumulator)
    "DeepLocalVerifier",
    "LSTMRiskAccumulator",
    # Step-Transformer verifier (exp_step_transformer: format-agnostic MiniLM encoder)
    "StepTransformerVerifier",
    "encode_step",
    # Mini-judge: $0 local judge distilled from Sonnet (exp159, AUROC 0.747)
    "MiniJudge",
    # Math domain symbolic verifier (exp_math_verifier: subprocess sandbox + SymPy)
    "MathVerifier",
    # Comprehensive math verifier (v0.28.0: SymPy + exec sandbox + step-pattern analysis, no LLM)
    "ComprehensiveMathVerifier",
    # Adversarial chain detection (v0.31.0): detect confident_wrong FARL-style failures ($0)
    "AdversarialChainDetector",
    "AdversarialResult",
    "AdversarialParams",
    "extract_chain_features",
    "ADVERSARIAL_FEATURE_NAMES",
    # Federated calibration (v0.30.0): privacy-preserving shared isotonic calibration
    "FederatedCalibrator",
    "CalibParams",
    # Conformal per-step risk budgeting (v0.29.0)
    "ConformalRiskBudget",
    "BudgetState",
    # Cross-domain novelty via structural question features (v0.29.0, AUROC 0.794)
    "QuestionComplexityScorer",
    "ComplexityResult",
    # Multi-agent structured debate triage (v0.29.0, precision 85.4%, OR=6.01)
    "DebateVerifier",
    "DebateResult",
    # OOD detection for agent chains (Mahalanobis + conformal + IsoForest)
    "LLMNoveltyDetector",
    "NoveltyResult",
    # Domain-adaptive calibration (QuickCalibrator: 20-chain isotonic setup)
    "QuickCalibrator",
    # Domain-adaptive calibration with warm-start conformal (DomainAdaptiveCalibrator)
    "DomainAdaptiveCalibrator",
    # Zero-label conformal calibration (ZeroShotCalibrator: P(True) pseudo-labels)
    "ZeroShotCalibrator",
    # NLI-based answer-observation entailment verifier ($0, pre-trained model)
    "NLIGroundingVerifier",
    # Chatbot (non-ReAct) reliability scoring (v0.26.0+)
    "ChatGuard",
    "ChatResult",
    "ChatFeatureExtractor",
    "ChatFeatureVector",
    # CoV feature extraction for 16-feature path (v0.27.0)
    "ChatCoVFeatureExtractor",
    "CoVFeatureVector",
    "COV_FEATURE_KEYS",
    # v0.33.0 new components
    "ToolCallExtractor",
    "GuardPipeline",
    "PipelineResult",
    "ActiveSamplingStrategy",
    "ModelTransferCalibrator",
    # v0.34.0 new
    "SelectiveVerificationRouter",
    "FailureRateAdaptiveSampler",
    "DirectGenerationGuard",
    "DirectGenResult",
    "FunctionCallGuard",
    "MultiTurnGuard",
    "MultiTurnResult",
    "langchain_trace_to_steps",
    # Composite 0-100 trust score with natural-language explanation (TrustScorer)
    "TrustScorer",
    "TrustResult",
    # Pydantic v2 schemas for result types (Task 3.4 — GET /schema, client code gen)
    "ChainTrustSchema",
    "DirectGenSchema",
    "TrustResultSchema",
    "MultiTurnResultSchema",
    # Generic framework (v0.14.0)
    "StepExtractor",
    "LLMReActExtractor",
    "ProcessReliabilityMonitor",
    "MonitorResult",
]
