"""
llm-guard-kit v0.66.0 — LLM agent reliability scoring and failure diagnosis.

All AUROC figures below are on live/unbiased chains (sklearn roc_auc_score) unless
explicitly marked FARL-only. Use the Tier 1 components for production; Tier 2 with
caveats; Tier 3 only for research.

─── TIER 1 — Production Validated ───────────────────────────────────────────────
  (Live-chain AUROC ≥ 0.70, CI lower bound ≥ 0.60)

  SC_OLD / DomainRouter (FEVER):     0.717  CI[0.622, 0.814]  [exp_fever_behavioral]
  DomainRouter fact_verification:    0.810  HIGH CI[0.681, 0.922]  [E2 val, faj_r signal]
  DomainRouter multi_hop_entity:     0.788  HIGH CI[0.620, 0.924]  [E2 val, mfe_sem_sim]
  DomainRouter trivia_recall:        0.751  HIGH CI[0.631, 0.858]  [E2 val, faj_a signal]
  MiniJudge v2 (transfer, live HP):  0.752  [exp_paper_baselines, sklearn]
  LLM quality scorer (continuous):   0.767  CI[0.697, 0.829]  [exp_ptrue_expanded, sklearn]
  ChatGuard with judge:              0.714  CI[0.611, 0.811]  [TruthfulQA n=100]
  MultilingualSCScorer:  AR=0.864, ES=0.897, ZH=0.873  [far-shuffle, live chains]
  Alert precision @ FPR ≤ 10%:      0.896  CI[0.826, 0.961]
  sycophancy_score() (NLP survey):   0.9296 CI[0.8754, 0.9703]  [Perez 2022, n=50, zero-shot]
    → direction corrected v0.66.0; cc > 0.004167 → prec=0.909, recall=0.800

─── TIER 2 — Usable with caveats (0.55–0.75; J5 confirmed on fresh holdout only): ─
  (Live-chain AUROC 0.55–0.75, or val-only validation)

  SC_OLD behavioral (HP):            0.625  [E1 val split, n=43; canonical eval pipeline]
    → Upper estimate. Production range: 0.55–0.63 on arbitrary HP questions.
    → exp_paper_baselines (0.557) used a different chain generation run; treat as lower bound.
  SC_OLD behavioral (TriviaQA):      0.709  [exp_paper_baselines, live TV chains]
  SC_OLD behavioral (NQ):            0.603  [exp_paper_baselines]
  DomainRouter mean (5 domains):     0.728  [E2 val-only, sklearn — val split, not test]
  DomainRouter open_factual (NQ):    0.667  LOW CI[0.487, 0.833]  [n=40, E2 val]
  DomainRouter multi_hop_bridge (HP):0.625  LOW CI[0.452, 0.785]  [n=43 — hard ceiling]
  MultiTurnGuard.score_factual_turn: 0.687  CI[0.620, 0.755]  [HP n=185]
  StepNormalizer browser adapter:    0.636  [SC_OLD on Playwright-normalized chains]
  J5 ensemble (Haiku judge, HP):    0.6755  [0.5057, 0.8217]  (fresh holdout, confirmed)
  J5 ensemble (Haiku judge, TV):    0.7247  [0.5816, 0.8522]  (fresh holdout, confirmed)
  J5 ensemble (Haiku judge, NQ):    0.7652  [0.6125, 0.8952]  (fresh holdout, confirmed)
  J5 ensemble (Haiku judge, 2Wiki): 0.7778  [0.6352, 0.8995]  (fresh holdout, confirmed)
  Cross-arch ensemble (Haiku+LLaMA-3.1-8B max, HP): 0.6926 [0.5993, 0.7773] — HP ceiling broken

─── TIER 3 — Experimental / Known-Bad ───────────────────────────────────────────
  (AUROC < 0.55 on live chains, or no live-chain evaluation)

  AdversarialChainDetector:          0.397  natural HP chains (NEGATIVELY correlated)
    → 0.9960 CV on FARL-engineered chains only. Deploying on real queries gives
      worse-than-random detection. Do NOT use for production reliability monitoring.
  ChatGuard cross-dataset:           0.467  [TruthfulQA; distributional mismatch vs HaluEval]
  MFE Phase B (Sonnet 4-6):          0.467  [Phase A→B gap, model-agnostic failure]
  NLIGroundingVerifier:              0.440  [HP AUROC, exp_nli_grounding — negative result]
  sycophancy_score() CC-based:       0.9296 CI[0.8754, 0.9703]  NLP survey format  ← TIER 1
    → Direction corrected v0.66.0: use cc directly (not 1-cc). Zero-shot, no training.
    → Scope: Perez 2022 NLP survey (multiple-choice). Free-form sycophancy untested.
  SycophancyPremiseFlipDetector:     0.480  [Perez 2022 opinion data; premise-flip approach]
  WhiteBoxProbe cross-domain:        0.5699  SCOPE_LIMITED (gate 0.70); HP in-domain 0.6521
  P(True) NQ (binary count):         0.623  GATE FAIL (<0.65) — tie-handling bug corrected
    → was 0.810 due to custom auroc() flaw. DomainRouter uses mfe_sc_pair (0.667) for NQ.

─── CORRECTION LOG (v0.66.0) ────────────────────────────────────────────────────
  - sycophancy_score() AUROC 0.0704 → 0.9296: experiment stored 1-cc (wrong direction).
    Correct predictor is cc directly. is_sycophantic threshold: cc < 0.5 → cc > 0.004167.
    Zero-shot, no retraining. Scope: Perez 2022 NLP survey multiple-choice format.
  - AgentShield: fixed AttributeError — ChainTrustResult has no `judge_risk` field.
    Now uses `result_j.risk_score` (J5 blend already computed in score_chain).

─── CORRECTION LOG (v0.65.0 → v0.65.2) ─────────────────────────────────────────
  - SC_OLD HP "~0.81": was FARL-augmented chains; live chains = 0.557-0.625
  - J5 "0.762/0.773": FARL-adjacent chains; not yet tested on live chains
  - MiniJudge "0.747": wrong experiment (exp159 soft-label); correct=0.816 CV / 0.752 transfer
  - P(True) NQ "0.810": tie-handling bug in custom auroc(); correct sklearn = 0.623
    Bug: sorted(zip(scores,labels), reverse=True) secondary sort is label DESC,
    ranking wrong items (label=1) above correct items (label=0) in tied score groups.
    SC_OLD unaffected (continuous); P(True) has 4 discrete values → +0.206 inflation.
  - P(True) 0.767 (exp_ptrue_expanded): continuous LLM quality scorer, sklearn — VALID.

─── FARL-ONLY FIGURES (for adversarial chain detection research only) ────────────
  SC_OLD on FARL-augmented HP:  0.817  [exp77, adversarial chains only]
  MiniJudge v2 5-fold CV:       0.816  [exp_minijudge_v2, FARL HP]
  AdversarialDetector CV:       0.9960 [exp_adversarial_detection, FARL chains only]

Planned subpackages (Wave 3 refactor in progress):
  llm_guard.scoring, llm_guard.calibration, llm_guard.multi_turn,
  llm_guard.adversarial, llm_guard.verification, llm_guard.features,
  llm_guard.healing, llm_guard.trust, llm_guard.monitoring,
  llm_guard.pipeline, llm_guard.infrastructure, llm_guard.integrations

Note: LabelFreeScorer, QppgMonitor, FailureTaxonomist, SelfHealer are in
      qppg_service, not llm_guard.
"""

from qppg.guard import QPPGLLMGuard as LLMGuard
from qppg.guard import GuardResult
from llm_guard.infrastructure.client import GuardClient, ScoreResult, MonitorResult as _ClientMonitorResult
from llm_guard.features.step_extractor import StepExtractor, LLMReActExtractor
from llm_guard.monitoring.process_monitor import ProcessReliabilityMonitor, MonitorResult
from llm_guard.trust.router import SmartRouter, RouterResult
from llm_guard.scoring.agent_guard import AgentGuard, AgentStepResult, ChainTrustResult, BatchHandle, PtrueWeightBandit, GuardPhaseController, PhaseStatus, SycophancyResult
from llm_guard.adversarial.novelty_detector import LLMNoveltyDetector, NoveltyResult, QuestionComplexityScorer, ComplexityResult, _complexity_features
from llm_guard.healing.debate_verifier import DebateVerifier, DebateResult
from llm_guard.trust.trust_object import A2ATrustObject, TrustHop, MeshResult, StreamGuardResult, TemporalValidity
from llm_guard.healing.query_rewriter import QueryRewriter, RewriteResult
from llm_guard.scoring.nano import QPPGNano
from llm_guard.scoring.local_verifier import LocalVerifier, FEATURE_NAMES, extract_features
from llm_guard.infrastructure.adapter_registry import AdapterRegistry, AdapterConfig, AdapterResult
from llm_guard.features.white_box_probe import WhiteBoxProbe, ProbeResult, probe_ensemble_blend
from llm_guard.features.step_normalizer import normalize_steps, validate_step_coverage
from llm_guard.pipeline.adaptive_cisc import AdaptiveCISC, AdaptiveCISCRegistry
from llm_guard.monitoring.drift_detector import DriftDetector, DriftMonitor, DriftEvent
from llm_guard.scoring.deep_verifier import DeepLocalVerifier, LSTMRiskAccumulator
from llm_guard.features.step_transformer import StepTransformerVerifier, encode_step
from llm_guard.calibration.quick_calibration import QuickCalibrator
from llm_guard.calibration.zero_shot_calibrator import ZeroShotCalibrator
from llm_guard.verification.nli_grounding import NLIGroundingVerifier
from llm_guard.scoring.mini_judge import MiniJudge
from llm_guard.verification.math_verifier import MathVerifier
from llm_guard.multi_turn.chat_guard import ChatGuard, ChatResult, COV_FEATURE_KEYS
from llm_guard.multi_turn.chat_features import ChatFeatureExtractor, ChatFeatureVector
from llm_guard.multi_turn.chat_cov_features import ChatCoVFeatureExtractor, CoVFeatureVector
from llm_guard.verification.comprehensive_math_verifier import ComprehensiveMathVerifier
from llm_guard.calibration.conformal_risk_budget import ConformalRiskBudget, BudgetState
from llm_guard.calibration.federated_calibrator import FederatedCalibrator, CalibParams
from llm_guard.adversarial.adversarial_detector import (
    AdversarialChainDetector, AdversarialResult, AdversarialParams,
    extract_chain_features, ADVERSARIAL_FEATURE_NAMES,
)
from llm_guard.calibration.domain_adaptive_calibrator import DomainAdaptiveCalibrator
from llm_guard.features.tool_call_extractor import ToolCallExtractor
from llm_guard.pipeline.guard_pipeline import GuardPipeline, PipelineResult
from llm_guard.pipeline.active_sampling import ActiveSamplingStrategy
from llm_guard.calibration.model_transfer_calibrator import ModelTransferCalibrator
from llm_guard.scoring.direct_generation_guard import DirectGenerationGuard, DirectGenResult
from llm_guard.scoring.function_call_guard import FunctionCallGuard
from llm_guard.multi_turn.multi_turn_guard import MultiTurnGuard, MultiTurnResult
from llm_guard.integrations.langchain import AgentGuardCallback, langchain_trace_to_steps
from llm_guard.integrations.llamaindex import AgentGuardEventHandler
from llm_guard.integrations.crewai import AgentGuardCrewCallback
from llm_guard.trust.selective_router import SelectiveVerificationRouter
from llm_guard.pipeline.adaptive_sampler import FailureRateAdaptiveSampler
from llm_guard.trust.trust_scorer import TrustScorer, TrustResult
from llm_guard.pipeline.schemas import (
    ChainTrustSchema, DirectGenSchema, TrustResultSchema, MultiTurnResultSchema
)
# Patent 7: gate FAIL in v0.44 — present in code, not exported
# from llm_guard.verification.density_matrix_scorer import DensityMatrixScorer, DensityMatrixResult
# Patent 6: gate PASS (window=5, threshold=0.05) — defaults updated in calibration_convergence_monitor.py
from llm_guard.calibration.calibration_convergence_monitor import CalibrationConvergenceMonitor, ConvergenceStatus
from llm_guard.calibration.energy_aware_calibrator import EnergyAwareCalibrator, EnergyCalibrationResult
from llm_guard.pipeline.continual_scorer import ContinualScorer
from llm_guard.features.mechanism_layer import (
    MechanismFeatureExtractor,
    EnsembleMechanismClassifier,
    FEATURE_NAMES as MECHANISM_FEATURE_NAMES,
    PHASE_B_FEATURE_NAMES,
    bootstrap_ci,
    run_cv_experiment,
    load_halueval_qa,
    load_halueval_summarization,
    load_halueval_dialogue,
    load_truthfulqa,
    load_sycophancy,
    load_pubmed_qa,
    load_quality_context_drift,
    load_arc_research,
)

from llm_guard.scoring.deep_chain_scorer import (
    DeepChainScorer, DeepChainResult, RetrievalCascadeScorer,
)

from llm_guard.adversarial.sycophancy_detector import SycophancyPremiseFlipDetector
from llm_guard.verification.sql_validator import (
    SQLSemanticValidator,
    SQLValidationResult,
    SQLExecutionOracle,
    SQLOracleResult,
    SQL_DIALECTS,
)
from llm_guard.verification.chart_validator import ChartValidator, ChartValidationResult
from llm_guard.trust.orchestrator_adapters import (
    AutoGenAdapter,
    CrewAIAdapter,
    OrchestratorScoringResult,
)
# Wave 1/2 research modules (v0.56.0)
from llm_guard.infrastructure.domain_invariant_selector import DomainInvariantSelector
from llm_guard.adversarial.min_perturbation_attacker import MinPerturbationAttacker
from llm_guard.infrastructure.multiagent_simulator import MultiAgentSimulator, SimulationResult
from llm_guard.monitoring.intra_chain_drift_detector import IntraChainDriftDetector, DriftResult
from llm_guard.features.multilingual_sc_scorer import MultilingualSCScorer
from llm_guard.verification.proof_step_verifier import ProofStepVerifier, ProofVerificationResult
from llm_guard.features.visual_caption_bridge import VisualCaptionBridge
from llm_guard.scoring.semantic_entropy import SemanticEntropyScorer, SemanticEntropyResult
from llm_guard.verification.retrieval_grounding import RetrievalGroundingVerifier, GroundingResult
from llm_guard.verification.hallufield import HalluFieldScorer, HalluFieldResult
from llm_guard.healing.repair_bandit import RepairBandit
from llm_guard.infrastructure.response_cache import ResponseCache
from llm_guard.domain_router import (
    QuestionType,
    DomainRoutedResult,
    QuestionTypeClassifier,
    DomainRouter,
)

__version__ = "0.66.1"
__all__ = [
    # Cost routing + caching (v0.62.0)
    "ResponseCache",
    # SQL semantic validation + execution oracle (v0.54.0)
    "SQLSemanticValidator",
    "SQLValidationResult",
    "SQLExecutionOracle",
    "SQLOracleResult",
    "SQL_DIALECTS",
    # Chart / table validation (v0.54.0)
    "ChartValidator",
    "ChartValidationResult",
    # AutoGen + CrewAI orchestrator adapters (v0.54.0)
    "AutoGenAdapter",
    "CrewAIAdapter",
    "OrchestratorScoringResult",
    # Deep multi-hop chain scoring (v0.52.0)
    "DeepChainScorer",
    "DeepChainResult",
    "RetrievalCascadeScorer",
    # Sycophancy detection — experimental, gated (v0.53.0)
    "SycophancyPremiseFlipDetector",
    # One-line SDK client (SaaS + local modes)
    "GuardClient",
    "ScoreResult",
    # Core guard (KNN + repair)
    "LLMGuard",
    "GuardResult",
    # Cost-optimal routing
    "SmartRouter",
    "RouterResult",
    # Agent monitoring (SC_OLD + Sonnet judge / local verifier)
    "AgentGuard",
    "AgentStepResult",
    "ChainTrustResult",
    "BatchHandle",
    "PtrueWeightBandit",
    # Phase controller: judge→guard transition with starvation injection (v0.29.0)
    "GuardPhaseController",
    "PhaseStatus",
    # A2A trust object + stream guard (exp113) + mesh routing (exp115) + multi-hop chain (v0.4)
    "A2ATrustObject",
    "TrustHop",
    "StreamGuardResult",
    "MeshResult",
    "TemporalValidity",
    # Query diversification
    "QueryRewriter",
    "RewriteResult",
    # Zero-install nano scorer
    "QPPGNano",
    # Local verifier (LogReg on 12 SC features, $0 inference)
    "LocalVerifier",
    "FEATURE_NAMES",
    "extract_features",
    # Adaptive adapter selection (exp116)
    "AdapterRegistry",
    "AdapterConfig",
    "AdapterResult",
    # White-box hidden-state probe (exp117) + ensemble blend (exp148)
    "WhiteBoxProbe",
    "ProbeResult",
    "probe_ensemble_blend",
    # Framework integrations (2-line drop-in callbacks)
    "AgentGuardCallback",
    "AgentGuardEventHandler",
    "AgentGuardCrewCallback",
    # Multi-format step normalization
    "normalize_steps",
    "validate_step_coverage",
    # Adaptive CISC threshold bandit
    "AdaptiveCISC",
    "AdaptiveCISCRegistry",
    # Drift detection (exp118: CUSUM + PSI, validated)
    "DriftDetector",
    "DriftMonitor",
    "DriftEvent",
    # Deep verifiers (exp138: bootstrap MLP + LSTM risk accumulator)
    "DeepLocalVerifier",
    "LSTMRiskAccumulator",
    # Step-Transformer verifier (exp_step_transformer: format-agnostic MiniLM encoder)
    "StepTransformerVerifier",
    "encode_step",
    # Mini-judge: $0 local judge distilled from Sonnet (exp159, AUROC 0.747)
    "MiniJudge",
    # Math domain symbolic verifier (exp_math_verifier: subprocess sandbox + SymPy)
    "MathVerifier",
    # Comprehensive math verifier (v0.28.0: SymPy + exec sandbox + step-pattern analysis, no LLM)
    "ComprehensiveMathVerifier",
    # Adversarial chain detection (v0.31.0): detect confident_wrong FARL-style failures ($0)
    "AdversarialChainDetector",
    "AdversarialResult",
    "AdversarialParams",
    "extract_chain_features",
    "ADVERSARIAL_FEATURE_NAMES",
    # Federated calibration (v0.30.0): privacy-preserving shared isotonic calibration
    "FederatedCalibrator",
    "CalibParams",
    # Conformal per-step risk budgeting (v0.29.0)
    "ConformalRiskBudget",
    "BudgetState",
    # Cross-domain novelty via structural question features (v0.29.0, AUROC 0.794)
    "QuestionComplexityScorer",
    "ComplexityResult",
    # Multi-agent structured debate triage (v0.29.0, precision 85.4%, OR=6.01)
    "DebateVerifier",
    "DebateResult",
    # OOD detection for agent chains (Mahalanobis + conformal + IsoForest)
    "LLMNoveltyDetector",
    "NoveltyResult",
    # Domain-adaptive calibration (QuickCalibrator: 20-chain isotonic setup)
    "QuickCalibrator",
    # Domain-adaptive calibration with warm-start conformal (DomainAdaptiveCalibrator)
    "DomainAdaptiveCalibrator",
    # Zero-label conformal calibration (ZeroShotCalibrator: P(True) pseudo-labels)
    "ZeroShotCalibrator",
    # NLI-based answer-observation entailment verifier ($0, pre-trained model)
    "NLIGroundingVerifier",
    # Chatbot (non-ReAct) reliability scoring (v0.26.0+)
    "ChatGuard",
    "ChatResult",
    "ChatFeatureExtractor",
    "ChatFeatureVector",
    # CoV feature extraction for 16-feature path (v0.27.0)
    "ChatCoVFeatureExtractor",
    "CoVFeatureVector",
    "COV_FEATURE_KEYS",
    # v0.33.0 new components
    "ToolCallExtractor",
    "GuardPipeline",
    "PipelineResult",
    "ActiveSamplingStrategy",
    "ModelTransferCalibrator",
    # v0.34.0 new
    "SelectiveVerificationRouter",
    "FailureRateAdaptiveSampler",
    "DirectGenerationGuard",
    "DirectGenResult",
    "FunctionCallGuard",
    "MultiTurnGuard",
    "MultiTurnResult",
    "langchain_trace_to_steps",
    # Composite 0-100 trust score with natural-language explanation (TrustScorer)
    "TrustScorer",
    "TrustResult",
    # Pydantic v2 schemas for result types (Task 3.4 — GET /schema, client code gen)
    "ChainTrustSchema",
    "DirectGenSchema",
    "TrustResultSchema",
    "MultiTurnResultSchema",
    # Patent 7: gate FAIL v0.44 — DensityMatrixScorer not exported
    # Patent 6: new calibration_monitor gate FAIL; old CalibrationConvergenceMonitor retained
    "CalibrationConvergenceMonitor",
    "ConvergenceStatus",
    # Patent 8: Energy-aware calibrator (v0.44.0)
    "EnergyAwareCalibrator",
    "EnergyCalibrationResult",
    # Patent 5: ContinualScorer with CPC (v0.44.0)
    "ContinualScorer",
    # Mechanism layer: Phase A/B feature extractor + dataset loaders + CV utilities
    "MechanismFeatureExtractor",
    "MECHANISM_FEATURE_NAMES",
    "PHASE_B_FEATURE_NAMES",
    "bootstrap_ci",
    "run_cv_experiment",
    "load_halueval_qa",
    "load_halueval_summarization",
    "load_halueval_dialogue",
    "load_truthfulqa",
    "load_sycophancy",
    # Generic framework (v0.14.0)
    "StepExtractor",
    "LLMReActExtractor",
    "ProcessReliabilityMonitor",
    "MonitorResult",
    # Domain router (v0.64.0)
    "QuestionType",
    "DomainRoutedResult",
    "QuestionTypeClassifier",
    "DomainRouter",
    # Wave 1/2 research modules (v0.56.0)
    "DomainInvariantSelector",
    "MinPerturbationAttacker",
    "MultiAgentSimulator",
    "SimulationResult",
    "IntraChainDriftDetector",
    "DriftResult",
    "MultilingualSCScorer",
    "ProofStepVerifier",
    "ProofVerificationResult",
    "VisualCaptionBridge",
]
