# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.4.0] - 2026-02-18

### Added

- `custom_fields` parameter on `evaluate()` and `EvaluationInput` model for including additional fields in model_data_logs
- Per-prompt `custom_fields` support in `evaluate_batch()` via prompt dicts

### Removed

- `extra_model_data` and `extra_model_data_fn` parameters (replaced by `custom_fields`)
- `metadata` parameter from `evaluate()` and `EvaluationInput` model

## [0.3.0] - 2026-02-16

### Added

- Custom model data fields support via `evaluate()` for passing additional metadata
- Automatic background calibration that triggers when `evaluate()` is called
- `invoke_model` parameter on `evaluate()` to automatically collect calibration responses
- Background thread management for calibration runs to prevent duplicate calibrations

### Changed

- `get_enabled_metrics()` now filters by model via `model_code` query parameter
- Calibration now runs automatically in the background when evaluating prompts

### Removed

- Manual `calibrate()` method (replaced by automatic background calibration)
- `collect_calibration_responses()` method (use `invoke_model` parameter in `evaluate()` instead)

## [0.2.0] - 2026-01-27

### Added

- `TelemetryClient` for fetching Azure Monitor data from Application Insights
- `fetch_telemetry()` method on `RAITClient` for standalone telemetry retrieval
- `include_telemetry` parameter on `evaluate()` and `evaluate_batch()` to include telemetry with evaluation results
- `azure_log_analytics_workspace_id` configuration option
- `TelemetryError` exception for telemetry-related failures
- MIT License file

### Changed

- Evaluation results now submit telemetry data alongside ethical dimension scores

## [0.1.0] - 2025-12-22

### Added

- `RAITClient` for evaluating LLM outputs across ethical dimensions
- `evaluate()` method for single prompt evaluation with automatic result posting
- `evaluate_batch()` method for bulk evaluation with callbacks
- Parallel evaluation with configurable worker count
- Hybrid encryption for secure API communication
- Configuration via environment variables or constructor parameters
- Support for 22 evaluation metrics across 8 ethical dimensions:
  - Bias and Fairness: Hate and Unfairness
  - Explainability and Transparency: Ungrounded Attributes, Groundedness, Groundedness Pro
  - Monitoring and Compliance: Content Safety
  - Legal and Regulatory Compliance: Protected Materials
  - Security and Adversarial Robustness: Code Vulnerability
  - Model Performance: Coherence, Fluency, QA, Similarity, F1 Score, BLEU, GLEU, ROUGE, METEOR, Retrieval
  - Human-AI Interaction: Relevance, Response Completeness
  - Social and Demographic Impact: Sexual, Violence, Self-Harm
