Metadata-Version: 2.4
Name: isage-eval
Version: 0.2.0
Summary: SAGE Evaluation Framework - Metrics, profilers, and judges for AI systems
Author-email: IntelliStream Team <shuhao_zhang@hust.edu.cn>
License-Expression: MIT
Project-URL: Homepage, https://github.com/intellistream/sage-eval
Project-URL: Repository, https://github.com/intellistream/sage-eval
Keywords: evaluation,metrics,profiler,judge,LLM,AI
Requires-Python: ==3.11.*
Description-Content-Type: text/markdown
Requires-Dist: pydantic>=2.0.0
Requires-Dist: typing-extensions>=4.0.0
Provides-Extra: full
Provides-Extra: dev
Requires-Dist: isage-eval[full]; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"

# sage-eval

SAGE L3 评估组件库，提供可组合的 `Metric`、`Profiler`、`LLM Judge` 实现。

- PyPI: `isage-eval`
- Import: `sage_libs.sage_eval`
- Layer: L3 (evaluation library, not benchmark)

## 边界与约束

- 本仓库只提供评测组件实现，不承载执行平台调度职责。
- 依赖 `sage.libs.eval` 接口层并在导入时注册实现。
- 采用 fail-fast：不保留 fallback / shim / 兼容分支。

## 当前组件

### Metrics

- `AccuracyMetric`
- `BLEUMetric`
- `F1Metric`

### Profilers

- `LatencyProfiler`
- `ThroughputProfiler`

### Judges

- `FaithfulnessJudge`
- `RelevanceJudge`

## 安装

```bash
pip install isage-eval
```

## 快速使用

```python
from sage_libs.sage_eval import AccuracyMetric, LatencyProfiler, RelevanceJudge

metric = AccuracyMetric()
metric_result = metric.compute([1, 0, 1], [1, 1, 1])

profiler = LatencyProfiler()
with profiler:
    _ = sum(range(10000))

def mock_llm(_: str) -> str:
    return "SCORE: 0.9\nREASONING: Relevant response."

judge = RelevanceJudge(llm_fn=mock_llm)
judge_result = judge.judge(
    response="Paris is the capital of France.",
    question="What is the capital of France?",
)
```

## License

Apache 2.0
