Metadata-Version: 2.1
Name: llmeval
Version: 0.4.0
Author-Email: Log10 team <team@log10.io>
Project-URL: Homepage, https://log10.io/
Requires-Python: >=3.10
Requires-Dist: click>=8.1.6
Requires-Dist: hydra-core>=1.3.2
Requires-Dist: semver>=3.0.1
Requires-Dist: log10-io>=0.4.0
Description-Content-Type: text/markdown

# llmeval

LLM evaluation and test framework to build internal test harnesses and automate reliable deployment workflows

## Prerequisite Software

- `pdm` - `pip install pdm`

## Development

1. Run `pdm sync` to install all dependencies managed by pdm tool.
   - If you need to add new dependency, run `pdm add [dependency-name]`.
1. Run `python llmeval/main.py` for the cli.
1. Run `python llmeval/main.py eval hydra.mode=MULTIRUN` for evaluation.

## Format files

1. Run `black llmeval` to format all the py files.

## Build & Publish

1. Run `pdm build`
1. Run `pdm publish`
