Metadata-Version: 2.1
Name: danoliterate
Version: 0.0.3
Summary: Benchmark of Generative Large Language Models in Danish
Home-page: https://github.com/sorenmulli/danoliterate
Author: Søren Winkel Holm
Author-email: swholm@protonmail.com
License: Apache License 2.0
Description-Content-Type: text/markdown
Requires-Dist: pandas ~=1.5.3
Requires-Dist: datasets ~=2.14.5
Requires-Dist: transformers ~=4.36.1
Requires-Dist: torch ~=2.1.1
Requires-Dist: accelerate ~=0.23.0
Requires-Dist: safetensors ~=0.3.3
Requires-Dist: matplotlib ~=3.8.0
Requires-Dist: simple-term-menu ~=1.6.1
Requires-Dist: danlp ~=0.1.2
Requires-Dist: gensim ~=4.3.2
Requires-Dist: deepspeed ~=0.12.4
Requires-Dist: sentencepiece ~=0.1.99
Requires-Dist: augmenty ~=1.4.3
Requires-Dist: dacy ~=2.7.6
Requires-Dist: google-cloud-aiplatform ~=1.38.1
Requires-Dist: openai ~=0.28.1
Requires-Dist: streamlit ~=1.27.0
Requires-Dist: huggingface-hub ~=0.19.4
Requires-Dist: hydra-core ~=1.3.2
Requires-Dist: hydra-colorlog ~=1.2.0
Requires-Dist: omegaconf ~=2.3.0
Requires-Dist: wandb ~=0.15.11
Requires-Dist: gitpython ~=3.1.37
Requires-Dist: black ~=23.7.0
Requires-Dist: isort ~=5.12.0
Requires-Dist: mypy ~=1.5.1
Requires-Dist: pylint ~=2.17.5
Requires-Dist: pandas-stubs ~=2.0.3
Requires-Dist: types-tqdm ~=4.66.0
Requires-Dist: types-requests ~=2.31.0
Provides-Extra: full
Requires-Dist: pypdf ~=3.16.1 ; extra == 'full'
Requires-Dist: fasttext-wheel ~=0.9.2 ; extra == 'full'
Requires-Dist: peft ~=0.5.0 ; extra == 'full'
Requires-Dist: scipy ~=1.11.3 ; extra == 'full'
Requires-Dist: nltk ~=3.8.1 ; extra == 'full'
Requires-Dist: rouge-score ~=0.1.2 ; extra == 'full'
Requires-Dist: evaluate ~=0.4.0 ; extra == 'full'
Requires-Dist: bert-score ~=0.3.13 ; extra == 'full'
Requires-Dist: spacy ~=3.5.4 ; extra == 'full'
Requires-Dist: levenshtein ~=0.23.0 ; extra == 'full'
Requires-Dist: seqeval ~=1.2.2 ; extra == 'full'
Requires-Dist: beautifulsoup4 ~=4.12.2 ; extra == 'full'
Requires-Dist: trl ~=0.7.4 ; extra == 'full'

# Are LLMs Danoliterate?


A benchmark tool for Generative Large Language Models in Danish.

Note: This repository is currently work-in-progress while the master's thesis is finished.
Implementations are subject to change.

## Installation

The package has been developed and used with Python 3.11.
To install the package in a base version, enabling model execution, install
```
pip install danoliterate
```

To install it with the entire suite of dependencies that were used in thesis, run 
```
make install
```

## Usage

```
python -m danoliterate do=evaluate
```
