Metadata-Version: 2.1
Name: danoliterate
Version: 0.0.3
Summary: Benchmark of Generative Large Language Models in Danish
Home-page: https://github.com/sorenmulli/danoliterate
Author: Søren Winkel Holm
Author-email: swholm@protonmail.com
License: Apache License 2.0
Description-Content-Type: text/markdown
Requires-Dist: pandas~=1.5.3
Requires-Dist: datasets~=2.14.5
Requires-Dist: transformers~=4.36.1
Requires-Dist: torch~=2.1.1
Requires-Dist: accelerate~=0.23.0
Requires-Dist: safetensors~=0.3.3
Requires-Dist: matplotlib~=3.8.0
Requires-Dist: simple-term-menu~=1.6.1
Requires-Dist: danlp~=0.1.2
Requires-Dist: gensim~=4.3.2
Requires-Dist: deepspeed~=0.12.4
Requires-Dist: sentencepiece~=0.1.99
Requires-Dist: augmenty~=1.4.3
Requires-Dist: dacy~=2.7.6
Requires-Dist: google-cloud-aiplatform~=1.38.1
Requires-Dist: openai~=0.28.1
Requires-Dist: streamlit~=1.27.0
Requires-Dist: huggingface_hub~=0.19.4
Requires-Dist: hydra-core~=1.3.2
Requires-Dist: hydra-colorlog~=1.2.0
Requires-Dist: omegaconf~=2.3.0
Requires-Dist: wandb~=0.15.11
Requires-Dist: gitpython~=3.1.37
Requires-Dist: black~=23.7.0
Requires-Dist: isort~=5.12.0
Requires-Dist: mypy~=1.5.1
Requires-Dist: pylint~=2.17.5
Requires-Dist: pandas-stubs~=2.0.3
Requires-Dist: types-tqdm~=4.66.0
Requires-Dist: types-requests~=2.31.0
Provides-Extra: full
Requires-Dist: pypdf~=3.16.1; extra == "full"
Requires-Dist: fasttext-wheel~=0.9.2; extra == "full"
Requires-Dist: peft~=0.5.0; extra == "full"
Requires-Dist: scipy~=1.11.3; extra == "full"
Requires-Dist: nltk~=3.8.1; extra == "full"
Requires-Dist: rouge_score~=0.1.2; extra == "full"
Requires-Dist: evaluate~=0.4.0; extra == "full"
Requires-Dist: bert_score~=0.3.13; extra == "full"
Requires-Dist: spacy~=3.5.4; extra == "full"
Requires-Dist: levenshtein~=0.23.0; extra == "full"
Requires-Dist: seqeval~=1.2.2; extra == "full"
Requires-Dist: beautifulsoup4~=4.12.2; extra == "full"
Requires-Dist: trl~=0.7.4; extra == "full"

# Are LLMs Danoliterate?


A benchmark tool for Generative Large Language Models in Danish.

Note: This repository is currently work-in-progress while the master's thesis is finished.
Implementations are subject to change.

## Installation

The package has been developed and used with Python 3.11.
To install the package in a base version, enabling model execution, install
```
pip install danoliterate
```

To install it with the entire suite of dependencies that were used in thesis, run 
```
make install
```

## Usage

```
python -m danoliterate do=evaluate
```
