Metadata-Version: 2.1
Name: openlrc
Version: 0.1.3
Summary: Transcribe (whisper) and translate (gpt) voice into LRC file.
Home-page: https://github.com/zh-plus/Open-Lyrics
License: MIT
Keywords: openai-gpt3,whisper,voice transcribe,lrc
Author: Hao Zheng
Author-email: zhenghaosustc@gmail.com
Requires-Python: >=3.8,<4.0
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering
Requires-Dist: audioread (>=3.0.0,<4.0.0)
Requires-Dist: colorlog (>=6.7.0,<7.0.0)
Requires-Dist: faster-whisper (>=0.6.0,<0.7.0)
Requires-Dist: ffmpeg-python (>=0.2.0,<0.3.0)
Requires-Dist: langcodes (>=3.3.0,<4.0.0)
Requires-Dist: language-data (>=1.1,<2.0)
Requires-Dist: lingua-language-detector (>=1.3.2,<2.0.0)
Requires-Dist: openai (>=0.27.6,<0.28.0)
Requires-Dist: punctuators (>=0.0.5,<0.0.6)
Requires-Dist: pytest (>=7.4.0,<8.0.0)
Requires-Dist: rich (>=12.6.0,<13.0.0)
Requires-Dist: tiktoken (>=0.3.1,<0.4.0)
Requires-Dist: tqdm (>=4.65.0,<5.0.0)
Requires-Dist: zhconv (>=1.4.3,<2.0.0)
Project-URL: Bug Tracker, https://github.com/zh-plus/Open-Lyrics/issues
Description-Content-Type: text/markdown

# Open-Lyrics

Open-Lyrics is a Python library that transcribes voice files using
[faster-whisper](https://github.com/guillaumekln/faster-whisper), and translates/polishes the resulting text
into `.lrc` files in the desired language using [OpenAI-GPT](https://github.com/openai/openai-python).

## Installation

1. Please install CUDA and cuDNN first according to https://opennmt.net/CTranslate2/installation.html to
   enable `faster-whisper`.

2. Add your [OpenAI API key](https://platform.openai.com/account/api-keys) to environment variable `OPENAI_API_KEY`.

3. Install [PyTorch](https://pytorch.org/get-started/locally/):
    ```shell
   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
   ```

4. Install [whisperx](https://github.com/m-bain/whisperX)

    ```shell
    pip install git+https://github.com/m-bain/whisperx.git
    ```

5. This project can be installed from PyPI:

    ```shell
    pip install openlrc
    ```

   or install directly from GitHub:

    ```shell
    pip install git+https://github.com/zh-plus/Open-Lyrics
    ```

## Usage

```python
from openlrc import LRCer

lrcer = LRCer()

# Single file
lrcer.run('./data/test.mp3', target_lang='zh-cn')  # Generate translated ./data/test.lrc with default translate prompt.

# Multiple files
lrcer.run(['./data/test1.mp3', './data/test2.mp3'], target_lang='zh-cn')
# Note we run the transcription sequentially, but run the translation concurrently for each file.

# Path can contain video
lrcer.run(['./data/test_audio.mp3', './data/test_video.mp4'], target_lang='zh-cn')

# Use context.yaml to improve translation
lrcer.run('./data/test.mp3', target_lang='zh-cn', context_path='./data/context.yaml')
```

### Context

Utilize the available context to enhance the quality of your translation.
Save them as `context.yaml` in the same directory as your audio file.

```yaml
background: "This is a multi-line background.
This is a basic example."
audio_type: Movie
synopsis_map: {
  movie_name1 (without extension): "This
  is a multi-line synopsis for movie1.",
  movie_name2 (without extension): "This
  is a multi-line synopsis for movie2.",
  movie_name3 (without extension): "This is a single-line synopsis for movie 3.",
}
```

## Todo

- [x] [Efficiency] Batched translate/polish for GPT request (enable contextual ability).
- [x] [Efficiency] Concurrent support for GPT request.
- [x] [Efficiency & Transcription Quality] Use [whisperx](https://github.com/m-bain/whisperX) for transcription.
- [x] [Translation Quality] Make translate prompt more robust according to https://github.com/openai/openai-cookbook.
- [x] [Usability] Automatically fix json encoder error using GPT.
- [x] [Efficiency] Asynchronously perform transcription and translation for multiple audio inputs.
- [x] [Quality] Improve batched translation/polish prompt according
  to [gpt-subtrans](https://github.com/machinewrapped/gpt-subtrans).
- [x] [Usability] Input video support.
- [ ] [Usability] Multiple output format support.
- [ ] [Quality]
  Use [multilingual language model](https://www.sbert.net/docs/pretrained_models.html#multi-lingual-models) to assess
  translation quality.
- [ ] [Quality] Speech enhancement for input audio.
- [ ] [Efficiency] Add Azure OpenAI Service support.
- [ ] [Usability] Add local LLM support.
- [ ] [Usability] Multiple translate engine (Microsoft, DeepL, Google, etc.) support.
- [ ] [Others] Add transcribed examples.
    - [ ] Song
    - [ ] Podcast
    - [ ] Audiobook

## Credits

- https://github.com/guillaumekln/faster-whisper
- https://github.com/m-bain/whisperX
- https://github.com/openai/openai-python
- https://github.com/openai/whisper
- https://github.com/machinewrapped/gpt-subtrans
- https://github.com/MicrosoftTranslator/Text-Translation-API-V3-Python
