Metadata-Version: 2.1
Name: ffmperative
Version: 0.0.3
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: pillow
Requires-Dist: opencv-python
Requires-Dist: ffprobe-python
Requires-Dist: ffmpeg-python
Requires-Dist: huggingface-hub (>=0.14.1)
Requires-Dist: transformers (>=4.30.0)
Requires-Dist: diffusers
Requires-Dist: accelerate
Requires-Dist: datasets
Requires-Dist: torch
Requires-Dist: soundfile
Requires-Dist: sentencepiece
Provides-Extra: full
Requires-Dist: demucs ; extra == 'full'
Requires-Dist: onnx ; extra == 'full'
Requires-Dist: onnxruntime ; extra == 'full'
Requires-Dist: scenedetect ; extra == 'full'

# FFMPerative
<p align="center">
  <img src="https://github.com/remyxai/FFMPerative/blob/main/assets/ffmperative.gif" height=400px>
  <br>
  <img src="https://img.shields.io/pypi/v/ffmperative.svg">
  <img src="https://img.shields.io/github/license/remyxai/ffmperative.svg">
  <img src="https://img.shields.io/docker/v/smellslikeml/ffmperative/latest">
</p>

## Video Production at the Speed of Chat

FFMPerative is your copilot for video production workflows. Powered by Large Language Models (LLMs) and an intuitive chat interface, it makes complex tasks as simple as typing a sentence. Leverage the power of FFmpeg and cutting-edge machine learning tools without dealing with complex command-line arguments or scripts.

* Get Video Metadata
* Sample Image from Video
* Change Video Playback Speed
* Apply FFmpeg xfade transition filters
* Resize, Crop, Flip, Reverse Video/GIF
* Make a Video from a Directory of Images 
* Overlay Image & Video for Picture-in-Picture
* Adjust Audio Levels, Background Noise Removal
* Speech-to-Text Transcription and Closed-Captions
* Split Video by N-second Gops or with Scene Detection
* Image Classifier Inference on every N-th Video Frame

## Setup 


### Debian Package (recommended)
For debian, build and install the package:
```bash
dpkg-deb --build package_build/ ffmperative.deb
sudo dpkg -i ffmperative.deb
```
Configure the package to mount the directory at `/home/$(hostname)/Videos/` by running:
```
echo -e "HUGGINGFACE_TOKEN=$HUGGINGFACE_TOKEN\nVIDEOS_PATH=/home/$(hostname)/Videos" | sudo tee /etc/ffmperative/config
```


## Quickstart

To sample an image from a video clip, simply run FFMPerative from the command-line:

```bash
ffmperative "sample the 5th frame from /path/to/video.mp4"
```

Similarly, it's simple to split a long video into short clips via scene detection:

```bash
ffmperative "split the video '/path/to/my_video.mp4' by scene"
```

Try adding closed-captions with:

```bash
ffmperative "merge subtitles '/path/to/captions.srt' with video '/path/to/my_video.mp4' calling it '/path/to/my_video_captioned.mp4'"
```

FFMPerative excels in task compositition. For instance, [curate video highlights](https://blog.remyx.ai/posts/data-processing-agents/) by analyzing speech transcripts:

![smart_trim](https://blog.remyx.ai/img/ffmperative-auto-edit-pipeline.png#center)


### Windows & Mac Setup
#### Get the Docker Image
Pull an image from DockerHub:
```bash
docker pull smellslikeml/ffmperative:latest
```

Or clone this repo and build an image with the `Dockerfile`:
```bash
git clone https://github.com/remyxai/FFMPerative.git
cd FFMPerative
docker build -t ffmperative .
```

#### Run FFMPerative in a Container
```bash
docker run -it -e HUGGINGFACE_TOKEN='YOUR_HF_TOKEN' -v /path/to/dir:/path/to/dir --entrypoint /bin/bash ffmperative:latest
```


## Features

### Python Usage
It's not required to install FFMPerative as a Debian package. If `ffmpeg` is already available on your system, you can simply install FFMPerative through pip.

#### Base Installation
You can install the base version of ffmperative with:
```bash
# from PyPI
pip install ffmperative

# from source
git clone https://github.com/remyxai/ffmperative.git
cd ffmperative
pip install .
```

#### Usage
Simply import the library and pass your command as a string to `ffmp`.

```python
from ffmperative import ffmp

ffmp("sample the 5th frame from '/path/to/video.mp4'")
```

You can also use the cli with:
```bash
ffmp do --p "sample the 5th frame from '/path/to/video.mp4'"
```

### Notebooks

Explore our notebooks for practical applications of FFMPerative:

* [Automatically Edit Videos from Google Drive in Colab](https://colab.research.google.com/drive/149byzCNd17dAehVuWXkiFQ2mVe_icLCa?usp=sharing)

### Roadmap

- [x] Basic Video Tools
- [x] Release to PyPI after Agents are Added to Transformers
- [x] Add ML-enabled Tools: [demucs](https://github.com/facebookresearch/demucs), [PySceneDetect](https://github.com/Breakthrough/PySceneDetect) 
- [x] Docker Image with Latest FFmpeg
- [ ] Host .deb package for `apt-get` installation
- [ ] Release LLM checkpoint fine-tuned to use FFMPerative Tools


### Contributing

* Have a video processing workflow in mind? Want to contribute to our project? We'd love to hear from you! Raise an issue and we'll work together to design it.


### Resources
* [Huggingface Transformers Agents](https://huggingface.co/docs/transformers/transformers_agents)
* [ffmpeg-python](https://github.com/kkroening/ffmpeg-python/)

### Community

* [B-Roll](https://b-roll.ai/)
* [@brollai](https://twitter.com/brollai)
