Metadata-Version: 2.1
Name: gen1
Version: 0.0.3
Summary: Text to Video synthesis
Home-page: https://github.com/kyegomez/gen1
License: MIT
Keywords: Transformers,zeta scale
Author: Gen1
Author-email: kye@apac.ai
Requires-Python: >=3.8,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3
Requires-Dist: einops
Requires-Dist: open_clip_torch
Requires-Dist: torch
Requires-Dist: transformers
Description-Content-Type: text/markdown

[![Multi-Modality](agorabanner.png)](https://discord.gg/qUtxnK2NMf)


# Gen1
My Implementation of " Structure and Content-Guided Video Synthesis with Diffusion Models" by RunwayML


The flow:

```
image => midas => clip => spacetime unet => diffusion
```


# Install
`pip3 install gen1`

# Usage
```python
import torch
from gen1.model import Gen1

model = Gen1()

images = torch.randn(1, 3, 128, 128)
video = torch.randn(1, 3, 16, 128, 128)

run_out = model.forward(images, video)

```

## Datasets
Here is a summary table of the datasets used in the Structure and Content-Guided Video Synthesis with Diffusion Models paper:

| Dataset | Type | Size | Domain | Description | Source |
|-|-|-|-|-|-|
| Internal dataset | Images | 240M | General | Uncaptioned images | Private |  
| Custom video dataset | Videos | 6.4M clips | General | Uncaptioned short video clips | Private |
| DAVIS | Videos | - | General | Video object segmentation | [Link](https://davischallenge.org/) |
| Stock footage | Videos | - | General | Diverse video clips | - |



## Citation
```
@misc{2302.03011,
Author = {Patrick Esser and Johnathan Chiu and Parmida Atighehchian and Jonathan Granskog and Anastasis Germanidis},
Title = {Structure and Content-Guided Video Synthesis with Diffusion Models},
Year = {2023},
Eprint = {arXiv:2302.03011},
```

