Metadata-Version: 2.1
Name: layered-vision
Version: 0.0.5
Summary: Compose final output(s), out of layers of images, videos or similar.
Home-page: https://github.com/de-code/layered-vision
Author: Daniel Ecer
License: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
Requires-Dist: mss (==6.1.0)
Requires-Dist: numpy (==1.19.5)
Requires-Dist: opencv-python (==4.5.1.48)
Requires-Dist: pafy (==0.5.5)
Requires-Dist: pyfakewebcam (==0.1.0)
Requires-Dist: PyYAML (==5.4.1)
Requires-Dist: tf-bodypix[tf,tfjs] (==0.3.4)
Requires-Dist: youtube-dl (==2021.2.4.1)

# Layered Vision


Goals of this project is:

* A tool to allow the composition of images or videos via a configuration file (e.g. as a virtual webcam).

This project is still very much experimental and may change significantly.

## Install

Install with all dependencies:

```bash
pip install layered-vision[all]
```

Install with minimal dependencies:

```bash
pip install layered-vision
```

Extras are provided to make it easier to provide or exclude dependencies
when using this project as a library:

| extra name | description
| ---------- | -----------
| bodypix    | For [bodypix](https://github.com/de-code/python-tf-bodypix) filter
| webcam     | Virtual Webcam support via [pyfakewebcam](https://pypi.org/project/pyfakewebcam/)
| youtube    | YouTube support via [pafy](https://pypi.org/project/pafy/) and [youtube_dl](https://pypi.org/project/youtube_dl/)
| all        | All of the libraries

## Virtual Webcam For Linux

You do not need to use a webcam to use the project, as you could feed a video file.
But if you do want to use a webcam (currently only supported on Linux), this section provides a bit more information.

On Linux, `/dev/video0` often refers to the true webcam device.

You can use [v4l2loopback](https://github.com/umlaeute/v4l2loopback)
to create a virtual webcam device. e.g. you could `/dev/video2`.

Most applications looking for a webcam should then be able to use that virtual device.
(Applications might include Chromium, Skype etc.)

Once installed, you can create `/dev/video2` via the following command:

```bash
modprobe v4l2loopback devices=1 video_nr=2 exclusive_caps=1 card_label="VirtualCam 1"
```

To create the device after every reboot, you might want to create `/etc/modprobe.d/v4l2loopback.conf`:

```text
options v4l2loopback devices=1 video_nr=2 exclusive_caps=1 card_label="VirtualCam 1"
```

## Configuration

The configuration format is file is [YAML](https://en.wikipedia.org/wiki/YAML).

There are a number of [example configuration files](https://github.com/de-code/layered-vision/tree/develop/example-config).

### Layers

Every configuration file will contain layers. Layers are generally described from top to down.
With the last layer usually being the output layer.

The source to the output layer will be the layer above.

A very simple configuration file that downloads the `numpy` logo and saves it to a file might look like (`example-config/save-image.yml`):

```yaml
layers:
- id: in
  input_path: "https://github.com/numpy/numpy/raw/master/branding/logo/logomark/numpylogoicon.png"
- id: out
  output_path: "numpy-logo.png"
```

You could also have two outputs (`example-config/two-outputs.yml`):

```yaml
layers:
  - id: in
    input_path: "https://github.com/numpy/numpy/raw/master/branding/logo/logomark/numpylogoicon.png"
  - id: out1
    output_path: "data/numpy-logo1.png"
  - id: out2
    output_path: "data/numpy-logo2.png"
```

In that case the source layer for both `out1` and `out2` is `in`.

By using `window` as the `output_path`, the image is displayed in a window (`example-config/display-image.yml`):

```yaml
layers:
  - id: in
    input_path: "https://github.com/numpy/numpy/raw/master/branding/logo/logomark/numpylogoicon.png"
    width: 480
    height: 300
    repeat: true
  - id: out
    output_path: window
```

### Input Layer (Source)

A layer that has an `input_path` property.

The following inputs are currently supported:

| type name | description |
| -----| ----------- |
| image | Static image (e.g. `.png`) |
| video | Video (e.g. `.mp4`) |
| webcam | Linux Webcam (`/dev/videoN`) |
| youtube | YouTube stream (e.g. `https://youtu.be/yswkqEBio2k`,  see [example config](https://github.com/de-code/layered-vision/tree/develop/example-config/display-video-bodypix-replace-background-youtube.yml)) |
| mss | Screen capture using [mss](https://python-mss.readthedocs.io/index.html) (see [example config](https://github.com/de-code/layered-vision/tree/develop/example-config/display-video-bodypix-replace-background-mss.yml)) |

The `input_path` may point to a remote location (as is the case with [all examples](https://github.com/de-code/layered-vision/tree/develop/example-config)). In that case it will be downloaded and cached locally.

In most cases the *type name* is inferred from the `input_path`.
You can also specify the type explicitly via the `type` property or by prefixing the path, e.g.: `webcam:/dev/video0`.

### Output Layer (Sink)

A layer that has an `output_path` property.

The following outputs are currently supported:

| type name | description |
| -----| ----------- |
| image_writer | Write to a static image (e.g. `.png`) |
| v4l2 | Linux Virtual Webcam (`/dev/videoN`) |
| window | Display a window |

As is the case with the `input_path`, in most cases the *type name* is inferred from the `output_path`.
You can also specify the type explicitly via the `type` property or by prefixing the path, e.g.: `v4l2:/dev/video2`.

### Filter Layer

A layer that has a `filter` property.

The following filters are currently supported:

| name | description |
| -----| ----------- |
| `box_blur` | Blurs the image or channel. |
| `bodypix` | Uses the [bodypix](https://github.com/de-code/python-tf-bodypix) model to mask a person. |
| `chroma_key` | Uses a chroma key (colour) to add a mask |
| `copy` | Copies the input. Mainly useful as a placeholder layer with `branches`. |
| `dilate` | Dilates the image or channel. For example to increase the alpha mask after using `erode` |
| `erode` | Erodes the image or channel. That could be useful to remove outliers from an alpha mask. |
| `bilateral` | Applies a [bilateral filter](https://en.wikipedia.org/wiki/Bilateral_filter), using `d`, `sigma_color` and `sigma_space` parameters. |
| `motion_blur` | Adds a motion blur to the image or channel. That could be used to make an alpha mask move more slowly |
| `pixelate` | Pixelates the input. |

Every *filter* may have additional properties. Please refer to the [examples](https://github.com/de-code/layered-vision/tree/develop/example-config) (or come back in the future) for more detailed information.

### Branches Layer

A layer that has an `branches` property.
Each *branch* is required to have a `layers` property.
The input to each set of *branch layers* is the input to the *branches* layer.
The *branches* are then combined (added on top of each other).
To make *branches* useful, at least the last *branch image* should have an alpha mask.

## CLI

### CLI Help

```bash
python -m layered_vision --help
```

or

```bash
python -m layered_vision <sub command> --help
```

### Example Command

```bash
python -m layered_vision start --config-file=example-config/display-image.yml
```

You could also load the config from a remote location:

```bash
python -m layered_vision start --config-file \
  "https://raw.githubusercontent.com/de-code/layered-vision/develop/example-config/display-video-chroma-key-replace-background.yml"
```

It is also possible to override config values via command line arguments, e.g.:

```bash
python -m layered_vision start --config-file=example-config/display-image.yml \
    --set out.output_path=/path/to/output.png
```

You could also try replacing the background with a YouTube stream:

```bash
python -m layered_vision start \
  --config-file \
  "https://raw.githubusercontent.com/de-code/layered-vision/develop/example-config/webcam-bodypix-replace-background-to-v4l2loopback.yml" \
  --set bg.input_path="https://youtu.be/yswkqEBio2k" \
  --set bg.fps=30 \
  --set in.input_path="/dev/video0" \
  --set out.output_path="/dev/video2"
```

Note: you may need to specify the fps

### Docker Usage

You could also use the Docker image if you prefer.
The entrypoint will by default delegate to the CLI, except for `python` or `bash` commands.

```bash
docker pull de4code/layered-vision

docker run --rm \
  --device /dev/video0 \
  --device /dev/video2 \
  de4code/layered-vision start \
  --config-file \
  "https://raw.githubusercontent.com/de-code/layered-vision/develop/example-config/webcam-bodypix-replace-background-to-v4l2loopback.yml" \
  --set bg.input_path="https://www.dropbox.com/s/4debg4lrgn5g36l/toy-train-3288425.mp4?dl=1" \
  --set in.input_path="/dev/video0" \
  --set out.output_path="/dev/video2"
```

(Background: [Toy Train](https://www.pexels.com/video/toy-train-3288425/))

## Acknowledgements

* [virtual_webcam_background](https://github.com/allo-/virtual_webcam_background), a somewhat similar project (more focused on bodypix)
* [OBS Studio](https://obsproject.com/), conceptually a source of inspiration. (with UI etc)

