# voxelpic

The `voxelpic` C library provides functionality to transcode point clouds
to and from RGB images. Starting with a sequence of colored point clouds like
the one below:

https://github.com/user-attachments/assets/de768bbf-b180-472a-873f-da10aa52e05a

The library builds an OcTree (to a specified depth) for each point cloud in
the sequence and then encodes a level of that OcTree to an RGB image, which
looks like this:

![Bird voxelpic](docs/bird_vp_thumb.png)

The octree is rendered from four orthographic cameras, and the system stores the
color of each voxel and its location in the octree along the axis of projection:

![Codec Layout](docs/codec_layout.png)

Resulting in a video like this one that encodes the above sequence:

https://github.com/user-attachments/assets/68bc1afb-3f37-49c5-b0ca-22594007c1e6

To decode this sequence back into a point cloud, the library decodes the image
into an OcTree level and then returns a point cloud consisting of voxel centroids
and voxel colors. In fact, the original animation above is the result of decoding
the voxelpic sequence shown.


> [!NOTE]
> **Encoding Depth as Color**
>
> The technique used to encode the depth as RGB color values has its origins in
> a codec developed by Intel for the RealSense camera. There is more information
> in the [Hue Codec Repository](https://github.com/jdtremaine/hue-codec/). We found
> that limiting the range of values encoded by this codec to the central 1273 values
> reduced ambiguity at decode time and resulted in much tighter error bars, especially
> after h.264 compression.

## Getting Started

To build this library, its tests, and tools, you will need
[CMake](https://cmake.org/) and a C compiler. The rests additionally require
Python to build the test files. The `vpic` tool further requires `libpng` if you
want to be able to read and write PNG images. Once you have the requirements in
place, you can build as normal, for example:

    mkdir build
    cd build
    cmake .. --preset release
    cmake --build .
    ctest

## `vpic` tool

In addition to the C library, we provide a [command-line tool](tools/vpic.c) that acts both as
a way to try out the library and as tutorial and example code on its usage.
You can build octrees from point cloud data, encode them as images, and decode
them back into octree levels. For example:

    ./vpic -m build -c sample_cloud.dat -v sample_voxels.dat -d 7
    Building an OcTree to depth 7 from 20000 points...done.
    Saving 17425 voxels to sample_voxels.dat...done.

    ./vpic -m encode -v sample_voxels.dat -i sample.png
    Loading voxels from sample_voxels.dat...done
    Encoding 17425 voxels at level 7 to sample.png...done.

    ./vpic -m decode -i sample.png -v sample_voxels_out.dat
    Decoding voxels from sample.png...done
    Saving 10666 voxels to sample_voxels_out.dat...done.

`sample_cloud.dat` is generated by [`cloud_gen.py`](tools/cloud_gen.py) and looks
like a spiral galaxy. The resulting image `sample.png` looks like this:

![Sample voxelpic](docs/sample.png)

Note that when decoded, we have lost some voxels. This is because any interior
voxels which are occluded from all four views cannot be encoded by this format
and are lost. `sample_cloud.dat` demonstrates the cloud data format, which is:

```
(size: int32 in network byte order (i.e. big-endian))
(x: float32, y: float32, z: float32, red: uint8, green: uint8, blue: uint8) x size
```

## `voxelpic` Python Library

We also provide a Python interface via the `voxelpic` Python library. To install
this library, run:

    pip install voxelpic

to get the latest released version or:

    pip install -e .

to build the library directly from a clone of this repository.

The library understands `numpy` arrays and can use them directly. To achieve the
same functionality as the vpic library above, we could use the following Python
script:

```python
import cv2
import voxelpic as vp

# NB all variables below are numpy arrays
sample_cloud = vp.PointCloud.load("sample_cloud.dat")
image = vp.encode(sample_cloud, depth=7)
image_bgr = cv2.cvtColor(image, cv2.COLOR_RGBA2BGRA)
cv2.imwrite("sample.png", image_bgr)
sample_cloud_out = vp.decode(image)
```

> **Trademarks** This project may contain trademarks or logos for projects, products, or services.
> Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft’s
> Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this
> project must not cause confusion or imply Microsoft sponsorship. Any use of third-party 
> trademarks or logos are subject to those third-party’s policies.
