Metadata-Version: 2.4
Name: d2nt
Version: 0.1.0
Summary: D2NT: A High-Performing Depth-to-Normal Translator
Author: Yi Feng, Bohuan Xue, Ming Liu, Qijun Chen, and Rui Fan
License: See LICENSE file
Project-URL: Homepage, https://mias.group/D2NT
Project-URL: Documentation, https://github.com/fengyi233/depth-to-normal-translator
Project-URL: Repository, https://github.com/fengyi233/depth-to-normal-translator
Project-URL: Bug Tracker, https://github.com/fengyi233/depth-to-normal-translator/issues
Keywords: depth,normal,computer-vision,robotics,3d-reconstruction
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20.0
Requires-Dist: opencv-python>=4.5.0
Provides-Extra: dev
Requires-Dist: matplotlib>=3.5.0; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Dynamic: license-file

# D2NT: A High-Performing Depth-to-Normal Translator

This repo is the official implementation of the paper:
> ["D2NT: A High-Performing Depth-to-Normal Translator"](https://arxiv.org/abs/2304.12031)

[//]: # (> by [Yi Feng]&#40;https://github.com/fengyi233&#41;, [Bohuan Xue]&#40;https://github.com/byronsit&#41;,)
[//]: # (> and [Rui Fan]&#40;https://ruirangerfan.com/&#41;.)

> [[arXiv]](https://arxiv.org/abs/2304.12031)
> [[homepage]](https://mias.group/D2NT)
> [[video]](https://www.bilibili.com/video/BV1GX4y1m7jF/)


<p align="center">
  <img src="assets/compare.gif" width="500" alt="trade-off"/>
</p>

<p align="center">
  <img src="assets/tradeoff.png" width="500" alt="trade-off"/>
</p>


# Installation

## Install from PyPI (Recommended)

```bash
pip install d2nt
```

## Install from Source

```bash
# Clone the repository
git clone https://github.com/fengyi233/depth-to-normal-translator.git
cd depth-to-normal-translator

# Install in development mode
pip install -e .

# Or install normally
pip install .
```

## Prerequisites

+ Python >= 3.7
+ numpy >= 1.20.0
+ opencv-python >= 4.5.0
+ matplotlib >= 3.5.0 (optional, for visualization)

# Dataset Preparation

Public real-world datasets generally obtain surface normals by local plane fitting,
which makes the surface normal ground truth unreliable. Therefore, we use the synthesis **3F2N dataset** provided
in this [paper](https://ieeexplore.ieee.org/document/9381580) to evaluate estimation performance.

The 3F2N dataset can be downloaded from: \
[BaiduDisk](https://pan.baidu.com/s/1BrzulgKfCCf-69mS-JSgNA?pwd=jveo) \
[GoogleDrive](https://drive.google.com/drive/folders/1TLj033oV0aplLE6xcQaSRcZpggDVTCHY) \
The dataset is organized as follows:

```
3F2N
 |-- Easy
 |  |-- android
 |  |  |-- depth
 |  |  |-- normal
 |  |  |-- params.txt
 |  |  |-- pose.txt
 |  |-- cube
 |  |-- ...
 |  |-- torusknot
 |-- Medium
 |  |-- ...
 |-- Hard
 |  |-- ...
```

# Usage

## Python Package Usage

After installation, you can use the `depth2normal()` function directly:

```python
import numpy as np
from d2nt import depth2normal

# Prepare depth map (example)
depth = np.random.rand(480, 640) * 10.0

# Camera intrinsic matrix
# Format: [[fx,  0, u0],
#          [ 0, fy, v0],
#          [ 0,  0,  1]]
cam_intrinsic = np.array([
    [525.0, 0, 320.0],  # fx=525.0, u0=320.0
    [0, 525.0, 240.0],  # fy=525.0, v0=240.0
    [0, 0, 1]
])

# Convert depth to normal
normal = depth2normal(depth, cam_intrinsic, version='d2nt_v3')

print(f"Normal map shape: {normal.shape}")  # (480, 640, 3)
```

### Algorithm Versions

- **`d2nt_basic`**: Basic version without any optimization method
- **`d2nt_v2`**: With Discontinuity-Aware Gradient (DAG) filter
- **`d2nt_v3`**: With DAG filter and MRF-based Normal Refinement (MNR) module (recommended)

## Python Demo

Navigate to `python/` directory and run `demo.py`, a result and the corresponding error map (degrees) will be displayed.
We also implement [3F2N SNE](https://ieeexplore.ieee.org/document/9381580) in python. The matlab and c++ implementation
can be found in this [repository](https://github.com/ruirangerfan/Three-Filters-to-Normal). 

It is recommended to run '**demo.py**' in your Python Console instead of the Terminal for the sake of visualization.
You can change the parameter 'VERSION' to select the D2NT version.\
'**d2nt_basic**' represents for the depth-to-normal translator without any optimization method.\
'**d2nt_v2**' represents for the D2NT with Discontinuity-Aware Gradient (DAG) filter.\
'**d2nt_v3**' represents for the D2NT with DAG filter and MRF-based Normal Refinement (MNR) module.


# Cite
This code is for non-commercial use. If you find our work useful in your research, please consider citing our paper:

```
@inproceedings{icra_2023_d2nt,
	author      = {{Yi Feng, Bohuan Xue, Ming Liu, Qijun Chen, and Rui Fan}},
	title       = {{D2NT: A High-Performing Depth-to-Normal Translator}},
	booktitle   = {{IEEE International Conference on Robotics and Automation (ICRA)}},
	year        = {{2023}}
}
```
