Metadata-Version: 2.1
Name: nitec
Version: 0.0.2
Summary: The official PyTorch implementation of NITEC for eye contact detection.
Author: Thorsten Hempel, Magnus Jung, Ahmed Abdelrahman
Project-URL: homepath, https://github.com/thohemp/nitec
Project-URL: repository, https://github.com/thohemp/nitec
Keywords: eye contact,estimation,human-robot interact,hri,deep-learning,pytorch,engagement,wacv
Classifier: Programming Language :: Python :: 3
Requires-Python: >3.8
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: wheel
Requires-Dist: numpy >=1.19.5
Requires-Dist: opencv-python >=4.8.0.74
Requires-Dist: pandas >=2.0.3
Requires-Dist: Pillow >=8.4.0
Requires-Dist: torch >=2.0.1
Requires-Dist: torchvision >=0.15.2

# <div align="center"> **NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction (Accepted at WACV24)** </div>

<p align="center">
  <img src="https://github.com/thohemp/archive/blob/main/nitec.gif" alt="animated" />
</p>

## **Citing**

If you find our work useful, please cite the paper:

```BibTeX
@misc{hempel2023nitec,
      title={NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction}, 
      author={Thorsten Hempel and Magnus Jung and Ahmed A. Abdelrahman and Ayoub Al-Hamadi},
      year={2023},
      eprint={2311.04505},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
```
## <div align="center"> **Paper**</div>
> [Thorsten Hempel, Magnus Jung, Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction", *accepted at WACV 2024*.](https://arxiv.org/abs/2311.04505)

## <div align="center"> **Abstract**</div>
>Eye contact is a crucial non-verbal interaction modality and plays an important role in our everyday social life. While humans are very sensitive to eye contact, the capabilities of machines to capture a person's gaze are still mediocre. We tackle this challenge and present NITEC, a hand-annotated eye contact dataset for ego-vision interaction. NITEC exceeds existing datasets for ego-vision eye contact in size and variety of demographics, social contexts, and lighting conditions, making it a valuable resource for advancing ego-vision-based eye contact research. Our extensive evaluations on NITEC demonstrate strong cross-dataset performance, emphasizing its effectiveness and adaptability in various scenarios, that allows seamless utilization to the fields of computer vision, human-computer interaction, and social robotics. We make our NITEC dataset publicly available to foster reproducibility and further exploration in the field of ego-vision interaction.


#  <div align="center"> Quick Usage: </div>

```sh
pip install face_detection@git+https://github.com/elliottzheng/face-detection
pip install nitec
```

Example usage:

```py
from nitec import NITEC_Classifier, visualize
import cv2

nitec_pipeline = NITEC_Classifier(
    weights= CWD / 'models' / 'nitec_rs18_e20.pth',
    device=torch.device('cuda') # or 'cpu'
)

cap = cv2.VideoCapture(0)

_, frame = cap.read()    
# Process frame and visualize
results = nitec_pipeline.predict(frame)
frame = visualize(frame, results, confidence=0.5)

```



# <div align="center">  Train / Test </div>

## NITEC Dataset
Prepare the dataset as explained [ here](data/README.MD).

## Snapshots

Download from here: https://drive.google.com/drive/folders/1zc6NZZ6yA4NJ52Nn0bgky1XpZs9Z0hSJ?usp=sharing

## Train
```py
 python train.py \
 --gpu 0 \
 --num_epochs 50 \
 --batch_size 64 \
 --lr 0.0001 \
```


## Test

```py
 python test.py \
 --snapshot models/nitec_rs18_20.pth \
 --gpu 0 \
```

