# Decentralized Data Parallel (Decent-DP)

<p align="center">
  <img src="./doc/source/_static/icon-light.png" alt="Icon created by zero_wing - Flaticon"/>
</p>

<p align="center">
    <a href="https://wangzesen.github.io/Decent-DP/">Documentation</a>
</p>


## Introduction

The repository is the official implementation of the paper **From Promise to Practice: Realizing High-performance Decentralized Training** ([arXiv version](https://arxiv.org/abs/2410.11998), [OpenReview](https://openreview.net/forum?id=lo3nlFHOft)) accepted by **ICLR 2025**.

The package is an PyTorch extension that faciliates efficient multi-worker decentralized data parallel training which fits in certain algorithm schemas.

## Documentation

For the full documentation, please refer to blabla

## Quickstart

### Installation

### Basic Usage

## Citation