Metadata-Version: 2.4
Name: torch-ttnn
Version: 0.0.1.dev1
Summary: PyTorch 2.0 TTNN Compiler
Home-page: https://github.com/tenstorrent/pytorch2.0_ttnn
Author: Tenstorrent
Author-email: Tenstorrent <info@tenstorrent.com>
License: Apache-2.0
Project-URL: Repository, https://github.com/tenstorrent/pytorch2.0_ttnn
Keywords: pytorch,ttnn,tenstorrent,compiler,machine-learning,deep-learning
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: POSIX :: Linux
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch==2.2.1+cpu
Requires-Dist: torchvision==0.17.1+cpu
Requires-Dist: tabulate==0.9.0
Requires-Dist: networkx==3.1
Requires-Dist: graphviz
Requires-Dist: matplotlib==3.7.1
Requires-Dist: paramiko==3.5.1
Provides-Extra: ttnn
Requires-Dist: ttnn==0.59.0; extra == "ttnn"
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: black>=23.7.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"
Requires-Dist: flake8>=6.1.0; extra == "dev"
Requires-Dist: mypy>=1.5.0; extra == "dev"
Requires-Dist: pre-commit>=3.3.3; extra == "dev"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python

[comment]: <> (This README.md was generated by tools/collect_metrics.py.)
[comment]: <> (Please modify docs/README.md.in and/or collect_metrics.py to make permanent changes.)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/tenstorrent/pytorch2.0_ttnn)

# PyTorch 2.0 TTNN Compiler
The PyTorch 2.0 TT-NN Compiler enables seamless execution of PyTorch models on [Tenstorrent](https://tenstorrent.com/) AI accelerators. 
By leveraging the TT-NN backend, you can achieve significant performance improvements while maintaining PyTorch's familiar API.

## 🚀 Quick Start

### Installation

### Requirements
- Python 3.10
- Ubuntu operating system
- PyTorch 2.0 or later

### Basic Installation
```bash
pip install torch-ttnn
```

### Installation with TTNN Support
To use the TTNN backend, you need to install the additional `ttnn` package:
```bash
pip install torch-ttnn[ttnn]
```

Note: The `ttnn` package is only supported on Ubuntu with Python 3.10. If you're using a different platform or Python version, you can still install `torch-ttnn` without the TTNN backend.

### ✨ Basic Usage

**Option 1: Eager Mode:** get your model running by switching to a TT device
```python
import torch
import torch_ttnn

model = YourModel()

device = ttnn.open_device(device_id=0)
model.to(torch_ttnn.ttnn_device_as_torch_device(device))

output = model(input_data)
```

**Option 2: Compilation Mode (Recommended):** get more perf with a JIT compiler
```python
import torch
import torch_ttnn

model = YourModel()

device = ttnn.open_mesh_device(ttnn.MeshShape(1, 2))  # 1x2 device grid
option = torch_ttnn.TorchTtnnOption(device=device, data_parallel=2)

model = torch.compile(model, backend=torch_ttnn.backend, options=option)
output = model(input_data)
```

## 📊 Model Support

We've extensively tested the compiler across a diverse range of model architectures. Here's a summary of our validation results:

| Model                                                                                                | Status   | Batch   | Compiled First Run (ms)   |   Original Throughput (Inferences Per Second) | Compiled Throughput (Inferences Per Second)   | Accuracy (%)   | Torch Ops Before (Unique Ops)   | Torch Ops Remain (Unique Ops)   | To/From Device Ops   |
|:-----------------------------------------------------------------------------------------------------|:---------|:--------|:--------------------------|----------------------------------------------:|:----------------------------------------------|:---------------|:--------------------------------|:--------------------------------|:---------------------|
| [Autoencoder (linear)](<docs/models/Autoencoder (linear)>)                                           | ✅       | 1       | 383.42                    |                                   0.811959    | 571.4285714285714                             | 100.0          | 22 (3)                          | 0 (0)                           | 0                    |
| [BERT](<docs/models/BERT>)                                                                           | ✅       | 8       | 44356.06                  |                                   0.0106441   | 41.62330905306972                             | 99.69          | 1465 (22)                       | 0 (0)                           | 0                    |
| [DPR](<docs/models/DPR>)                                                                             | ✅       | 1       | 17562.24                  |                                   0.353536    | 77.33952049497293                             | 99.38          | 720 (22)                        | 0 (0)                           | 1                    |
| [HardNet](<docs/models/HardNet>)                                                                     | ✅       | 1       | 145519.22                 |                                   0.189912    | 22.461814914645103                            | 98.39          | 245 (10)                        | 0 (0)                           | 124                  |
| [MLPMixer](<docs/models/MLPMixer>)                                                                   | ✅       | 1       | 19806.28                  |                                   0.182941    | 78.86435331230284                             | 99.99          | 253 (11)                        | 0 (0)                           | 0                    |
| [Mnist](<docs/models/Mnist>)                                                                         | ✅       | 1       | 5716.13                   |                                  25.7666      | 423.7288135593221                             | 99.42          | 14 (8)                          | 0 (0)                           | 1                    |
| [MobileNetV2](<docs/models/MobileNetV2>)                                                             | ✅       | 1       | 99053.15                  |                                   1.07832     | 39.447731755424066                            | 99.09          | 154 (9)                         | 0 (0)                           | 0                    |
| [OpenPose V2](<docs/models/OpenPose V2>)                                                             | ✅       | 1       | 18350.21                  |                                   0.317068    | 30.82614056720099                             | 91.49          | 155 (7)                         | 0 (0)                           | 6                    |
| [Perceiver IO](<docs/models/Perceiver IO>)                                                           | ✅       | 1       | 49339.44                  |                                   0.020488    | 18.765246762994934                            | 99.95          | 1531 (20)                       | 0 (0)                           | 1                    |
| [ResNet18](<docs/models/ResNet18>)                                                                   | ✅       | 1       | 43653.35                  |                                   0.478941    | 90.82652134423252                             | 98.86          | 70 (9)                          | 0 (0)                           | 1                    |
| [ResNet50](<docs/models/ResNet50>)                                                                   | ✅       | 4       | 68551.31                  |                                   0.887709    | 46.52244708071644                             | 98.61          | 176 (9)                         | 0 (0)                           | 1                    |
| [RoBERTa](<docs/models/RoBERTa>)                                                                     | ✅       | 1       | 32614.22                  |                                   0.0806585   | 19.03311762466692                             | 28.56          | 719 (21)                        | 0 (0)                           | 3                    |
| [U-Net](<docs/models/U-Net>)                                                                         | ✅       | 1       | 67494.19                  |                                   0.0154445   | 74.12898443291327                             | 100.0          | 68 (6)                          | 0 (0)                           | 12                   |
| [Unet-brain](<docs/models/Unet-brain>)                                                               | ✅       | 1       | 3103.27                   |                                   0.0153888   | 77.39938080495357                             | N/A            | 68 (6)                          | 0 (0)                           | 12                   |
| [Unet-carvana](<docs/models/Unet-carvana>)                                                           | ✅       | 1       | 29599.48                  |                                   0.0115197   | 26.990553306342782                            | 99.69          | 67 (5)                          | 0 (0)                           | 12                   |
| [albert/albert-base-v2](<docs/models/albert/albert-base-v2>)                                         | ✅       | 1       | 30146.69                  |                                   0.700324    | 41.373603640877114                            | 98.82          | 791 (21)                        | 0 (0)                           | 3                    |
| [albert/albert-base-v2-classification](<docs/models/albert/albert-base-v2-classification>)           | ✅       | 1       | 9718.05                   |                                   0.708436    | 45.28985507246377                             | 99.97          | 779 (21)                        | 0 (0)                           | 2                    |
| [albert/albert-large-v2](<docs/models/albert/albert-large-v2>)                                       | ✅       | 1       | 24344.19                  |                                   0.296342    | 24.36647173489279                             | 98.95          | 1547 (21)                       | 0 (0)                           | 3                    |
| [albert/albert-xlarge-v2](<docs/models/albert/albert-xlarge-v2>)                                     | ✅       | 1       | 40934.46                  |                                   0.112899    | 13.089005235602093                            | 97.36          | 1547 (21)                       | 0 (0)                           | 3                    |
| [densenet121](<docs/models/densenet121>)                                                             | ✅       | 1       | 128892.59                 |                                   0.283193    | 14.126289023873428                            | 99.74          | 432 (10)                        | 0 (0)                           | 597                  |
| [densenet161](<docs/models/densenet161>)                                                             | ✅       | 1       | 175834.91                 |                                   0.119096    | 9.562972171750982                             | 99.49          | 572 (10)                        | 0 (0)                           | 1147                 |
| [densenet169](<docs/models/densenet169>)                                                             | ✅       | 1       | 234407.58                 |                                   0.228034    | 7.477753682793689                             | 99.58          | 600 (10)                        | 0 (0)                           | 1241                 |
| [densenet201](<docs/models/densenet201>)                                                             | ✅       | 1       | 85614.51                  |                                   0.180097    | 7.341065922771986                             | 99.39          | 712 (10)                        | 0 (0)                           | 1905                 |
| [distilbert-base-uncased](<docs/models/distilbert-base-uncased>)                                     | ✅       | 1       | 23799.27                  |                                   0.743494    | 95.69377990430623                             | 72.37          | 361 (16)                        | 0 (0)                           | 1                    |
| [dla34.in1k](<docs/models/dla34.in1k>)                                                               | ✅       | 1       | 78008.86                  |                                   0.249834    | 46.14674665436087                             | 99.5           | 135 (9)                         | 0 (0)                           | 23                   |
| [ese_vovnet19b_dw.ra_in1k](<docs/models/ese_vovnet19b_dw.ra_in1k>)                                   | ✅       | 1       | 68013.14                  |                                   0.588183    | 54.31830526887561                             | 99.44          | 111 (12)                        | 0 (0)                           | 19                   |
| [ghostnet_100.in1k](<docs/models/ghostnet_100.in1k>)                                                 | ✅       | 1       | 169552.06                 |                                   1.00715     | 17.77777777777778                             | 99.6           | 515 (14)                        | 0 (0)                           | 64                   |
| [mobilenet_v2](<docs/models/mobilenet_v2>)                                                           | ✅       | 1       | 76682.69                  |                                   1.0369      | 33.898305084745765                            | 99.09          | 154 (9)                         | 0 (0)                           | 0                    |
| [mobilenet_v3_large](<docs/models/mobilenet_v3_large>)                                               | ✅       | 1       | 95479.35                  |                                   1.21957     | 26.553372278279344                            | 99.15          | 188 (11)                        | 0 (0)                           | 0                    |
| [mobilenet_v3_small](<docs/models/mobilenet_v3_small>)                                               | ✅       | 1       | 127515.98                 |                                   1.98389     | 37.993920972644375                            | 99.09          | 158 (11)                        | 0 (0)                           | 0                    |
| [mobilenetv1_100.ra4_e3600_r224_in1k](<docs/models/mobilenetv1_100.ra4_e3600_r224_in1k>)             | ✅       | 1       | 70441.67                  |                                   1.05934     | 62.26650062266501                             | 96.04          | 85 (7)                          | 0 (0)                           | 0                    |
| [regnet_x_16gf](<docs/models/regnet_x_16gf>)                                                         | ✅       | 1       | 58745.32                  |                                   0.0666041   | 13.711778417660769                            | 99.59          | 235 (8)                         | 0 (0)                           | 0                    |
| [regnet_x_1_6gf](<docs/models/regnet_x_1_6gf>)                                                       | ✅       | 1       | 49694.01                  |                                   0.526227    | 34.55425017277125                             | 99.47          | 195 (8)                         | 0 (0)                           | 0                    |
| [regnet_x_32gf](<docs/models/regnet_x_32gf>)                                                         | ✅       | 1       | 97323.87                  |                                   0.0340421   | 7.972574344255759                             | 99.27          | 245 (8)                         | 0 (0)                           | 0                    |
| [regnet_x_3_2gf](<docs/models/regnet_x_3_2gf>)                                                       | ✅       | 1       | 80719.6                   |                                   0.292484    | 26.5041081367612                              | 99.5           | 265 (8)                         | 0 (0)                           | 0                    |
| [regnet_x_400mf](<docs/models/regnet_x_400mf>)                                                       | ✅       | 1       | 72663.59                  |                                   0.537201    | 30.703101013202332                            | 99.66          | 235 (8)                         | 0 (0)                           | 0                    |
| [regnet_x_800mf](<docs/models/regnet_x_800mf>)                                                       | ✅       | 1       | 47396.85                  |                                   0.860896    | 38.97116134060795                             | 99.44          | 175 (8)                         | 0 (0)                           | 0                    |
| [regnet_x_8gf](<docs/models/regnet_x_8gf>)                                                           | ✅       | 1       | 52436.51                  |                                   0.13043     | 19.70831690973591                             | 98.99          | 245 (8)                         | 0 (0)                           | 0                    |
| [regnet_y_16gf](<docs/models/regnet_y_16gf>)                                                         | ✅       | 1       | 133312.64                 |                                   0.0647662   | 11.477103179157583                            | 99.66          | 303 (10)                        | 0 (0)                           | 0                    |
| [regnet_y_1_6gf](<docs/models/regnet_y_1_6gf>)                                                       | ✅       | 1       | 117495.85                 |                                   0.508282    | 17.857142857142858                            | 99.69          | 447 (10)                        | 0 (0)                           | 0                    |
| [regnet_y_32gf](<docs/models/regnet_y_32gf>)                                                         | ✅       | 1       | 117099.36                 |                                   0.0305392   | 7.944704854214667                             | 99.72          | 335 (10)                        | 0 (0)                           | 0                    |
| [regnet_y_3_2gf](<docs/models/regnet_y_3_2gf>)                                                       | ✅       | 1       | 82998.34                  |                                   0.284312    | 20.466639377814165                            | 99.82          | 351 (10)                        | 0 (0)                           | 0                    |
| [regnet_y_400mf](<docs/models/regnet_y_400mf>)                                                       | ✅       | 1       | 90614.31                  |                                   1.16836     | 27.495188342040148                            | 99.64          | 271 (10)                        | 0 (0)                           | 0                    |
| [regnet_y_800mf](<docs/models/regnet_y_800mf>)                                                       | ✅       | 1       | 63349.07                  |                                   0.758196    | 32.2268772155978                              | 99.59          | 239 (10)                        | 0 (0)                           | 0                    |
| [regnet_y_8gf](<docs/models/regnet_y_8gf>)                                                           | ✅       | 1       | 76950.55                  |                                   0.116881    | 17.41250217656277                             | 99.83          | 287 (10)                        | 0 (0)                           | 0                    |
| [resnet101](<docs/models/resnet101>)                                                                 | ✅       | 1       | 16482.4                   |                                   0.129942    | 21.645021645021643                            | 99.25          | 346 (9)                         | 0 (0)                           | 1                    |
| [resnet152](<docs/models/resnet152>)                                                                 | ✅       | 1       | 89185.59                  |                                   0.074999    | 13.766519823788546                            | 99.09          | 516 (9)                         | 0 (0)                           | 1                    |
| [resnet18](<docs/models/resnet18>)                                                                   | ✅       | 1       | 25158.48                  |                                   0.378949    | 85.61643835616438                             | 99.61          | 70 (9)                          | 0 (0)                           | 1                    |
| [resnet34](<docs/models/resnet34>)                                                                   | ✅       | 1       | 46602.1                   |                                   0.239212    | 52.71481286241434                             | 99.07          | 126 (9)                         | 0 (0)                           | 1                    |
| [resnet50](<docs/models/resnet50>)                                                                   | ✅       | 1       | 46720.36                  |                                   0.224649    | 40.19292604501608                             | 98.61          | 176 (9)                         | 0 (0)                           | 1                    |
| [resnext101_32x8d](<docs/models/resnext101_32x8d>)                                                   | ✅       | 1       | 51962.12                  |                                   0.052611    | 8.340283569641368                             | 99.57          | 346 (9)                         | 0 (0)                           | 1                    |
| [resnext101_64x4d](<docs/models/resnext101_64x4d>)                                                   | ✅       | 1       | 90273.59                  |                                   0.066646    | 8.367500627562547                             | 99.65          | 346 (9)                         | 0 (0)                           | 1                    |
| [resnext50_32x4d](<docs/models/resnext50_32x4d>)                                                     | ✅       | 1       | 32675.8                   |                                   0.192436    | 33.80662609871535                             | 99.44          | 176 (9)                         | 0 (0)                           | 1                    |
| [textattack/albert-base-v2-imdb](<docs/models/textattack/albert-base-v2-imdb>)                       | ✅       | 1       | 33301.7                   |                                   0.768096    | 48.1000481000481                              | 100.0          | 782 (22)                        | 0 (0)                           | 2                    |
| [tf_efficientnet_lite0.in1k](<docs/models/tf_efficientnet_lite0.in1k>)                               | ✅       | 1       | 107808.07                 |                                   1.03156     | 26.79528403001072                             | 99.3           | 149 (9)                         | 0 (0)                           | 5                    |
| [tf_efficientnet_lite1.in1k](<docs/models/tf_efficientnet_lite1.in1k>)                               | ✅       | 1       | 70764.58                  |                                   0.8064      | 19.70831690973591                             | 99.56          | 194 (9)                         | 0 (0)                           | 5                    |
| [tf_efficientnet_lite2.in1k](<docs/models/tf_efficientnet_lite2.in1k>)                               | ✅       | 1       | 133347.35                 |                                   0.481       | 13.05142260506395                             | 99.21          | 194 (9)                         | 0 (0)                           | 5                    |
| [twmkn9/albert-base-v2-squad2](<docs/models/twmkn9/albert-base-v2-squad2>)                           | ✅       | 1       | 20964.66                  |                                   0.68519     | 51.786639047125846                            | 99.86          | 783 (23)                        | 0 (0)                           | 2                    |
| [vgg11](<docs/models/vgg11>)                                                                         | ✅       | 1       | 35162.28                  |                                   0.0825115   | 80.12820512820512                             | 99.65          | 33 (8)                          | 0 (0)                           | 5                    |
| [vgg11_bn](<docs/models/vgg11_bn>)                                                                   | ✅       | 1       | 2790.29                   |                                   0.0819141   | 74.07407407407408                             | 98.99          | 41 (9)                          | 0 (0)                           | 5                    |
| [vgg13](<docs/models/vgg13>)                                                                         | ✅       | 1       | 52950.64                  |                                   0.0477291   | 70.87172218284904                             | 99.38          | 37 (8)                          | 0 (0)                           | 5                    |
| [vgg13_bn](<docs/models/vgg13_bn>)                                                                   | ✅       | 1       | 6428.33                   |                                   0.0506591   | 62.5782227784731                              | 97.35          | 47 (9)                          | 0 (0)                           | 5                    |
| [vgg16](<docs/models/vgg16>)                                                                         | ✅       | 1       | 2414.84                   |                                   0.0364174   | 54.6448087431694                              | 99.44          | 43 (8)                          | 0 (0)                           | 5                    |
| [vgg16_bn](<docs/models/vgg16_bn>)                                                                   | ✅       | 1       | 58603.66                  |                                   0.0377755   | 48.99559039686428                             | 98.38          | 56 (9)                          | 0 (0)                           | 5                    |
| [vgg19](<docs/models/vgg19>)                                                                         | ✅       | 1       | 2734.68                   |                                   0.0313835   | 43.917435221783045                            | 99.27          | 49 (8)                          | 0 (0)                           | 5                    |
| [vgg19_bn](<docs/models/vgg19_bn>)                                                                   | ✅       | 1       | 4644.62                   |                                   0.0310762   | 40.16064257028113                             | 96.97          | 65 (9)                          | 0 (0)                           | 5                    |
| [wide_resnet101_2](<docs/models/wide_resnet101_2>)                                                   | ✅       | 1       | 89228.41                  |                                   0.0362154   | 16.818028927009756                            | 99.2           | 346 (9)                         | 0 (0)                           | 1                    |
| [wide_resnet50_2](<docs/models/wide_resnet50_2>)                                                     | ✅       | 1       | 71856.29                  |                                   0.0821048   | 34.376074252320386                            | 98.8           | 176 (9)                         | 0 (0)                           | 1                    |
| [xception71.tf_in1k](<docs/models/xception71.tf_in1k>)                                               | ✅       | 1       | 129869.68                 |                                   0.0474214   | 4.1118421052631575                            | 99.21          | 393 (9)                         | 0 (0)                           | 0                    |
| [Autoencoder (conv)](<docs/models/Autoencoder (conv)>)                                               | 🚧       | 1       | 4340.96                   |                                   0.748733    | 258.39793281653743                            | 100.0          | 9 (3)                           | 1 (1)                           | 1                    |
| [Autoencoder (conv)-train](<docs/models/Autoencoder (conv)-train>)                                   | 🚧       | 1       | 14269.54                  |                                   0.515631    | 142.45014245014247                            | 100.0          | 24 (7)                          | 11 (4)                          | 0                    |
| [Autoencoder (linear)-train](<docs/models/Autoencoder (linear)-train>)                               | 🚧       | 1       | 12046.67                  |                                   0.565985    | 59.630292188431724                            | 100.0          | 104 (8)                         | 14 (2)                          | 0                    |
| [Bloom](<docs/models/Bloom>)                                                                         | 🚧       | 1       | 44135.12                  |                                   0.0320436   | 0.9556757583287142                            | 98.86          | 1405 (27)                       | 2 (2)                           | 0                    |
| [CLIP](<docs/models/CLIP>)                                                                           | 🚧       | 1       | 52784.19                  |                                   0.168796    | 5.765350245027387                             | 99.56          | 1397 (30)                       | 7 (6)                           | 2                    |
| [CLIP-train](<docs/models/CLIP-train>)                                                               | 🚧       | 1       | 82372.28                  |                                   0.0440882   | 0.5801540889260187                            | 100.0          | 3944 (44)                       | 265 (16)                        | 5                    |
| [DETR](<docs/models/DETR>)                                                                           | 🚧       | 1       | 144337.54                 |                                   0.00897003  | 0.18551292468546285                           | 45.81          | 1663 (42)                       | 9 (6)                           | 3                    |
| [DINOv2](<docs/models/DINOv2>)                                                                       | 🚧       | 1       | 29998.75                  |                                   0.0527538   | 14.925373134328359                            | 98.99          | 928 (25)                        | 16 (1)                          | 2                    |
| [GLPN-KITTI](<docs/models/GLPN-KITTI>)                                                               | 🚧       | 1       | 253614.99                 |                                   0.0074878   | 0.016908800692584474                          | 99.75          | 2959 (26)                       | 22 (2)                          | 6                    |
| [GPT-2](<docs/models/GPT-2>)                                                                         | 🚧       | 1       | 26621.82                  |                                   0.418053    | 31.347962382445147                            | 99.98          | 745 (29)                        | 2 (2)                           | 2                    |
| [HardNet-train](<docs/models/HardNet-train>)                                                         | 🚧       | 1       | 150265.12                 |                                   0.074868    | 0.10989240434690395                           | 100.0          | 867 (21)                        | 412 (9)                         | 120                  |
| [MLPMixer-train](<docs/models/MLPMixer-train>)                                                       | 🚧       | 1       | 37040.61                  |                                   0.057748    | 0.10531426302658448                           | 100.0          | 616 (19)                        | 100 (5)                         | 0                    |
| [Mnist-train](<docs/models/Mnist-train>)                                                             | 🚧       | 1       | 17129.5                   |                                   0.392026    | 25.833118057349523                            | 100.0          | 46 (15)                         | 10 (6)                          | 0                    |
| [MobileNetSSD](<docs/models/MobileNetSSD>)                                                           | 🚧       | 1       | 220974.42                 |                                   1.62914     | 0.4929435136027762                            | 43.63          | 522 (31)                        | 7 (4)                           | 32                   |
| [OpenPose V2-train](<docs/models/OpenPose V2-train>)                                                 | 🚧       | 1       | 71154.03                  |                                   0.0937387   | 0.12389439737145645                           | 100.0          | 523 (14)                        | 246 (7)                         | 6                    |
| [ResNet18-train](<docs/models/ResNet18-train>)                                                       | 🚧       | 1       | 40024.73                  |                                   0.169332    | 0.23849272597185786                           | 100.0          | 241 (19)                        | 121 (9)                         | 0                    |
| [ResNet50-train](<docs/models/ResNet50-train>)                                                       | 🚧       | 1       | 64086.18                  |                                   0.0668139   | 0.09140700982076913                           | 100.0          | 616 (19)                        | 318 (9)                         | 0                    |
| [SegFormer](<docs/models/SegFormer>)                                                                 | 🚧       | 1       | 26205.58                  |                                   0.0251613   | 2.97645623121112                              | 99.85          | 676 (22)                        | 16 (1)                          | 4                    |
| [SegFormer-train](<docs/models/SegFormer-train>)                                                     | 🚧       | 1       | 161612.04                 |                                   0.0128654   | 0.02964470231382831                           | 100.0          | 1794 (36)                       | 156 (12)                        | 4                    |
| [U-Net-train](<docs/models/U-Net-train>)                                                             | 🚧       | 1       | 105412.29                 |                                   0.00857157  | 0.01921014049528353                           | 100.0          | 236 (15)                        | 122 (8)                         | 8                    |
| [Unet-brain-train](<docs/models/Unet-brain-train>)                                                   | 🚧       | 1       | 59323.37                  |                                   0.00888899  | 0.02151077485467858                           | 100.0          | 236 (15)                        | 122 (8)                         | 8                    |
| [Unet-carvana-train](<docs/models/Unet-carvana-train>)                                               | 🚧       | 1       | 141510.2                  |                                   0.00582711  | 0.011355714462376758                          | 100.0          | 232 (13)                        | 121 (7)                         | 8                    |
| [YOLOS](<docs/models/YOLOS>)                                                                         | 🚧       | 1       | 37286.81                  |                                   0.0689129   | 3.597251699701428                             | 98.46          | 952 (27)                        | 17 (2)                          | 2                    |
| [YOLOv3](<docs/models/YOLOv3>)                                                                       | 🚧       | 1       | 71688.97                  |                                   0.00513066  | 17.31301939058172                             | 98.63          | 250 (7)                         | 2 (1)                           | 4                    |
| [albert/albert-xxlarge-v2](<docs/models/albert/albert-xxlarge-v2>)                                   | 🚧       | 1       | 17987.37                  |                                   0.0566237   | 7.282260413632391                             | 98.54          | 791 (21)                        | 24 (1)                          | 3                    |
| [dla34.in1k-train](<docs/models/dla34.in1k-train>)                                                   | 🚧       | 1       | 52239.74                  |                                   0.101584    | 0.14390021383571774                           | 100.0          | 469 (18)                        | 230 (8)                         | 17                   |
| [ese_vovnet19b_dw.ra_in1k-train](<docs/models/ese_vovnet19b_dw.ra_in1k-train>)                       | 🚧       | 1       | 100480.17                 |                                   0.206964    | 0.2903389707483487                            | 100.0          | 383 (25)                        | 176 (10)                        | 16                   |
| [facebook/deit-base-patch16-224](<docs/models/facebook/deit-base-patch16-224>)                       | 🚧       | 1       | 25305.31                  |                                   0.0674106   | 8.9349535382416                               | 98.34          | 685 (17)                        | 1 (1)                           | 1                    |
| [facebook/deit-base-patch16-224-train](<docs/models/facebook/deit-base-patch16-224-train>)           | 🚧       | 1       | 33755.02                  |                                   0.013853    | 0.7183856437812947                            | 100.0          | 1854 (27)                       | 127 (8)                         | 2                    |
| [ghostnet_100.in1k-train](<docs/models/ghostnet_100.in1k-train>)                                     | 🚧       | 1       | 171132.06                 |                                   0.803923    | 0.4899270988476915                            | 100.0          | 1469 (33)                       | 562 (12)                        | 64                   |
| [ghostnetv2_100.in1k](<docs/models/ghostnetv2_100.in1k>)                                             | 🚧       | 1       | 74364.28                  |                                   1.00102     | 8.680555555555555                             | 99.65          | 683 (18)                        | 24 (2)                          | 68                   |
| [ghostnetv2_100.in1k-train](<docs/models/ghostnetv2_100.in1k-train>)                                 | 🚧       | 1       | 83330.89                  |                                   0.368118    | 0.23244570068432016                           | 100.0          | 2001 (39)                       | 852 (17)                        | 68                   |
| [googlenet](<docs/models/googlenet>)                                                                 | 🚧       | 1       | 131641.04                 |                                   0.537377    | 26.53927813163482                             | 99.67          | 214 (15)                        | 1 (1)                           | 51                   |
| [hrnet_w18.ms_aug_in1k](<docs/models/hrnet_w18.ms_aug_in1k>)                                         | 🚧       | 1       | 131463.73                 |                                   0.168589    | 5.218117303276978                             | 99.61          | 1209 (11)                       | 31 (1)                          | 0                    |
| [hrnet_w18.ms_aug_in1k-train](<docs/models/hrnet_w18.ms_aug_in1k-train>)                             | 🚧       | 1       | 203796.57                 |                                   0.0706861   | 0.09821176026902166                           | 100.0          | 3998 (21)                       | 1973 (9)                        | 0                    |
| [inception_v4.tf_in1k](<docs/models/inception_v4.tf_in1k>)                                           | 🚧       | 1       | 122077.71                 |                                   0.070889    | 5.862695667467903                             | 99.11          | 495 (11)                        | 14 (1)                          | 84                   |
| [inception_v4.tf_in1k-train](<docs/models/inception_v4.tf_in1k-train>)                               | 🚧       | 1       | 176051.36                 |                                   0.0226193   | 0.03144037144912445                           | 100.0          | 1851 (24)                       | 932 (11)                        | 80                   |
| [mixer_b16_224.goog_in21k](<docs/models/mixer_b16_224.goog_in21k>)                                   | 🚧       | 1       | 17663.49                  |                                   0.0852769   | 9.703085581214827                             | 3.65           | 356 (11)                        | 1 (1)                           | 0                    |
| [mixer_b16_224.goog_in21k-train](<docs/models/mixer_b16_224.goog_in21k-train>)                       | 🚧       | 1       | 32424.28                  |                                   0.0172035   | 0.6438818348056765                            | 100.0          | 959 (18)                        | 101 (6)                         | 0                    |
| [mobilenetv1_100.ra4_e3600_r224_in1k-train](<docs/models/mobilenetv1_100.ra4_e3600_r224_in1k-train>) | 🚧       | 1       | 55225.13                  |                                   0.34785     | 0.3428920198740215                            | 100.0          | 258 (16)                        | 164 (7)                         | 0                    |
| [regnet_y_128gf](<docs/models/regnet_y_128gf>)                                                       | 🚧       | 1       | 331195.99                 |                                   0.00182639  | 0.01674862606833205                           | 98.91          | 447 (10)                        | 3 (1)                           | 0                    |
| [ssd300_vgg16](<docs/models/ssd300_vgg16>)                                                           | 🚧       | 1       | 149011.7                  |                                   0.26909     | 0.597575040485709                             | N/A            | 332 (30)                        | 8 (5)                           | 37                   |
| [ssdlite320_mobilenet_v3_large](<docs/models/ssdlite320_mobilenet_v3_large>)                         | 🚧       | 1       | 176237.75                 |                                   1.74064     | 0.4501807475701495                            | 41.24          | 522 (31)                        | 7 (4)                           | 32                   |
| [swin_b](<docs/models/swin_b>)                                                                       | 🚧       | 1       | 136701.0                  |                                   0.0695169   | 3.745178083217857                             | 91.49          | 2492 (32)                       | 110 (2)                         | 479                  |
| [swin_s](<docs/models/swin_s>)                                                                       | 🚧       | 1       | 29441.2                   |                                   0.121467    | 4.093327875562832                             | 91.4           | 2492 (32)                       | 110 (2)                         | 479                  |
| [swin_t](<docs/models/swin_t>)                                                                       | 🚧       | 1       | 114700.87                 |                                   0.222576    | 7.809449433814915                             | 97.21          | 1238 (32)                       | 50 (2)                          | 227                  |
| [swin_v2_b](<docs/models/swin_v2_b>)                                                                 | 🚧       | 1       | 147201.22                 |                                   0.0428878   | 3.017046311660884                             | 24.15          | 3140 (40)                       | 158 (3)                         | 473                  |
| [swin_v2_s](<docs/models/swin_v2_s>)                                                                 | 🚧       | 1       | 35255.49                  |                                   0.0939288   | 3.2570107155652543                            | 46.94          | 3140 (40)                       | 158 (3)                         | 473                  |
| [swin_v2_t](<docs/models/swin_v2_t>)                                                                 | 🚧       | 1       | 102353.85                 |                                   0.176827    | 5.688929343497554                             | 51.91          | 1562 (40)                       | 74 (3)                          | 221                  |
| [tf_efficientnet_lite0.in1k-train](<docs/models/tf_efficientnet_lite0.in1k-train>)                   | 🚧       | 1       | 124752.46                 |                                   0.369907    | 0.13828772143321394                           | 100.0          | 452 (17)                        | 285 (8)                         | 5                    |
| [tf_efficientnet_lite1.in1k-train](<docs/models/tf_efficientnet_lite1.in1k-train>)                   | 🚧       | 1       | 126119.21                 |                                   0.297247    | 0.10388034646173153                           | 100.0          | 587 (17)                        | 370 (8)                         | 5                    |
| [tf_efficientnet_lite2.in1k-train](<docs/models/tf_efficientnet_lite2.in1k-train>)                   | 🚧       | 1       | 110428.23                 |                                   0.24354     | 0.08122711426055708                           | 100.0          | 587 (17)                        | 370 (8)                         | 5                    |
| [tf_efficientnet_lite3.in1k](<docs/models/tf_efficientnet_lite3.in1k>)                               | 🚧       | 1       | 100526.28                 |                                   0.371497    | 3.626473254759746                             | 99.15          | 221 (9)                         | 5 (1)                           | 5                    |
| [tf_efficientnet_lite3.in1k-train](<docs/models/tf_efficientnet_lite3.in1k-train>)                   | 🚧       | 1       | 102959.25                 |                                   0.146498    | 0.05607952525317102                           | 100.0          | 668 (17)                        | 426 (9)                         | 5                    |
| [tf_efficientnet_lite4.in1k](<docs/models/tf_efficientnet_lite4.in1k>)                               | 🚧       | 1       | 141541.03                 |                                   0.176362    | 2.0662437754406264                            | 99.21          | 275 (9)                         | 6 (1)                           | 5                    |
| [tf_efficientnet_lite4.in1k-train](<docs/models/tf_efficientnet_lite4.in1k-train>)                   | 🚧       | 1       | 171026.86                 |                                   0.0630897   | 0.021454508396758224                          | 100.0          | 830 (17)                        | 529 (9)                         | 5                    |
| [vit_b_16](<docs/models/vit_b_16>)                                                                   | 🚧       | 1       | 32585.29                  |                                   0.0669537   | 6.713662302786171                             | 99.54          | 552 (17)                        | 1 (1)                           | 1                    |
| [vit_b_32](<docs/models/vit_b_32>)                                                                   | 🚧       | 1       | 15789.18                  |                                   0.192324    | 7.1103526734926055                            | 98.68          | 552 (17)                        | 1 (1)                           | 1                    |
| [vit_h_14](<docs/models/vit_h_14>)                                                                   | 🚧       | 1       | 691117.65                 |                                   0.0013098   | 0.37087586043199616                           | 98.59          | 1452 (17)                       | 1 (1)                           | 1                    |
| [vit_l_16](<docs/models/vit_l_16>)                                                                   | 🚧       | 1       | 55829.49                  |                                   0.0195771   | 3.627525664744078                             | 99.74          | 1092 (17)                       | 1 (1)                           | 1                    |
| [vit_l_32](<docs/models/vit_l_32>)                                                                   | 🚧       | 1       | 35848.34                  |                                   0.0569023   | 4.707654646455136                             | 98.97          | 1092 (17)                       | 1 (1)                           | 1                    |
| [xception71.tf_in1k-train](<docs/models/xception71.tf_in1k-train>)                                   | 🚧       | 1       | 164520.82                 |                                   0.0171423   | 0.017446645106102644                          | 100.0          | 1378 (18)                       | 806 (7)                         | 0                    |
| [FLAN-T5](<docs/models/FLAN-T5>)                                                                     | ❌       | N/A     | N/A                       |                                   0.301573    | N/A                                           | N/A            | 20020 (38)                      | N/A                             | N/A                  |
| [Falcon-7B](<docs/models/Falcon-7B>)                                                                 | ❌       | N/A     | N/A                       |                                   0.0145124   | N/A                                           | N/A            | 2600 (27)                       | N/A                             | N/A                  |
| [GPTNeo](<docs/models/GPTNeo>)                                                                       | ❌       | N/A     | N/A                       |                                   0.08369     | N/A                                           | N/A            | 2733 (35)                       | N/A                             | N/A                  |
| [Llama](<docs/models/Llama>)                                                                         | ❌       | N/A     | N/A                       |                                   0.00548268  | N/A                                           | N/A            | 3690 (35)                       | N/A                             | N/A                  |
| [OPT](<docs/models/OPT>)                                                                             | ❌       | N/A     | N/A                       |                                   0.0405035   | N/A                                           | N/A            | 4003 (32)                       | N/A                             | N/A                  |
| [Stable Diffusion V2](<docs/models/Stable Diffusion V2>)                                             | ❌       | N/A     | N/A                       |                                   0.000477051 | N/A                                           | N/A            | 1870 (29)                       | N/A                             | N/A                  |
| [ViLT](<docs/models/ViLT>)                                                                           | ❌       | N/A     | N/A                       |                                   0.064811    | N/A                                           | N/A            | 766 (29)                        | N/A                             | N/A                  |
| [Whisper](<docs/models/Whisper>)                                                                     | ❌       | N/A     | N/A                       |                                   0.00418479  | N/A                                           | N/A            | 4310 (21)                       | N/A                             | N/A                  |
| [YOLOv5](<docs/models/YOLOv5>)                                                                       | ❌       | N/A     | N/A                       |                                   0.042243    | N/A                                           | N/A            | 236 (13)                        | N/A                             | N/A                  |
| [codegen](<docs/models/codegen>)                                                                     | ❌       | N/A     | N/A                       |                                   0.148409    | N/A                                           | N/A            | 9183 (37)                       | N/A                             | N/A                  |
| [t5-base](<docs/models/t5-base>)                                                                     | ❌       | N/A     | N/A                       |                                   0.183717    | N/A                                           | N/A            | 14681 (38)                      | N/A                             | N/A                  |
| [t5-large](<docs/models/t5-large>)                                                                   | ❌       | N/A     | N/A                       |                                   0.0633669   | N/A                                           | N/A            | 22696 (38)                      | N/A                             | N/A                  |
| [t5-small](<docs/models/t5-small>)                                                                   | ❌       | N/A     | N/A                       |                                   0.403019    | N/A                                           | N/A            | 6118 (38)                       | N/A                             | N/A                  |

### Explanation of Metrics

**Model**: Name of the model.  
**Status**: Indicates whether the model is:
- ✅ End-to-end on device: All PyTorch operations have been converted to TT-NN operations.
- 🚧 Compiled: The converted model runs but some operations still fallback to PyTorch. This may be due to an unsupported operation or configuration.
- ❌ Traced: The model does not run but its PyTorch operations are traced for future development. This may indicate a temporary incompatibility with a compiler pass.  

**Batch**: Batch size used for inference  
**Compiled First Run (ms)**: Time until the first compiled run finishes (ms), including compilation time and warming caches.  
**Original Throughput (Inferences Per Second)**: Execution throughput (in inferences per second) of the model before conversion.  
**Compiled Throughput (Inferences Per Second)**: Execution throughput (in inferences per second) of the model after conversion, once caches are warm.  
**Accuracy (%)**: Model accuracy on a predefined test dataset after conversion.  
**Torch Ops Before (Unique Ops)**: The total number of operations used by the model in the original Torch implementation. The number in parentheses represents the total unique ops.  
**Torch Ops Remain (Unique Ops)**: The total number of operations used after conversion to TT-NN. The number in parentheses represents the total unique ops.  
**To/From Device Ops**: The number of `to/from_device` operations (data transfer to/from the device).  

***

# Contributing

Whether you are new to Tenstorrent hardware or an experienced developer, there are many ways to contribute.

## Getting Started

Start with our high level [Contribution guide](docs/Contributing.md).
You can find more information here:
* [Discussions](https://github.com/tenstorrent/pytorch2.0_ttnn/discussions)
* [Operations Report](docs/OperationsReport.md)
* [Lowering TT-NN Operation to PyTorch](docs/AddNewOperationLowering.md)
* [Native Device Integration Extension](docs/OpenRegistrationAPI.md)
* [Build with Metal from Source](docs/DevelopWithMetalFromSources.md)
* [Known Issues](docs/KnownIssues.md)
* [Problem Solving](docs/ProblemSolving.md)

We encourage contributions and offer [🤑 Bounties](https://github.com/tenstorrent/pytorch2.0_ttnn/issues?q=is%3Aissue%20state%3Aopen%20label%3Abounty) for some issues.

## Development Environment

To get started with development, you'll need a Wormhole or Blackhole Tenstorrent accelerator card, which:
* can be ordered on the [Tenstorrent website](https://tenstorrent.com/) 
* can be requested on [Koyeb](https://www.koyeb.com/blog/tenstorrent-cloud-instances-unveiling-next-gen-ai-accelerators)

Install the development dependencies:
```shell
pip install -r requirements-dev.txt
pip install -e .
```

You can build the wheel file with
```shell
python -m build
```

## Project Structure

- `torch_ttnn/`: Main package directory containing the core implementation
- `tests/`: Test files for the project including model suites. We use `pytest` as our testing framework.
- `tools/`: Development and utility scripts
- `docs/`: Project documentation and reports
- `demo/`: Example code and usage demonstrations

## Questions and Support

If you have questions or need help getting started, please:
1. Review the existing documentation
2. Ask [PyTorch TT-NN DeepWiki](https://deepwiki.com/tenstorrent/pytorch2.0_ttnn) or [TT-Metal DeepWiki](https://deepwiki.com/tenstorrent/tt_metal)
3. Ask on [Discord](https://discord.gg/tenstorrent)
4. Open an issue on GitHub

