A detailed comparsion of Recent Vision Transformers on ImageNet1k
| Model | Trait | ImgSize | Parms(M) | MACs(G) | Top1 | Code | Publish |
|---|---|---|---|---|---|---|---|
| New-ResNet18 | Conv | 224 | 12 | 1.8 | 70.6 | timm | 2021 Oct arXiv |
| New-ResNet34 | Conv | 224 | 22 | 3.7 | 75.5 | timm | 2021 Oct arXiv |
| New-ResNet50 | Conv | 224 | 26 | 4.1 | 79.8 | timm | 2021 Oct arXiv |
| New-ResNet101 | Conv | 224 | 45 | 7.9 | 81.3 | timm | 2021 Oct arXiv |
| New-ResNet152 | Conv | 224 | 60 | 11.6 | 81.8 | timm | 2021 Oct arXiv |
| ViT-B/16 | SA | 224 | 86 | 17.6 | 79.7 | timm | 2021 ICLR |
| ViT-L/16 | SA | 224 | 304.3 | 63.6 | 81.1 | timm | 2021 ICLR |
| DeiT-S | SA | 224 | 22 | 4.6 | 79.8 | timm | 2021 ICML |
| DeiT-B | SA | 224 | 86 | 17.5 | 81.8 | timm | 2021 ICML |
| PVT-Tiny | SA | 224 | 13 | 1.9 | 75.1 | code | 2021 ICCV |
| PVT-Small | SA | 224 | 25 | 3.8 | 79.8 | code | 2021 ICCV |
| PVT-Medium | SA | 224 | 44 | 6.7 | 81.2 | code | 2021 ICCV |
| PVT-Large | SA | 224 | 61 | 9.8 | 81.7 | code | 2021 ICCV |
| PVTv2-B1 | SA | 224 | 13.1 | 2.1 | 78.7 | code | 2022 CVM |
| PVTv2-B2 | SA | 224 | 25.4 | 4.0 | 82.0 | code | 2022 CVM |
| PVTv2-B3 | SA | 224 | 45.2 | 6.9 | 83.2 | code | 2022 CVM |
| PVTv2-B4 | SA | 224 | 62.6 | 10.1 | 83.6 | code | 2022 CVM |
| PVTv2-B5 | SA | 224 | 82.0 | 11.8 | 83.8 | code | 2022 CVM |
| MLP-Mixer-B/16 | MLP | 224 | 59 | 12.7 | 76.4 | code | 2021 NIPS |
| ResMLP-S12 | MLP | 224 | 15 | 3 | 76.6 | timm | 2021 May arXiv |
| ResMLP-S24 | MLP | 224 | 30 | 6 | 79.4 | timm | 2021 May arXiv |
| ResMLP-B24 | MLP | 224 | 116 | 23 | 81 | timm | 2021 May arXiv |
| Swin-Mixer-T/D24 | MLP | 256 | 20 | 4 | 79.4 | code | 2021 ICCV |
| Swin-Mixer-T/D6 | MLP | 256 | 23 | 4 | 79.7 | code | 2021 ICCV |
| Swin-Mixer-B/D24 | MLP | 224 | 61 | 10.4 | 81.3 | code | 2021 ICCV |
| gMLP-S | MLP | 224 | 20 | 4.5 | 79.6 | timm | 2021 NIPS |
| gMLP-B | MLP | 224 | 73 | 15.8 | 81.6 | timm | 2021 NIPS |
| PoolFormer-S12 | Pool | 224 | 12 | 2 | 77.2 | code | 2022 CVPR |
| PoolFormer-S24 | Pool | 224 | 21 | 3.6 | 80.3 | code | 2022 CVPR |
| PoolFormer-S36 | Pool | 224 | 31 | 5.2 | 81.4 | code | 2022 CVPR |
| PoolFormer-M36 | Pool | 224 | 56 | 9.1 | 82.1 | code | 2022 CVPR |
| PoolFormer-M48 | Pool | 224 | 73 | 11.9 | 82.5 | code | 2022 CVPR |
| Swin-T | SA | 224 | 29 | 4.5 | 81.3 | code | 2021 ICCV |
| Swin-S | SA | 224 | 50 | 8.7 | 83.0 | code | 2021 ICCV |
| Swin-B | SA | 224 | 88 | 15.4 | 83.5 | code | 2021 ICCV |
| Swin-B | SA | 384 | 88 | 47.0 | 84.5 | code | 2021 ICCV |
| ActiveMLP-xT | MLP | 224 | 15 | 2.2 | 79.7 | code, x | 2022 Mar |
| ActiveMLP-T | MLP | 224 | 27 | 4.0 | 82.0 | code, x | 2022 Mar |
| ActiveMLP-S | MLP | 224 | 39 | 6.9 | 83.0 | code, x | 2022 Mar |
| ActiveMLP-B | MLP | 224 | 52 | 10.1 | 83.5 | code, x | 2022 Mar |
| ActiveMLP-L | MLP | 224 | 76 | 12.3 | 83.6 | code, x | 2022 Mar |
| VAN-Tiny | DW-Conv | 224 | 4.1 | 0.9 | 75.4 | code | 2022 Feb |
| VAN-Small | DW-Conv | 224 | 13.9 | 2.5 | 81.1 | code | 2022 Feb |
| VAN-Base | DW-Conv | 224 | 26.6 | 5.0 | 82.8 | code | 2022 Feb |
| VAN-Large | DW-Conv | 224 | 44.8 | 9.0 | 83.9 | code | 2022 Feb |
| VAN-Huge | DW-Conv | 224 | 60.3 | 12.2 | 84.2 | code | 2022 Feb |
| NesT-T | SA | 224 | 17 | 5.8 | 81.5 | code | 2022 AAAI |
| NesT-S | SA | 224 | 38 | 10.4 | 83.3 | code | 2022 AAAI |
| NesT-B | SA | 224 | 68 | 17.9 | 83.8 | code | 2022 AAAI |