This is the source code for our paper: Channel Pruning Guided by Spatial and Channel Attention for DNNs in Intelligent Edge Computing. A brief introduction of this work is as follows:
Deep Neural Networks (DNNs) have achieved remarkable success in many computer vision tasks recently, but the huge number of parameters and the high computation overhead hinder their deployments on resource-constrained edge devices. It is worth noting that channel pruning is an effective approach for compressing DNN models. A critical challenge is to determine which channels are to be removed, so that the model accuracy will not be negatively affected. In this paper, we first propose Spatial and Channel Attention (SCA), a new attention module combining both spatial and channel attention that respectively focuses on "where" and "what" are the most informative parts. Guided by the scale values generated by SCA for measuring channel importance, we further propose a new channel pruning approach called Channel Pruning guided by Spatial and Channel Attention (CPSCA). Experimental results indicate that SCA achieves the best inference accuracy, while incurring negligibly extra resource consumption, compared to other state-of-the-art attention modules. Our evaluation on two benchmark datasets shows that, with the guidance of SCA, our CPSCA approach achieves higher inference accuracy than other state-of-the-art pruning methods under the same pruning ratios.
This paper has been accepted and has been published by Applied Soft Computing (ASOC), and the preprint version can be downloaded from here. You can also download the formal version from here.
We only provide our SCA and CPSCA here. You can find the implementation of other attention models mentioned in our paper from PytorchInsight. Due to some reason, we didn't provide the scaler_for_prune.txt file required by prune.py in the released code. If you want to know how to generate it, please contact the 1st author with [email protected].
PyTorch
If you use these models in your research, please cite:
@article{LIU2021107636,
title = {Channel pruning guided by spatial and channel attention for DNNs in intelligent edge computing},
journal = {Applied Soft Computing},
volume = {110},
pages = {107636},
year = {2021},
issn = {1568-4946},
doi = {https://doi.org/10.1016/j.asoc.2021.107636},
url = {https://www.sciencedirect.com/science/article/pii/S1568494621005573},
author = {Mengran Liu and Weiwei Fang and Xiaodong Ma and Wenyuan Xu and Naixue Xiong and Yi Ding},
}
Mengran Liu ([email protected])
Please note that the open source code in this repository was mainly completed by the graduate student author during his master's degree study. Since the author did not continue to engage in scientific research work after graduation, it is difficult to continue to maintain and update these codes. We sincerely apologize that these codes are for reference only.