Where to Prune: Using LSTM to Guide Data-Dependent Soft Pruning

Guiguang Ding, Shuo Zhang, Zizhou Jia, Jing Zhong, Jungong Han

Allbwn ymchwil: Cyfraniad at gyfnodolynErthygladolygiad gan gymheiriaid

118 Wedi eu Llwytho i Lawr (Pure)


While convolutional neural network (CNN) has achieved overwhelming success in various vision tasks, its heavy computational cost and storage overhead limit the practical use on mobile or embedded devices. Recently, compressing CNN models has attracted considerable attention, where pruning CNN filters, also known as the channel pruning, has generated great research popularity due to its high compression rate. In this paper, a new channel pruning framework is proposed, which can significantly reduce the computational complexity while maintaining sufficient model accuracy. Unlike most existing approaches that seek to-be-pruned filters layer by layer, we argue that choosing appropriate layers for pruning is more crucial, which can result in more complexity reduction but less performance drop. To this end, we utilize a long short-term memory (LSTM) to learn the hierarchical characteristics of a network and generate a global network pruning scheme. On top of it, we propose a data-dependent soft pruning method, dubbed Squeeze-Excitation-Pruning (SEP), which does not physically prune any filters but selectively excludes some kernels involved in calculating forward and backward propagations depending on the pruning scheme. Compared with the hard pruning, our soft pruning can better retain the capacity and knowledge of the baseline model. Experimental results demonstrate that our approach still achieves comparable accuracy even when reducing 70.1% Floating-point operation per second (FLOPs) for VGG and 47.5% for Resnet-56.
Iaith wreiddiolSaesneg
Rhif yr erthygl9258919
Tudalennau (o-i)293 - 304
Nifer y tudalennau12
CyfnodolynIEEE Transactions on Image Processing
Dyddiad ar-lein cynnar13 Tach 2020
Dynodwyr Gwrthrych Digidol (DOIs)
StatwsCyhoeddwyd - 20 Tach 2020

Ôl bys

Gweld gwybodaeth am bynciau ymchwil 'Where to Prune: Using LSTM to Guide Data-Dependent Soft Pruning'. Gyda’i gilydd, maen nhw’n ffurfio ôl bys unigryw.

Dyfynnu hyn