Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs

Xiaohan Ding, Xiangyu Zhang, Jungong Han, Guiguang Ding

Allbwn ymchwil: Cyfraniad arall

216 Dyfyniadau(SciVal)

Crynodeb

We revisit large kernel design in modern convolutional neural networks (CNNs). Inspired by recent advances in vision transformers (ViTs), in this paper, we demonstrate that using a few large convolutional kernels instead of a stack of small kernels could be a more powerful paradigm. We suggested five guidelines, e.g., applying re-parameterized large depthwise convolutions, to design efficient high-performance large-kernel CNNs. Following the guidelines, we propose RepLKNet, a pure CNN architecture whose kernel size is as large as 31×31, in contrast to commonly used 3×3. RepLKNet greatly closes the performance gap between CNNs and ViTs, e.g., achieving comparable or superior results than Swin Transformer on ImageNet and a few typical downstream tasks, with lower latency. RepLKNet also shows nice scalability to big data and large models, obtaining 87.8% top-1 accuracy on ImageNet and 56.0% mIoU on ADE20K, which is very competitive among the state-of-the-arts with similar model sizes. Our study further reveals that, in contrast to small-kernel CNNs, large-kernel CNNs have much larger effective receptive fields and higher shape bias rather than texture bias. Code & models at https://github.com/megvii-research/RepLKNet.

Iaith wreiddiolSaesneg
CyhoeddwrIEEE Press
Nifer y tudalennau13
ISBN (Argraffiad)1063-6919, 978-1-6654-6946-3
ISBN (Electronig)9781665469463
Dynodwyr Gwrthrych Digidol (DOIs)
StatwsCyhoeddwyd - 2022

Cyfres gyhoeddiadau

EnwProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Cyfrol2022-June
ISSN (Argraffiad)1063-6919

Ôl bys

Gweld gwybodaeth am bynciau ymchwil 'Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs'. Gyda’i gilydd, maen nhw’n ffurfio ôl bys unigryw.

Dyfynnu hyn