Long-tailed visual recognition with deep models: A methodological survey and evaluation

Yu Fu, Liuyu Xiang, Yumna Zahid, Guiguang Ding, Tao Mei, Qiang Shen, Jungong Han*

*Corresponding author for this work

Research output: Contribution to journalReview articlepeer-review

10 Citations (SciVal)
31 Downloads (Pure)

Abstract

In the real world, large-scale datasets for visual recognition typically exhibit a long-tailed distribution, where only a few classes contain adequate samples but the others have (much) fewer samples. With the advancement of data-hungry deep models for visual recognition, the low-tail power-law data distribution that biases the model training has attracted significant attention. When training with the long-tailed data, the majority classes dominate the training procedure, resulting in poor performance in instance-scarce classes. To tackle this problem, numerous strategies, such as re-sampling, cost-sensitive loss, meta-learning and transfer learning, have been proposed. This paper systematically reviews contemporary approaches for the long-tailed visual recognition task and categorizes these methods based on the stage applied as training, fine-tuning, and inference. Furthermore, we categorize training stage methods into data augmentation, re-sampling strategy, cost-sensitive loss, as well as multiple experts and transfer learning. Next, comprehensive comparisons are made in the balanced test set performance of long-tailed benchmarks and method robustness in diverse test distributions using metrics including top-1 accuracy, per-class accuracy, multi-class ROC AUC and Expected Calibration Error (ECE). At last, we outline the challenges in this field and future research trends. Our reviews and intriguing findings can be a tutorial for researchers working in the field of open-world deep learning.

Original languageEnglish
Pages (from-to)290-309
Number of pages20
JournalNeurocomputing
Volume509
Early online date07 Sept 2022
DOIs
Publication statusPublished - 14 Oct 2022

Keywords

  • Deep learning
  • Long-tail
  • Visual recognition

Fingerprint

Dive into the research topics of 'Long-tailed visual recognition with deep models: A methodological survey and evaluation'. Together they form a unique fingerprint.

Cite this