Deep Learning for Video Object Segmentation: A Review

Mingqi Gao, Feng Zheng, James Yu, Caifeng Shan, Guiguang Ding, Jungong Han

Research output: Contribution to journalArticlepeer-review

5 Citations (SciVal)
101 Downloads (Pure)


As one of the fundamental problems in the field of video understanding, video object segmentation aims at segmenting objects of interest throughout the given video sequence. Recently, with the advancements of deep learning techniques, deep neural networks have shown outstanding performance improvements in many computer vision applications, with video object segmentation being one of the most advocated and intensively investigated. In this paper, we present a systematic review of the deep learning-based video segmentation literature, highlighting the pros and cons of each category of approaches. Concretely, we start by introducing the definition, background concepts and basic ideas of algorithms in this field. Subsequently, we summarise the datasets for training and testing a video object segmentation algorithm, as well as common challenges and evaluation metrics. Next, previous works are grouped and reviewed based on how they extract and use spatial and temporal features, where their architectures, contributions and the differences among each other are elaborated. At last, the quantitative and qualitative results of several representative methods on a dataset with many remaining challenges are provided and analysed, followed by further discussions on future research directions. This article is expected to serve as a tutorial and source of reference for learners intended to quickly grasp the current progress in this research area and practitioners interested in applying the video object segmentation methods to their problems. A public website is built to collect and track the related works in this field:
Original languageEnglish
Pages (from-to)457-531
Number of pages75
JournalArtificial Intelligence Review
Issue number1
Early online date08 Apr 2022
Publication statusPublished - 01 Jan 2023


  • Convolutional neural network
  • Deep learning
  • Video object segmentation


Dive into the research topics of 'Deep Learning for Video Object Segmentation: A Review'. Together they form a unique fingerprint.

Cite this