Video Object Segmentation using Point-based Memory Network

Mingqi Gao, Jungong Han*, Feng Zheng, James J.Q. Yu, Giovanni Montana

*Awdur cyfatebol y gwaith hwn

Allbwn ymchwil: Cyfraniad at gyfnodolynErthygladolygiad gan gymheiriaid

4 Dyfyniadau(SciVal)
56 Wedi eu Llwytho i Lawr (Pure)


Recent years have witnessed the prevalence of memory-based methods for Semi-supervised Video Object Segmentation (SVOS) which utilise past frames efficiently for label propagation. When conducting feature matching, fine-grained multi-scale feature matching has typically been performed using all query points, which inevitably results in redundant computations and thus makes the fusion of multi-scale results ineffective. In this paper, we develop a new Point-based Memory Network, termed as PMNet, to perform fine-grained feature matching on hard samples only, assuming that easy samples can already obtain satisfactory matching results without the need for complicated multi-scale feature matching. Our approach first generates an uncertainty map from the initial decoding outputs. Next, the fine-grained features at uncertain locations are sampled to match the memory features on the same scale. Finally, the matching results are further decoded to provide a refined output. The point-based scheme works with the coarsest feature matching in a complementary and efficient manner. Furthermore, we propose an approach to adaptively perform global or regional matching based on the motion history of memory points, making our method more robust against ambiguous backgrounds. Experimental results on several benchmark datasets demonstrate the superiority of our proposed method over state-of-the-art methods.
Iaith wreiddiolSaesneg
Rhif yr erthygl109073
Nifer y tudalennau13
CyfnodolynPattern Recognition
Dyddiad ar-lein cynnar06 Hyd 2022
Dynodwyr Gwrthrych Digidol (DOIs)
StatwsCyhoeddwyd - 01 Chwef 2023

Ôl bys

Gweld gwybodaeth am bynciau ymchwil 'Video Object Segmentation using Point-based Memory Network'. Gyda’i gilydd, maen nhw’n ffurfio ôl bys unigryw.

Dyfynnu hyn