Employing Bilinear Fusion and Saliency Prior Information for RGB-D Salient Object Detection

Nianchang Huang, Yang Yang, Dingwen Zhang, Qiang Zhang, Jungong Han

Research output: Contribution to journalArticlepeer-review

29 Citations (SciVal)
250 Downloads (Pure)

Abstract

Multi-modal feature fusion and saliency reasoning are two core sub-tasks of RGB-D salient object detection. However, most existing models employ linear fusion strategies (e.g., concatenation) for multi-modal feature fusion and use a simple coarse-to-fine structure for saliency reasoning. Despite their simpleness, they can neither fully capture the cross-modal complementary information nor exploit the multi-level complementary information among the cross-modal features at different levels. To address these issues, a novel RGB-D salient object detection model is presented, where we pay special attention to the aforementioned two sub-tasks. Concretely, a multi-modal feature interaction module is first presented to explore more interactions between the unimodal RGB and depth features. It helps to capture their cross-modal complementary information by jointly using some simple linear fusion strategies and bilinear fusion ones. Then, a saliency prior information guided fusion module is presented to exploit the multi-level complementary information among the fused cross-modal features at different levels. Instead of employing a simple convolutional layer for the final saliency prediction, a saliency refinement and prediction module is designed to better exploit those extracted multi-level cross-modal information for RGB-D saliency detection. Experimental results on several benchmark datasets verify the effectiveness and superiority of the proposed framework over some state-of-the-art methods.

Original languageEnglish
Pages (from-to)1651-1664
Number of pages14
JournalIEEE Transactions on Multimedia
Volume24
Early online date31 Mar 2021
DOIs
Publication statusPublished - 01 Jan 2022

Keywords

  • Bilinear fusion strategy
  • RGB-D salient object detection
  • saliency prior information guided fusion
  • saliency refinement and prediction

Fingerprint

Dive into the research topics of 'Employing Bilinear Fusion and Saliency Prior Information for RGB-D Salient Object Detection'. Together they form a unique fingerprint.

Cite this