LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers

Zhuoxu Huang, Zhiyou Zhao, Banghuai Li, Jungong Han

Research output: Contribution to journalArticlepeer-review

18 Citations (Scopus)
100 Downloads (Pure)

Abstract

Transformer with its underlying attention mechanism and the ability to capture long-range dependencies makes it become a natural choice for unordered point cloud data. However, local regions separated from the general sampling architecture corrupt the structural information of the instances, and the inherent relationships between adjacent local regions lack exploration. In other words, the transformer only focuses on the long-range dependence, while local structural information is still crucial in a transformer-based 3D point cloud model. To enable transformers to incorporate local structural information, we proposed a straightforward solution based on the natural structure of the point clouds to exploit the message passing between neighboring local regions, thus making their representations more comprehensive and discriminative. Concretely, the proposed module, named Local Context Propagation (LCP), is inserted between two transformer layers. It takes advantage of the overlapping points of adjacent local regions (statistically shown to be prevalent) as intermediaries, then re-weighs the features of these shared points from different local regions before passing them to the next layers. Finally, we design a flexible LCPFormer architecture equipped with the LCP module, which is applicable to several different tasks. Experimental results demonstrate that our proposed LCPFormer outperforms various transformer-based methods in benchmarks including 3D shape classification and dense prediction tasks such as 3D object detection and semantic segmentation. Code will be released for reproduction.

Original languageEnglish
Pages (from-to)4985-4996
Number of pages12
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume33
Issue number9
Early online date22 Feb 2023
DOIs
Publication statusPublished - 01 Sept 2023

Keywords

  • 3D vision
  • Context propagation
  • Convolution
  • Feature extraction
  • Point cloud compression
  • Point cloud learning
  • Solid modeling
  • Task analysis
  • Three-dimensional displays
  • Transformer
  • Transformers
  • point cloud learning
  • transformer
  • context propagation

Fingerprint

Dive into the research topics of 'LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers'. Together they form a unique fingerprint.

Cite this