Convolutional LSTM-Based Hierarchical Feature Fusion for Multispectral Pan-Sharpening

Dong Wang, Yunpeng Bai, Chanyue Wu, Ying Li*, Changjing Shang, Qiang Shen

*Awdur cyfatebol y gwaith hwn

Allbwn ymchwil: Cyfraniad at gyfnodolynErthygladolygiad gan gymheiriaid

49 Wedi eu Llwytho i Lawr (Pure)


Multispectral (MS) pan-sharpening aims at producing high-resolution (HR) MS images in both spatial and spectral domains, by merging single-band panchromatic (PAN) images and corresponding MS images with low spatial resolution. The intuitive way to accomplish such MS pan-sharpening tasks, or to reconstruct ideal HR-MS images, is to extract feature pairs from the given PAN and MS images and to fuse the results. Therefore, feature extraction and feature fusion are two key components for MS pan-sharpening. This article presents a novel MS pan-sharpening network (MPNet), including a heterogeneous pair of feature extraction pathways (FEPs) and a convolutional long short-term memory (ConvLSTM)-based hierarchical feature fusion module (HFFM). Specifically, we design a PAN FEP to extract 2-D feature maps via 2-D convolutions and dual attention, while an MS FEP is introduced in an effort to obtain 3-D representations of MS image by 3-D convolutions and triple attention. To merge the resulting hierarchical features, the ConvLSTM-based HFFM is developed, leveraging intralevel fusion, interlevel fusion, and information exchange within one single framework. Here, the interlevel fusion is implemented with the ConvLSTM to capture the dependencies among hierarchical features, reduce redundant information, and effectively integrate them via its recurrent architecture. The information exchange between different FEPs helps enhance the representations for subsequent processing. Systematic comparative experiments have been conducted on three publicly available datasets at both reduced resolution and full resolution, demonstrating that the proposed MPNet outperforms state-of-the-art methods in the literature.

Iaith wreiddiolSaesneg
Rhif yr erthygl5404016
CyfnodolynIEEE Transactions on Geoscience and Remote Sensing
Dyddiad ar-lein cynnar20 Awst 2021
Dynodwyr Gwrthrych Digidol (DOIs)
StatwsCyhoeddwyd - 26 Ion 2022

Ôl bys

Gweld gwybodaeth am bynciau ymchwil 'Convolutional LSTM-Based Hierarchical Feature Fusion for Multispectral Pan-Sharpening'. Gyda’i gilydd, maen nhw’n ffurfio ôl bys unigryw.

Dyfynnu hyn