Learning transferable cross-modality representations for few-shot hyperspectral and LiDAR collaborative classification | |
Dai, Mofan; Xing, Shuai; Xu, Qing; Wang, Hanyun; Li, Pengcheng; Sun, Yifan; Pan, Jiechen; Li YQ(李玉琼) | |
通讯作者 | Xing, Shuai(xing972403@163.com) ; Li, Yuqiong(liyuqiong@imech.ac.cn) |
发表期刊 | INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION |
2024-02-01 | |
卷号 | 126页码:11 |
ISSN | 1569-8432 |
摘要 | Hyperspectral image (HSI) classification, incorporating both spatial and spectral information, is a crucial topic in earth observation and land cover analysis. However, ground objects with similar spectral attributes are still the challenges for finer classifications. Recently, deep learning-based multimodality fusion provides promising solutions by exploiting LiDAR data with its geometric information to fuse with spectral attributes. However, the labor-intensive and time-consuming multimodality data annotation limits the performance of supervised deep learning technologies. How to address the semantic disparity between the LiDAR data and HSIs, and learning transferable representations for cross-scene classifications are still challenging. In this paper, we propose a multimodal fusion relational network with meta-learning (MFRN-ML) to solve these challenges. Specifically, the MFRN-ML incorporates the multimodal learning and few-shot learning (FSL) into a three-stage task-based learning framework to learn the transferable cross-modality representations for few-shot HSI and LiDAR collaborative classification. First, a multimodal fusion relational network, composed of a cross-modality feature fusion module and a relation learning module, is proposed to address the challenge of limited annotations in multimodal learning in a data-adaptive way. Then, a three-stage task-based learning framework can train the network to learn transferable representations with few labeled samples for cross-scene classification. We perform experiments on four multimodal datasets collected by different sensors. Compared with existing supervised, semi-supervised, and meta-learning methods, MFRN-ML attains state-of-the-art performances in few-shot tasks. Particularly, our method shows promising generalization ability on unseen categories across different domains. |
关键词 | Multimodal remote sensing data Meta-learning Few-shot learning Cross-modality feature learning |
DOI | 10.1016/j.jag.2023.103640 |
收录类别 | SCI |
语种 | 英语 |
WOS记录号 | WOS:001152464700001 |
关键词[WOS] | LAND-COVER CLASSIFICATION ; IMAGE CLASSIFICATION ; NETWORK |
WOS研究方向 | Remote Sensing |
WOS类目 | Remote Sensing |
资助项目 | National Natural Science Foundation of China[42271457] ; National Natural Science Foundation of China[41876105] ; Henan Province[202300410535] ; Joint Fund of Collaborative Innovation Center of Geo-Information Technology for Smart Central Plains, Henan Province ; Key Laboratory of Spatiotemporal Perception and Intelligent Processing, Ministry of Natural Resources[212108] |
项目资助者 | National Natural Science Foundation of China ; Henan Province ; Joint Fund of Collaborative Innovation Center of Geo-Information Technology for Smart Central Plains, Henan Province ; Key Laboratory of Spatiotemporal Perception and Intelligent Processing, Ministry of Natural Resources |
论文分区 | 一类 |
力学所作者排名 | 1 |
RpAuthor | Xing, Shuai ; Li, Yuqiong |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://dspace.imech.ac.cn/handle/311007/94229 |
专题 | 流固耦合系统力学重点实验室 |
推荐引用方式 GB/T 7714 | Dai, Mofan,Xing, Shuai,Xu, Qing,et al. Learning transferable cross-modality representations for few-shot hyperspectral and LiDAR collaborative classification[J]. INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION,2024,126:11. |
APA | Dai, Mofan.,Xing, Shuai.,Xu, Qing.,Wang, Hanyun.,Li, Pengcheng.,...&李玉琼.(2024).Learning transferable cross-modality representations for few-shot hyperspectral and LiDAR collaborative classification.INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION,126,11. |
MLA | Dai, Mofan,et al."Learning transferable cross-modality representations for few-shot hyperspectral and LiDAR collaborative classification".INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION 126(2024):11. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论