基于注意力机制的多尺度特征融合图像去雨方法
作者:
中图分类号:

TP391.4

基金项目:

国家自然科学基金(61971167,62101275,62101274);江苏省信息与通信工程优势学科建设项目


Image rain removal via multi-scale feature fusion based on attention mechanism
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • | | |
  • 文章评论
    摘要:

    雨纹分布和形状具有多样性,现有去雨算法在去雨的同时会产生图像背景模糊、泛化性能差等问题.因此,本文提出一种基于注意力机制的多尺度特征融合图像去雨方法.特征提取阶段由多个包含两个多尺度注意力残差块的残差组构成,多尺度注意力残差块利用多尺度特征提取模块提取及聚合不同尺度的特征信息,并通过坐标注意力进一步提高网络的特征提取能力.在组内进行局部特征融合,组间利用全局特征融合注意力模块更好地融合不同层次的特征,通过像素注意力使网络重点关注于雨纹区域.在仿真和真实雨像数据集上与其他现有图像去雨算法相比,本文方法的定量指标有着明显提高,去雨后的图像视觉效果较好且具有良好的泛化性.

    Abstract:

    Due to the diversity of the distribution and shape of rain streaks, existing rain removal algorithms produce problems such as blurred image background and poor generalization performance while removing rain.A multi-scale feature fusion image rain removal approach based on attention mechanism is proposed to address these problems.The feature extraction consists of multiple residual groups containing two multi-scale attention residual blocks, which use the multi-scale feature extraction module to extract and aggregate feature information at different scales and further improve the feature extraction capability of the network through coordinate attention.Local feature fusion is performed within groups, and the global feature fusion attention module is used between groups to better fuse features at different levels and to focus the network on rain streak regions through pixel attention.The quantitative metrics of the proposed approach are significantly improved compared with other existing image rain removal algorithms on both simulated and real rain image datasets, and the rain removal images are greatly improved in both visual effects and generalization performance.

    参考文献
    [1] Kang L W,Lin C W,Fu Y H.Automatic single-image-based rain streaks removal via image decomposition[J].IEEE Transactions on Image Processing,2012,21(4):1742-1755
    [2] Li Y,Tan R T,Guo X J,et al.Rain streak removal using layer priors[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA.IEEE,2016:2736-2744
    [3] Fu X Y,Huang J B,Ding X H,et al.Clearing the skies:a deep network architecture for single-image rain removal[J].IEEE Transactions on Image Processing,2017,26(6):2944-2956
    [4] Ren D W,Zuo W M,Hu Q H,et al.Progressive image deraining networks:a better and simpler baseline[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach,CA,USA.IEEE,2019:3932-3941
    [5] Fu X Y,Liang B R,Huang Y,et al.Lightweight pyramid networks for image deraining[J].IEEE Transactions on Neural Networks and Learning Systems,2020,31(6):1794-1807
    [6] Wang C,Xing X Y,Wu Y T,et al.DCSFN:deep cross-scale fusion network for single image rain removal[C]//Proceedings of the 28th ACM International Conference on Multimedia.Seattle,WA,USA.New York,NY,USA:ACM,2020:1643-1651
    [7] Yi Q S,Li J C,Dai Q Y,et al.Structure-preserving deraining with residue channel prior guidance[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV).Montreal,QC,Canada.IEEE,2021:4218-4227
    [8] Hou Q B,Zhou D Q,Feng J S.Coordinate attention for efficient mobile network design[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Nashville,TN,USA.IEEE,2021:13708-13717
    [9] He K M,Zhang X Y,Ren S Q,et al.Identity mappings in deep residual networks[M]//Computer Vision-ECCV 2016.Cham:Springer International Publishing,2016:630-645
    [10] 康健,管海燕,于永涛,等.基于RFA-LinkNet模型的高分遥感影像水体提取[J].南京信息工程大学学报(自然科学版),2023,15(2):160-168 KANG Jian,GUAN Haiyan,YU Yongtao,et al.RFA-LinkNet:a novel deep learning network for water-body extraction from high-resolution remote sensing images[J].Journal of Nanjing University of Information Science & Technology (Natural Science Edition),2023,15(2):160-168
    [11] 胡序洋,高尚兵,汪长春,等.LaneSegNet:一种高效的车道线检测方法[J].南京信息工程大学学报(自然科学版),2022,14(5):551-558 HU Xuyang,GAO Shangbing,WANG Changchun,et al.LaneSegNet:an efficient lane line detection method[J].Journal of Nanjing University of Information Science & Technology (Natural Science Edition),2022,14(5):551-558
    [12] 张凯,郭剑黎,胡军星,等.基于注意力残差网络的Wi-Fi设备的射频指纹识别[J].南京信息工程大学学报(自然科学版),2022,14(3):324-330 ZHANG Kai,GUO Jianli,HU Junxing,et al.Radio frequency fingerprint identification of Wi-Fi device based on attention residual network[J].Journal of Nanjing University of Information Science & Technology (Natural Science Edition),2022,14(3):324-330
    [13] Qin X,Wang Z L,Bai Y C,et al.FFA-net:feature fusion attention network for single image dehazing[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York,USA:AAAI,2020:11908-11915
    [14] 马婧婧,黄煜峰,陈翔.多尺度沙漏结构的单幅图像去雨算法研究[J].小型微型计算机系统,2021,42(3):561-565 MA Jingjing,HUANG Yufeng,CHEN Xiang.Multi-scale hourglass network for single image deraining[J].Journal of Chinese Computer Systems,2021,42(3):561-565
    [15] Yang W H,Tan R T,Feng J S,et al.Deep joint rain detection and removal from a single image[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,USA.IEEE,2017:1685-1694
    [16] Cai L W,Li S W,Ren D W,et al.Dual recursive network for fast image deraining[C]//Proceedings of the IEEE International Conference on Image Processing.Taipei,China.IEEE,2019:2756-2760
    [17] Yang Y Z,Lu H.Single image deraining via recurrent hierarchy enhancement network[C]//Proceedings of the 27th ACM International Conference on Multimedia.Nice,France.New York:ACM,2019:1814-1822
    [18] Shang W,Zhu P F,Ren D W,et al.Bilateral recurrent network for single image deraining[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing.Barcelona,Spain.IEEE,2020:2503-2507
    相似文献
    引证文献
引用本文

刘忠洋,周杰,陆加新,缪则林,邵根富,江凯强,高伟.基于注意力机制的多尺度特征融合图像去雨方法[J].南京信息工程大学学报(自然科学版),2023,15(5):505-513
LIU Zhongyang, ZHOU Jie, LU Jiaxin, MIAO Zelin, SHAO Genfu, JIANG Kaiqiang, GAO Wei. Image rain removal via multi-scale feature fusion based on attention mechanism[J]. Journal of Nanjing University of Information Science & Technology, 2023,15(5):505-513

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2022-07-18
  • 在线发布日期: 2023-10-24

地址:江苏省南京市宁六路219号    邮编:210044

联系电话:025-58731025    E-mail:nxdxb@nuist.edu.cn

南京信息工程大学学报 ® 2025 版权所有  技术支持:北京勤云科技发展有限公司