基于多特征交互和密集残差的图像去雨
作者:
基金项目:

国家重点研发计划(2018YFB1403303);辽宁省教育厅高等学校基本科研项目(LJKMZ20220615)


Image rain removal based on multi-feature interaction and dense residual
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • | | |
  • 文章评论
    摘要:

    针对雨天环境下获取图像质量差,导致后续机器视觉任务效率低下的问题,提出一种基于多特征交互和密集残差的图像去雨算法.首先,提出多重特征交互卷积模块提取不同空间下雨线的语义特征,增强信息利用程度;其次,构建多维空间权重注意模块,将不同空间信息权重初步融合并增强雨线特征;然后,结合密集连接和残差网络的优点,设计一种密集残差融合模块,在提高网络学习能力的同时实现对信息的重复利用,进一步校正雨纹信息;最后,通过将多种损失函数线性组合,并结合雨天成像模型提高输出图像质量.在多个公开数据集上的实验结果表明,本文所提算法的主客观评价指标均优于所对比的经典及新颖算法,在去除雨纹的同时能更有效地保留图像背景细节信息.

    Abstract:

    To solve the poor image quality and subsequent low efficiency of machine vision tasks on rainy days,an image rain removal algorithm based on multi-feature interaction and dense residual is proposed.First,a multi-feature interactive convolution module is proposed to extract the semantic features of rain streaks in different spaces to enhance information utilization.Second,a multidimensional space weight attention module is constructed,and the weights of different spatial information are preliminarily integrated to enhance the characteristics of rain streaks.Then combining the advantages of dense connection and residual network,a dense residual fusion module is designed,which improves the learning ability of the network,realizes the reuse of information,and further corrects the rain information.Finally,the output image quality is improved through the linear combination of various loss functions as well as the rainy day imaging model.Experiments on several public datasets show that the subjective and objective evaluation indexes of the proposed algorithm outperform those of the classical algorithm and novel algorithms,and the detailed background information of the images can be better preserved while removing the rain streaks.

    参考文献
    [1] 张俊, 江凡, 彭德猛, 等. 雨天环境下的交通流影响研究[J]. 中国安全科学学报, 2023, 33(5):134-143 ZHANG Jun, JIANG Fan, PENG Demeng, et al. Research on impact of traffic flow in rainy environments[J]. China Safety Science Journal, 2023, 33(5):134-143
    [2] 施赛楠, 姜丽, 曹鼎, 等. 基于频域相对样本熵的海面小目标特征检测[J]. 南京信息工程大学学报(自然科学版), 2023, 15(4):429-438 SHI Sainan, JIANG Li, CAO Ding, et al. Feature detection of sea-surface small targets via relative sample entropy in frequency domain[J]. Journal of Nanjing University of Information Science & Technology (Natural Science Edition), 2023, 15(4):429-438
    [3] 方莉娜, 王康. 基于车载激光点云的道路交叉口检测与识别[J]. 南京信息工程大学学报(自然科学版), 2021, 13(6):635-644 FANG Lina, WANG Kang. Road intersection detection and recognition based on mobile laser scanning[J]. Journal of Nanjing University of Information Science & Technology (Natural Science Edition), 2021, 13(6):635-644
    [4] Kim J H, Lee C, Sim J Y, et al. Single-image deraining using an adaptive nonlocal means filter[C]//2013 IEEE International Conference on Image Processing. September 15-18, 2013, Melbourne, VIC, Australia. IEEE, 2014:914-917
    [5] Luo Y, Xu Y, Ji H. Removing rain from a single image via discriminative sparse coding[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2016:3397-3405
    [6] Li Y, Tan R T, Guo X J, et al. Rain streak removal using layer priors[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016:2736-2744
    [7] Chang Y, Yan L X, Zhong S. Transformed low-rank model for line pattern noise removal[C]//2017 IEEE International Conference on Computer Vision (ICCV). October 22-29, 2017, Venice, Italy. IEEE, 2017:1735-1743
    [8] Deng S, Wei M Q, Wang J, et al. Detail-recovery image deraining via context aggregation networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020:14548-14557
    [9] Jiang K, Wang Z Y, Yi P, et al. Multi-scale progressive fusion network for single image deraining[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020:8343-8352
    [10] Zhang H, Sindagi V, Patel V M. Image de-raining using a conditional generative adversarial network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11):3943-3956
    [11] Huang H B, Yu A J, He R. Memory oriented transfer learning for semi-supervised image deraining[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 20-25, 2021, Nashville, TN, USA. IEEE, 2021:7728-7737
    [12] Zhou M, Xiao J, Chang Y F, et al. Image de-raining via continual learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 20-25, 2021, Nashville, TN, USA. IEEE, 2021:4905-4914
    [13] Zheng S, Lu C J, Wu Y X, et al. SAPNet:segmentation-aware progressive network for perceptual contrastive deraining[C]//2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW). January 4-8, 2022, Waikoloa, HI, USA. IEEE, 2022:52-62
    [14] 秦晓琪, 刘小勤, 马浩东. 多尺度生成对抗网络单幅图像去雨方法研究[J]. 仪表技术, 2021(4):47-50 QIN Xiaoqi, LIU Xiaoqin, MA Haodong. Research on rain removal method for single image of multi-scale generative adversarial network[J]. Instrumentation Technology, 2021(4):47-50
    [15] 刘西宁. 基于深度残差网络的红外热波图像去模糊[D]. 成都:电子科技大学, 2021 LIU Xining. Infrared thermal wave image deblurring based on depth residual network[D]. Chengdu:University of Electronic Science and Technology of China, 2021
    [16] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016:770-778
    [17] Huang G, Liu Z, Maaten L V D, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:2261-2269
    [18] Ioffe S, Szegedy C. Batch normalization:accelerating deep network training by reducing internal covariate shift[J]. Proceedings of the 32nd International Conference on Machine Learning, 2015, 37:448-456
    [19] Zhao H, Gallo O, Frosio I, et al. Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017, 3(1):47-57
    [20] 徐景秀, 张青. 改进小波软阈值函数在图像去噪中的研究应用[J]. 计算机工程与科学, 2022, 44(1):92-101 XU Jingxiu, ZHANG Qing. Research and application of an improved wavelet soft threshold function in image denoising[J]. Computer Engineering and Science, 2022, 44(1):92-101
    [21] 王海涛, 林森, 陶志勇. 双注意力机制与改进U-Net的水下图像增强[J]. 电子测量技术, 2023, 46(1):181-187 WANG Haitao, LIN Sen, TAO Zhiyong. Underwater image enhancement based on dual attention mechanism and improved U-Net[J]. Electronic Measurement Technology, 2023, 46(1):181-187
    [22] Ran W, Yang Y Z, Lu H. Single image rain removal boosting via directional gradient[C]//2020 IEEE International Conference on Multimedia and Expo (ICME). July 6-10, 2020, London, UK. IEEE, 2020:1-6
    [23] Chen D D ,He M M,Fan O N,et al.Gated context aggregation network for image dehazing and deraining( C ]//2019 IEEE Winter Conference on Applications of Computer Vision ( WACV) .January 7 - 11 2019 , Waikoloa,HI USA.IEEE 2019.1375-1383
    [23] Li X A , Wu J LLin Z C , et al.Recurrent squeeze-and-excitation context aggregation net for single image deraining[ M]//Computer Vision-ECCV 2018.Cham;Springer International Publishing,2018:262-277
    [25] Ren D W, Zuo W M, Hu Q H, et al. Progressive image deraining networks:a better and simpler baseline[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2020:3932-3941
    [26] Shang W, Zhu P F, Ren D W, et al. Bilateral recurrent network for single image deraining[C]//2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). May 4-8, 2020, Barcelona, Spain. IEEE, 2020:2503-2507
    [27] Yang W H, Tan R T, Feng J S, et al. Deep joint rain detection and removal from a single image[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:1685-1694
    [28] Yasarla R, Valanarasu J M J, Patel V M. Exploring overcomplete representations for single image deraining using CNNs[J]. IEEE Journal of Selected Topics in Signal Processing, 2021, 15(2):229-239
    [29] 柳长源, 王琪, 毕晓君. 基于多通道多尺度卷积神经网络的单幅图像去雨方法[J]. 电子与信息学报, 2020, 42(9):2285-2292 LIU Changyuan, WANG Qi, BI Xiaojun. Research on rain removal method for single image based on multi-channel and multi-scale CNN[J]. Journal of Electronics & Information Technology, 2020, 42(9):2285-2292
    [30] Huynh-Thu Q, Ghanbari M. Scope of validity of PSNR in image/video quality assessment[J]. Electronics Letters, 2008, 44(13):800-801
    [31] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2004, 13(4):600-612
    [32] Mittal A, Soundararajan R, Bovik A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20(3):209-212
    [33] Wang J F , Chen Y, Dong Z K, et al.Improved YOLOv5 network for real-time multi-scale traffic sign detection [J].Neural Computing and Applications ,2023 , 35( 10) :7853-7865
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

林森,邱庆澳.基于多特征交互和密集残差的图像去雨[J].南京信息工程大学学报(自然科学版),2024,16(4):472-481
LIN Sen, QIU Qingao. Image rain removal based on multi-feature interaction and dense residual[J]. Journal of Nanjing University of Information Science & Technology, 2024,16(4):472-481

复制
分享
文章指标
  • 点击次数:81
  • 下载次数: 830
  • HTML阅读次数: 37
  • 引用次数: 0
历史
  • 收稿日期:2023-07-18
  • 在线发布日期: 2024-08-07
  • 出版日期: 2024-07-28

地址:江苏省南京市宁六路219号    邮编:210044

联系电话:025-58731025    E-mail:nxdxb@nuist.edu.cn

南京信息工程大学学报 ® 2025 版权所有  技术支持:北京勤云科技发展有限公司