基于自适应感知金字塔网络的图像去雨

doi:10.12068/j.issn.1005-3026.2022.04.003

东北大学学报（自然科学版） ›› 2022, Vol. 43 ›› Issue (4): 470-479.DOI: 10.12068/j.issn.1005-3026.2022.04.003

基于自适应感知金字塔网络的图像去雨

杨爱萍，王朝臣，王建，张腾飞

(天津大学电气自动化与信息工程学院，天津300072)

修回日期:2021-05-14 接受日期:2021-05-14 发布日期:2022-05-18
通讯作者: 杨爱萍
作者简介:杨爱萍(1977-)，女，山东聊城人，天津大学副教授.
基金资助:
国家自然科学基金资助项目(62071323，61771329，61632018); 天津市科技重大专项研发计划项目(18ZXZNGX00320).

Image Deraining Based on Adaptive Perceptual Pyramid Network

YANG Ai-ping， WANG Chao-chen， WANG Jian， ZHANG Teng-fei

School of Electrical and Information Engineering， Tianjin University， Tianjin 300072，China.

Revised:2021-05-14 Accepted:2021-05-14 Published:2022-05-18
Contact: YANG Ai-ping
About author:-
Supported by:
-

摘要/Abstract

摘要： 基于深度学习的单幅图像去雨已经取得了较大进展，但现有方法去雨后的图像仍然存在细节丢失、密集雨纹去除不彻底等问题.为此，本文提出一种基于自适应感知金字塔网络的单幅图像去雨方法，能够在有效去除密集雨纹的同时对细节进行修正，显著改善去雨图像的视觉质量.首先，基于小波变换构建多尺度金字塔网络，在各尺度子网络之间进行递进式连接，实现雨纹迭代提取和去除;各尺度子网络内部以自适应雨纹感知模块为核心，设计对称跳跃连接将提取到的浅层特征反馈至深层，实现浅层特征的有效复用.其中，所设计的自适应雨纹感知模块通过非局部感知运算和共享扩张卷积扩大感受野，可有效感知雨纹特征，并融入注意力机制实现雨纹的自适应去除.为了更好地约束网络训练和去除不同尺度的雨纹，设计了一种多尺度损失函数，由粗及细逐步完成雨纹去除，可有效防止伪影现象.在合成和真实数据集上的大量实验表明，本文方法优于现有的主流方法，能够在去雨的同时较好地保持图像细节，视觉效果理想.

关键词: 图像去雨;小波变换;金字塔网络;自适应感知;特征融合

Abstract: Although deep learning-based methods have made great progresses in the field of single image deraining， existing methods still have some problems such as loss of detail and residue of rain streaks. To overcome these short-comings， this paper proposed an adaptive perceptual pyramid network to remove dense rain streaks while revising image details at the same time and thus significantly improve the visual quality. Firstly， a multi-scale pyramid network with progressive connected sub-networks is constructed based on wavelet transform to extract and remove rain streaks iteratively. Each sub-network with the adaptive rain streak perceptual block as its core module is constructed through symmetric skip-connection to extract the shallow layer features to deeper layers to be reused. The adaptive rain streak perceptual block uses non-local operation and shared dilated convolution to expand the receptive fields， which can effectively perceive and remove rain streak adaptively. In order to remove rain streaks with different scales and impose some constrains of the network training， a multi-scale loss function is designed so as to gradually remove rain streaks from coarse to fine， which can effectively prevent artifacts. Extensive experimental results on synthetic and real datasets demonstrate the superiority of the proposed method over the state-of-the-art methods， which can remove rain streaks effectively with preserving vivid image details.

Key words: image deraining; wavelet transform; pyramid network; adaptive perception; feature fusion

中图分类号:

TP391

杨爱萍，王朝臣，王建，张腾飞. 基于自适应感知金字塔网络的图像去雨[J]. 东北大学学报（自然科学版）, 2022, 43(4): 470-479.

YANG Ai-ping， WANG Chao-chen， WANG Jian， ZHANG Teng-fei. Image Deraining Based on Adaptive Perceptual Pyramid Network[J]. Journal of Northeastern University(Natural Science), 2022, 43(4): 470-479.

参考文献

[1]Yang W，Tan R T，Wang S，et al.Single image deraining:from model-based to data-driven and beyond［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43(11):4059-4077.
[2]Kim J H，Lee C，Sim J Y，et al.Single-image deraining using an adaptive nonlocal means filter［C］//2013 IEEE International Conference on Image Processing.Melbourne:IEEE，2013:914-917.
[3]Wang Y T，Zhao X L，Jiang T X，et al.Rain streak removal for single image via kernel guided CNN［J］.arXiv preprint arXiv:1808.08545，2018.
[4]Chen Y L，Hsu C T.A generalized low-rank appearance model for spatiotemporally correlated rain streaks［C］//Proceedings of the IEEE International Conference on Computer Vision.Sydney:IEEE，2013:1968-1975.
[5]Kim J H，Sim J Y，Kim C S.Video deraining and desnowing using temporal correlation and low-rank matrix completion［J］.IEEE Transactions on Image Processing，2015，24(9):2658-2670.
[6]Gu S，Meng D，Zuo W，et al.Joint convolutional analysis and synthesis sparse representation for single image layer separation［C］//Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE，2017:1717-1725.
[7]Kang L W，Lin C W，Fu Y H.Automatic single-image-based rain streaks removal via image decomposition［J］.IEEE Transactions on Image Processing，2012，21(4):1742-1755.
[8]Wang Y，Liu S，Chen C，et al.A hierarchical approach for rain or snow removing in a single color image［J］.IEEE Transactions on Image Processing，2017，26(8):3936-3950.
[9]Luo Y，Xu Y，Ji H.Removing rain from a single image via discriminative sparse coding［C］//Proceedings of the IEEE International Conference on Computer Vision.Santiago:IEEE，2015:3397-3405.
[10]Li Y，Tan R T，Guo X，et al.Rain streak removal using layer priors［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE，2016:2736-2744.
[11]Deng L J，Huang T Z，Zhao X L，et al.A directional global sparse model for single image rain removal［J］.Applied Mathematical Modelling，2018，59:662-679.
[12]Eigen D，Krishnan D，Fergus R.Restoring an image taken through a window covered with dirt or rain［C］//Proceedings of the IEEE International Conference on Computer Vision.Sydney:IEEE，2013:633-640.
[13]Fu X，Huang J，Zeng D，et al.Removing rain from single images via a deep detail network［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE，2017:1715-1723.
[14]Zhang H，Sindagi V，Patel V M.Image deraining using a conditional generative adversarial network［J］.IEEE Transactions on Circuits and Systems for Video Technology，2019，30(11):3943-3956.
[15]Li S，Ren W，Zhang J，et al.Single image rain removal via a deep decomposition-composition network［J］.Computer Vision and Image Understanding，2019，186:48-57.
[16]Pan J，Liu S，Sun D，et al.Learning dual convolutional neural networks for low-level vision［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE，2018:3070-3079.
[17]Zhang H，Patel V M.Density-aware single image deraining using a multi-stream dense network［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE，2018:695-704.
[18]Wang Y T，Zhao X L，Jiang T X，et al.Rain streaks removal for single image via kernel-guided convolutional neural network［J］.IEEE Transactions on Neural Networks and Learning Systems，2020，32(8):3664-3676.
[19]Hu X，Fu C W，Zhu L，et al.Depth-attentional features for single-image rain removal［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE，2019:8022-8031.
[20]Yang W，Tan R T，Feng J，et al.Deep joint rain detection and removal from a single image［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Puerto Rico:IEEE，2017:1357-1366.
[21]Fu X，Liang B，Huang Y，et al.Lightweight pyramid networks for image deraining［J］.IEEE Transactions on Neural Networks and Learning Systems，2019，31(6):1794-1807.
[22]Li R，Cheong L F，Tan R T.Single image deraining using scale-aware multi-stage recurrent network［J］.arXiv preprint arXiv:1712.06830，2017.
[23]Li X，Wu J，Lin Z，et al.Recurrent squeeze-and-excitation context aggregation net for single image deraining［C］//Proceedings of the European Conference on Computer Vision(ECCV).Munich，2018:254-269.
[24]Ren D，Zuo W，Hu Q，et al.Progressive image deraining networks:a better and simpler baseline［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE，2019:3937-3946.
[25]Deng S，Wei M，Wang J，et al.Detail-recovery image deraining via context aggregation networks［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE，2020:14560-14569.
[26]Jiang K，Wang Z，Yi P，et al.Multi-scale progressive fusion network for single image deraining［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE，2020:8346-8355.
[27]Zhao J，Xie J，Xiong R，et al.Pyramid convolutional network for single image deraining［C］//CVPR Workshops.Long Beach，2019:9-16.
[28]Johnson J，Alahi A，Li F F.Perceptual losses for real-time style transfer and super-resolution［C］//European Conference on Computer Vision.Amsterdam，2016:694-711.

基于自适应感知金字塔网络的图像去雨

Image Deraining Based on Adaptive Perceptual Pyramid Network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	庞彦伟，苏畅，龙涛. 自适应构造与聚合多尺度代价体的双目立体匹配[J]. 东北大学学报（自然科学版）, 2023, 44(4): 457-468.
[2]	丁其川，王力，刘成. 融合长距离信道注意力与病理特征的肺结节分类[J]. 东北大学学报（自然科学版）, 2023, 44(4): 476-485.
[3]	李海燕，熊立昌，郭磊，李海江. 基于U-net边缘生成和超图卷积的两阶段修复算法[J]. 东北大学学报（自然科学版）, 2023, 44(3): 331-339.
[4]	杨丹，刘国如，任梦成，裴宏杨. 多尺度卷积核U-Net模型的视网膜血管分割方法[J]. 东北大学学报（自然科学版）, 2021, 42(1): 7-14.
[5]	韩东红，张宏亮，朱帅伟，齐孝龙. 面向新浪微博的情感社区检测算法[J]. 东北大学学报（自然科学版）, 2021, 42(1): 21-31.
[6]	原培新，陈鼎夫. 双能X射线高动态范围安检图像压缩算法[J]. 东北大学学报（自然科学版）, 2021, 42(1): 96-101.
[7]	魏颖, 徐楚翘, 刁兆富, 李伯群. 基于生成对抗网络的多目标行人跟踪算法[J]. 东北大学学报（自然科学版）, 2020, 41(12): 1673-1680.
[8]	李稷，徐安军. 炼钢车间多天车动态调度仿真方案[J]. 东北大学学报:自然科学版, 2020, 41(12): 1699-1707.
[9]	李占山，吕艾娜. 基于新冗余度的特征选择方法[J]. 东北大学学报:自然科学版, 2020, 41(11): 1550-1556.
[10]	王鑫，王翠荣，王聪，苑迎. 双通道多感知卷积神经网络图像超分辨率重建[J]. 东北大学学报:自然科学版, 2020, 41(11): 1564-1570.
[11]	张春雷，戴丽，刘宇，李鹤. 基于三点法和ICP算法的手术导航系统患者配准[J]. 东北大学学报:自然科学版, 2020, 41(11): 1584-1590.
[12]	陈剑，何涛，闻英友，马林涛. 基于BERT模型的司法文书实体识别方法[J]. 东北大学学报:自然科学版, 2020, 41(10): 1382-1387.
[13]	张田，田勇，王子，王昭东. 基于清晰度评价的自适应阈值图像分割法[J]. 东北大学学报:自然科学版, 2020, 41(9): 1231-1238.
[14]	张丽杰，刘建昌，谭树彬. 复杂建筑火灾中的人员疏散路径多目标规划[J]. 东北大学学报:自然科学版, 2020, 41(6): 761-766.
[15]	赵海，周冰玲，朱宏博，窦圣昶. 基于连续最大流的三维肺实质快速分割算法[J]. 东北大学学报:自然科学版, 2020, 41(4): 470-474.