Image Deraining Based on Adaptive Perceptual Pyramid Network

doi:10.12068/j.issn.1005-3026.2022.04.003

Abstract

Abstract: Although deep learning-based methods have made great progresses in the field of single image deraining， existing methods still have some problems such as loss of detail and residue of rain streaks. To overcome these short-comings， this paper proposed an adaptive perceptual pyramid network to remove dense rain streaks while revising image details at the same time and thus significantly improve the visual quality. Firstly， a multi-scale pyramid network with progressive connected sub-networks is constructed based on wavelet transform to extract and remove rain streaks iteratively. Each sub-network with the adaptive rain streak perceptual block as its core module is constructed through symmetric skip-connection to extract the shallow layer features to deeper layers to be reused. The adaptive rain streak perceptual block uses non-local operation and shared dilated convolution to expand the receptive fields， which can effectively perceive and remove rain streak adaptively. In order to remove rain streaks with different scales and impose some constrains of the network training， a multi-scale loss function is designed so as to gradually remove rain streaks from coarse to fine， which can effectively prevent artifacts. Extensive experimental results on synthetic and real datasets demonstrate the superiority of the proposed method over the state-of-the-art methods， which can remove rain streaks effectively with preserving vivid image details.

Key words: image deraining; wavelet transform; pyramid network; adaptive perception; feature fusion

CLC Number:

TP391

YANG Ai-ping， WANG Chao-chen， WANG Jian， ZHANG Teng-fei. Image Deraining Based on Adaptive Perceptual Pyramid Network[J]. Journal of Northeastern University(Natural Science), 2022, 43(4): 470-479.

References

[1]Yang W，Tan R T，Wang S，et al.Single image deraining:from model-based to data-driven and beyond［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43(11):4059-4077.
[2]Kim J H，Lee C，Sim J Y，et al.Single-image deraining using an adaptive nonlocal means filter［C］//2013 IEEE International Conference on Image Processing.Melbourne:IEEE，2013:914-917.
[3]Wang Y T，Zhao X L，Jiang T X，et al.Rain streak removal for single image via kernel guided CNN［J］.arXiv preprint arXiv:1808.08545，2018.
[4]Chen Y L，Hsu C T.A generalized low-rank appearance model for spatiotemporally correlated rain streaks［C］//Proceedings of the IEEE International Conference on Computer Vision.Sydney:IEEE，2013:1968-1975.
[5]Kim J H，Sim J Y，Kim C S.Video deraining and desnowing using temporal correlation and low-rank matrix completion［J］.IEEE Transactions on Image Processing，2015，24(9):2658-2670.
[6]Gu S，Meng D，Zuo W，et al.Joint convolutional analysis and synthesis sparse representation for single image layer separation［C］//Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE，2017:1717-1725.
[7]Kang L W，Lin C W，Fu Y H.Automatic single-image-based rain streaks removal via image decomposition［J］.IEEE Transactions on Image Processing，2012，21(4):1742-1755.
[8]Wang Y，Liu S，Chen C，et al.A hierarchical approach for rain or snow removing in a single color image［J］.IEEE Transactions on Image Processing，2017，26(8):3936-3950.
[9]Luo Y，Xu Y，Ji H.Removing rain from a single image via discriminative sparse coding［C］//Proceedings of the IEEE International Conference on Computer Vision.Santiago:IEEE，2015:3397-3405.
[10]Li Y，Tan R T，Guo X，et al.Rain streak removal using layer priors［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE，2016:2736-2744.
[11]Deng L J，Huang T Z，Zhao X L，et al.A directional global sparse model for single image rain removal［J］.Applied Mathematical Modelling，2018，59:662-679.
[12]Eigen D，Krishnan D，Fergus R.Restoring an image taken through a window covered with dirt or rain［C］//Proceedings of the IEEE International Conference on Computer Vision.Sydney:IEEE，2013:633-640.
[13]Fu X，Huang J，Zeng D，et al.Removing rain from single images via a deep detail network［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE，2017:1715-1723.
[14]Zhang H，Sindagi V，Patel V M.Image deraining using a conditional generative adversarial network［J］.IEEE Transactions on Circuits and Systems for Video Technology，2019，30(11):3943-3956.
[15]Li S，Ren W，Zhang J，et al.Single image rain removal via a deep decomposition-composition network［J］.Computer Vision and Image Understanding，2019，186:48-57.
[16]Pan J，Liu S，Sun D，et al.Learning dual convolutional neural networks for low-level vision［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE，2018:3070-3079.
[17]Zhang H，Patel V M.Density-aware single image deraining using a multi-stream dense network［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE，2018:695-704.
[18]Wang Y T，Zhao X L，Jiang T X，et al.Rain streaks removal for single image via kernel-guided convolutional neural network［J］.IEEE Transactions on Neural Networks and Learning Systems，2020，32(8):3664-3676.
[19]Hu X，Fu C W，Zhu L，et al.Depth-attentional features for single-image rain removal［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE，2019:8022-8031.
[20]Yang W，Tan R T，Feng J，et al.Deep joint rain detection and removal from a single image［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Puerto Rico:IEEE，2017:1357-1366.
[21]Fu X，Liang B，Huang Y，et al.Lightweight pyramid networks for image deraining［J］.IEEE Transactions on Neural Networks and Learning Systems，2019，31(6):1794-1807.
[22]Li R，Cheong L F，Tan R T.Single image deraining using scale-aware multi-stage recurrent network［J］.arXiv preprint arXiv:1712.06830，2017.
[23]Li X，Wu J，Lin Z，et al.Recurrent squeeze-and-excitation context aggregation net for single image deraining［C］//Proceedings of the European Conference on Computer Vision(ECCV).Munich，2018:254-269.
[24]Ren D，Zuo W，Hu Q，et al.Progressive image deraining networks:a better and simpler baseline［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE，2019:3937-3946.
[25]Deng S，Wei M，Wang J，et al.Detail-recovery image deraining via context aggregation networks［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE，2020:14560-14569.
[26]Jiang K，Wang Z，Yi P，et al.Multi-scale progressive fusion network for single image deraining［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE，2020:8346-8355.
[27]Zhao J，Xie J，Xiong R，et al.Pyramid convolutional network for single image deraining［C］//CVPR Workshops.Long Beach，2019:9-16.
[28]Johnson J，Alahi A，Li F F.Perceptual losses for real-time style transfer and super-resolution［C］//European Conference on Computer Vision.Amsterdam，2016:694-711.