一种基于条件生成对抗网络的高感知图像压缩方法

doi:10.12068/j.issn.1005-3026.2022.06.004

摘要/Abstract

摘要： 针对如何获得符合人类视觉感知的压缩图像问题，提出了基于条件生成对抗网络的图像压缩模型(HPIC).在HPIC中，首先利用一个超先验概率模型对原始图像进行编码量化，将条件附加标签和残差模块相结合的生成器用于压缩图像的重建，基于深度卷积神经网络搭建的判别器则用于区分压缩后的图像和真实图像间的差异.损失函数是基于比特率-失真-感知优化理论来设计的，一方面选用基于预训练Inception网络特征值的感知失真指标来实现具有高感知质量的图像压缩重建，另一方面利用生成对抗网络损失来消除压缩伪影，提高压缩精度.实验结果表明，HPIC在比特率-失真-感知三重权衡中取得了较好的平衡，即使目前的常见算法使用两倍于本文算法的比特率，本文算法在所有的感知指标得分上均优于前者，HPIC仍能够实现具有高感知质量的压缩.

关键词: 图像压缩;比特率-失真-感知优化理论;条件生成对抗网络;损失函数

Abstract: A conditional generative adversarial network-based high perceptual image compression(HPIC) model was proposed to obtain a compressed image that conforms to human visual perception. In HPIC， the original image was encoded and quantized based on a hyper prior probability model. A generator based on the conditional additional label and residual module was used for the reconstruction of the compressed image， and a discriminator based on the convolutional neural network was used to distinguish the difference between the compressed image and the real image. Based on the bit rate-distortion-perception optimization theory to design the loss function， the perceptual distortion metric was chosen based on the eigenvalues of the pre-trained inception network to achieve image compression reconstruction with high perceptual quality on the one hand， and use the adversarial generation network loss to eliminate compression artifacts and improve the compression accuracy on the other hand. The experimental results show that HPIC achieves a good balance in the bit rate-distortion-perception tradeoff and outperforms the current common algorithms in all perceptual metrics scores， even though the latter uses twice as much bit rate as ours. HPIC can achieve compression with high perceptual quality.

Key words: image compression; bit rate-distortion-perception theory; conditional generative adversarial network; loss function

中图分类号:

TP301.6

张雪峰，许华文，杨棉子美. 一种基于条件生成对抗网络的高感知图像压缩方法[J]. 东北大学学报（自然科学版）, 2022, 43(6): 783-791.

ZHANG Xue-feng， XU Hua-wen， YANG Mian-zimei. High Perceptual Image Compression Based on Conditional GAN[J]. Journal of Northeastern University(Natural Science), 2022, 43(6): 783-791.

参考文献

[1]Wallace G K.The JPEG still picture compression standard［J］.IEEE Transactions on Consumer Electronics，1992，38(1):18-34.
[2]Rabbani M.JPEG2000:image compression fundamentals，standards and practice［M］.Berlin:Springer，2002.
[3]Developers G.Compression techniques ［EB/OL］.［2021-07-23］.https://developers.google.com/speed/webp/docs.
[4]Bellard F.BPG image format［EB/OL］.［2021-07-23］.https://bellard.org/bpg/.
[5]Sullivan G J，Ohm J R，Han W J，et al.Overview of the high efficiency video coding(HEVC)standard［J］.IEEE Transactions on Circuits and Systems for Video Technology，2012，22(12):1649-1668.
[6]Toderici G，O′Malley S M，Hwang S J，et al.Variable rate image compression with recurrent neural networks［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1511.06085v5.
[7]Toderici G，Vincent D，Johnston N，et al.Full resolution image compression with recurrent neural networks［C］// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Honolulu，2017:5306-5314.
[8]Ballé J，Laparra V，Simoncelli E P.End-to-end optimized image compression［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1611.01704.
[9]Theis L，Shi W，Cunningham A，et al.Lossy image compression with compressive autoencoders［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1703.00395.
[10]Agustsson E，Mentzer F，Tschannen M，et al.Soft-to-hard vector quantization for end-to-end learning compressible representations ［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1704.00648.
[11]Ballé J，Minnen D，Singh S，et al.Variational image compression with a scale hyperprior［EB/OL］.［2021-07-23］.http://export.arxiv.org/abs/1802.01436.
[12]Mentzer F，Agustsson E，Tschannen M，et al.Conditional probability models for deep image compression［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach，2018:4394-4402.
[13]Li M，Zuo W，Gu S，et al.Learning convolutional networks for content-weighted image compression［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach，2018:3214-3223.
[14]Rippel O，Bourdev L.Real-time adaptive image compression［C］// International Conference on Machine Learning.Sydney，2017:2922-2930.
[15]Minnen D，Ballé J，Toderici G.Joint autoregressive and hierarchical priors for learned image compression［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1809.02736.
[16]Goodfellow I J，Pouget Abadie J，Mirza M，et al.Generative adversarial networks［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1406.2661.
[17]王雪松，晁杰，程玉虎.基于自注意力生成对抗网络的图像超分辨率重建［J］.控制与决策，2021，36(6):1324-1332.(Wang Xue-song，Chao Jie，Cheng Yu-hu.Image super-resolution reconstruction based on self-attention GAN［J］.Control and Decision，2021，36(6):1324-1332.)
[18]Karras T，Laine S，Aila T.A style-based generator architecture for generative adversarial networks［C］// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach，2019:4401-4410.
[19]Park T，Liu M Y，Wang T C，et al.Semantic image synthesis with spatially-adaptive normalization［C］// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach，2019:2337-2346.
[20]邓廷权，盛春冬.结合变精度粗糙熵和遗传算法的图像阈值分割方法［J］.控制与决策，2011，26(7):1079-1082.(Deng Ting-quan，Sheng Chun-dong.Image threshold segmentation based on entropy of variable precision rough sets and genetic algorithm［J］.Control and Decision，2011，26(7):1079-1082.)
[21]张雪峰，闫慧.基于中值滤波和分数阶滤波的图像去噪与增强算法［J］.东北大学学报(自然科学版)，2020，41(4):482-487.(Zhang Xue-feng，Yan Hui.Image denoising and enhancement algorithm based on median filtering and fractional order filtering［J］.Journal of Northeastern University(Natural Science)，2020，41(4):482-487.)
[22]Zhu X，Zhang X，Zhang X Y，et al.A novel framework for semantic segmentation with generative adversarial network［J］.Journal of Visual Communication and Image Representation，2019，58:532-543.
[23]Vo D M，Nguyen D M，Le T P，et al.HI-GAN:a hierarchical generative adversarial network for blind denoising of real photographs［J］.Information Sciences，2021，570:225-240.
[24]Tschannen M，Agustsson E，Lucic M.Deep generative models for distribution-preserving lossy compression ［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1805.11057.
[25]Blau Y，Michaeli T.Rethinking lossy compression:the rate-distortion-perception tradeoff［C］// International Conference on Machine Learning.Sydney，2019:675-685.
[26]Agustsson E，Tschannen M，Mentzer F，et al.Generative adversarial networks for extreme learned image compression［C］// Proceedings of the IEEE/CVF Internat ional Conference on Computer Vision.Seoul， 2019:221-231.
[27]Mentzer F，Toderici G，Tschannen M，et al.High-fidelity generative image compression［EB/OL］.［2021-07-23］.https://arxiv.org/abs/2006.09965.
[28]Wang T C，Liu M Y，Zhu J Y，et al.High-resolution image synthesis and semantic manipulation with conditional gans［C］// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Long Beach，2018:8798-8807.
[29]Mirza M，Osindero S.Conditional generative adversarial nets［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1411.1784.
[30]Ding K，Ma K，Wang S，et al.Image quality assessment:unifying structure and texture similarity［EB/OL］.［2021-07-23］.https://arxiv.org/abs/2004.07728.［31］Zhang R，Isola P，Efros A A，et al.The unreasonable effectiveness of deep features as a perceptual metric［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach，2018:586-595.［32］Lin T Y，Maire M，Belongie S，et al.Microsoft COCO:common objects in context［C］// European Conference on Computer Vision.Zurich，2014:740-755.［33］Asuni N，Giachetti A.Testimages:a largescale archive for testing visual devices and basic image processing algorithms［C］// STAG:Smart Tools & Apps for Graphics.Cagliari，2014:6370.［34］Heusel M，Ramsauer H，Unterthiner T，et al.GANS trained by a two timescale update rule converge to a local Nash equilibrium ［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1706.08500.［35］Bińkowski M，Sutherland D J，Arbel M，et al.Demystifying MMD GANS［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1801.01401.