High Perceptual Image Compression Based on Conditional GAN

doi:10.12068/j.issn.1005-3026.2022.06.004

Abstract

Abstract: A conditional generative adversarial network-based high perceptual image compression(HPIC) model was proposed to obtain a compressed image that conforms to human visual perception. In HPIC， the original image was encoded and quantized based on a hyper prior probability model. A generator based on the conditional additional label and residual module was used for the reconstruction of the compressed image， and a discriminator based on the convolutional neural network was used to distinguish the difference between the compressed image and the real image. Based on the bit rate-distortion-perception optimization theory to design the loss function， the perceptual distortion metric was chosen based on the eigenvalues of the pre-trained inception network to achieve image compression reconstruction with high perceptual quality on the one hand， and use the adversarial generation network loss to eliminate compression artifacts and improve the compression accuracy on the other hand. The experimental results show that HPIC achieves a good balance in the bit rate-distortion-perception tradeoff and outperforms the current common algorithms in all perceptual metrics scores， even though the latter uses twice as much bit rate as ours. HPIC can achieve compression with high perceptual quality.

Key words: image compression; bit rate-distortion-perception theory; conditional generative adversarial network; loss function

CLC Number:

TP301.6

ZHANG Xue-feng， XU Hua-wen， YANG Mian-zimei. High Perceptual Image Compression Based on Conditional GAN[J]. Journal of Northeastern University(Natural Science), 2022, 43(6): 783-791.

References

[1]Wallace G K.The JPEG still picture compression standard［J］.IEEE Transactions on Consumer Electronics，1992，38(1):18-34.
[2]Rabbani M.JPEG2000:image compression fundamentals，standards and practice［M］.Berlin:Springer，2002.
[3]Developers G.Compression techniques ［EB/OL］.［2021-07-23］.https://developers.google.com/speed/webp/docs.
[4]Bellard F.BPG image format［EB/OL］.［2021-07-23］.https://bellard.org/bpg/.
[5]Sullivan G J，Ohm J R，Han W J，et al.Overview of the high efficiency video coding(HEVC)standard［J］.IEEE Transactions on Circuits and Systems for Video Technology，2012，22(12):1649-1668.
[6]Toderici G，O′Malley S M，Hwang S J，et al.Variable rate image compression with recurrent neural networks［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1511.06085v5.
[7]Toderici G，Vincent D，Johnston N，et al.Full resolution image compression with recurrent neural networks［C］// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Honolulu，2017:5306-5314.
[8]Ballé J，Laparra V，Simoncelli E P.End-to-end optimized image compression［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1611.01704.
[9]Theis L，Shi W，Cunningham A，et al.Lossy image compression with compressive autoencoders［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1703.00395.
[10]Agustsson E，Mentzer F，Tschannen M，et al.Soft-to-hard vector quantization for end-to-end learning compressible representations ［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1704.00648.
[11]Ballé J，Minnen D，Singh S，et al.Variational image compression with a scale hyperprior［EB/OL］.［2021-07-23］.http://export.arxiv.org/abs/1802.01436.
[12]Mentzer F，Agustsson E，Tschannen M，et al.Conditional probability models for deep image compression［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach，2018:4394-4402.
[13]Li M，Zuo W，Gu S，et al.Learning convolutional networks for content-weighted image compression［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach，2018:3214-3223.
[14]Rippel O，Bourdev L.Real-time adaptive image compression［C］// International Conference on Machine Learning.Sydney，2017:2922-2930.
[15]Minnen D，Ballé J，Toderici G.Joint autoregressive and hierarchical priors for learned image compression［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1809.02736.
[16]Goodfellow I J，Pouget Abadie J，Mirza M，et al.Generative adversarial networks［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1406.2661.
[17]王雪松，晁杰，程玉虎.基于自注意力生成对抗网络的图像超分辨率重建［J］.控制与决策，2021，36(6):1324-1332.(Wang Xue-song，Chao Jie，Cheng Yu-hu.Image super-resolution reconstruction based on self-attention GAN［J］.Control and Decision，2021，36(6):1324-1332.)
[18]Karras T，Laine S，Aila T.A style-based generator architecture for generative adversarial networks［C］// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach，2019:4401-4410.
[19]Park T，Liu M Y，Wang T C，et al.Semantic image synthesis with spatially-adaptive normalization［C］// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach，2019:2337-2346.
[20]邓廷权，盛春冬.结合变精度粗糙熵和遗传算法的图像阈值分割方法［J］.控制与决策，2011，26(7):1079-1082.(Deng Ting-quan，Sheng Chun-dong.Image threshold segmentation based on entropy of variable precision rough sets and genetic algorithm［J］.Control and Decision，2011，26(7):1079-1082.)
[21]张雪峰，闫慧.基于中值滤波和分数阶滤波的图像去噪与增强算法［J］.东北大学学报(自然科学版)，2020，41(4):482-487.(Zhang Xue-feng，Yan Hui.Image denoising and enhancement algorithm based on median filtering and fractional order filtering［J］.Journal of Northeastern University(Natural Science)，2020，41(4):482-487.)
[22]Zhu X，Zhang X，Zhang X Y，et al.A novel framework for semantic segmentation with generative adversarial network［J］.Journal of Visual Communication and Image Representation，2019，58:532-543.
[23]Vo D M，Nguyen D M，Le T P，et al.HI-GAN:a hierarchical generative adversarial network for blind denoising of real photographs［J］.Information Sciences，2021，570:225-240.
[24]Tschannen M，Agustsson E，Lucic M.Deep generative models for distribution-preserving lossy compression ［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1805.11057.
[25]Blau Y，Michaeli T.Rethinking lossy compression:the rate-distortion-perception tradeoff［C］// International Conference on Machine Learning.Sydney，2019:675-685.
[26]Agustsson E，Tschannen M，Mentzer F，et al.Generative adversarial networks for extreme learned image compression［C］// Proceedings of the IEEE/CVF Internat ional Conference on Computer Vision.Seoul， 2019:221-231.
[27]Mentzer F，Toderici G，Tschannen M，et al.High-fidelity generative image compression［EB/OL］.［2021-07-23］.https://arxiv.org/abs/2006.09965.
[28]Wang T C，Liu M Y，Zhu J Y，et al.High-resolution image synthesis and semantic manipulation with conditional gans［C］// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Long Beach，2018:8798-8807.
[29]Mirza M，Osindero S.Conditional generative adversarial nets［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1411.1784.
[30]Ding K，Ma K，Wang S，et al.Image quality assessment:unifying structure and texture similarity［EB/OL］.［2021-07-23］.https://arxiv.org/abs/2004.07728.［31］Zhang R，Isola P，Efros A A，et al.The unreasonable effectiveness of deep features as a perceptual metric［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach，2018:586-595.［32］Lin T Y，Maire M，Belongie S，et al.Microsoft COCO:common objects in context［C］// European Conference on Computer Vision.Zurich，2014:740-755.［33］Asuni N，Giachetti A.Testimages:a largescale archive for testing visual devices and basic image processing algorithms［C］// STAG:Smart Tools & Apps for Graphics.Cagliari，2014:6370.［34］Heusel M，Ramsauer H，Unterthiner T，et al.GANS trained by a two timescale update rule converge to a local Nash equilibrium ［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1706.08500.［35］Bińkowski M，Sutherland D J，Arbel M，et al.Demystifying MMD GANS［EB/OL］.［2021-07-23］.https://arxiv.org/abs/1801.01401.